Protest and discussions?

[onliberty]

ji20874,

Yeah it came through a little confusing. It would have been more clear if the 3:01 posting was first, followed by the 12:58. The reason we strive so hard to communicate clearly, keep offerors informed, be transparent and prepare well throughout the entire process is because of the limitations described in the 3:01 posting.

Because we can't divulge other offerors' ratings, it becomes that much more important that we freely divulge what we can release, but have typically been hesitant to - like the government estimate for example. The key, I think, is knowing exactly where the line is between what we can release and what we can't. Then, release everything on the releasable side of the line. Don't be cryptic or coy. Just tell them what you can tell them, and explain that you can't release ratings of any other offeror. They'll recognize you're being straight-up and they'll appreciate it.

The way I look at it, we should be preparing for debriefings from day one of the acquisition. If we wait until the end to start preparing, we've missed a thousand opportunities along the way to demonstrate that we're sincere, transparent, and we pay attention to detail. If the losing offeror has no idea what we've done along the way or how he's been evaluated, of course he's going to be suspicious that he's been evaluated incorrectly. So we nip that in the bud from the very beginning.

If the offeror has confidence in the process and the Government team evaluating them, I've found that they're not nearly as preoccupied with the other offerors' ratings anyway.

[onliberty]

Vern,

I fully understand your point and your argument. And really, I think we agree on just about all of it. I'm no apologist for GAO and I get that their opinions are advisory only (although it's a rare day that anyone has enough top-cover to go against them). And I totally agree about them being the author of the very confusion they claim they're trying to clear up. It would almost be funny if it weren't so dang annoying.

I was speaking forest-level, you were speaking tree-level, but I think we actually have the same opinions. You just seem to prefer more precise semantics. I agree words and definitions are important, so we need to use them correctly. I was just trying to make a larger point at the moment so I may have been guilty of using imprecise semantics.

Perhaps I misled you, but I also think you missed my main point. I never "[Referred] to the GAO's statement about the purpose of debriefings as if it had some legal force." In fact, I explicitly stated the opposite. I closed that post by saying, "Obviously there are very few hard and fast rules and each agency/office may do things differently, but I've found this works well for what we do."

Regardless, I've thoroughly enjoyed this debate and have learned a lot from it, so thank you!

[Don Mansfield]

Because we can't divulge other offerors' ratings, it becomes that much more important that we freely divulge what we can release, but have typically been hesitant to - like the government estimate for example. The key, I think, is knowing exactly where the line is between what we can release and what we can't. Then, release everything on the releasable side of the line. Don't be cryptic or coy. Just tell them what you can tell them, and explain that you can't release ratings of any other offeror. They'll recognize you're being straight-up and they'll appreciate it.

onliberty,

What would be your explanation of why you can't release the ratings of any other offeror?

[onliberty]

Don,

I would say for a few related reasons:

1) The losing offeror wasn't rated against other offerors. They were rated against the evaluation criteria. So we debrief them on how they fared against the evaluation criteria.

2) The FAR prohibits a point-by-point comparison of proposals. Revealing other offerors' ratings could very easily digress into comparing and/or debating features of proposals.

3) And similarly, the FAR prohibits revealing protected or privileged information, some of which could be discerned from other offerors' ratings (for example, quality ratings).

[Don Mansfield]

onliberty,

None of those reasons supports the assertion that you can't release the ratings of other offerors. They are reasons why you prefer not to. If you don't want to release such information, that's fine. But I wouldn't tell an offeror that I could not--as if I were prohibited from doing so by the FAR. Release of other offerors' ratings does not, per se, run afoul of FAR 15.506(e).

[Retreadfed]

onliberty, I agree with Don. Also, from what you have written previously, it seems you are not providing disappointed offerors with the technical rating of the winner or the overall ranking of offerors when this is called for by FAR 15.506(d). Also, how do you handle questions relating to whether you complied with the selection criteria relating to the solicitation?

[Guest Vern Edwards]

onliberty, Don, Retread:

You all are not being very clear about what you mean by "ratings" and "technical rating." See FAR 3.104-4:

(a) Except as specifically provided for in this subsection, no person or other entity may disclose contractor bid or proposal information or source selection information to any person other than a person authorized, in accordance with applicable agency regulations or procedures, by the agency head or the contracting officer to receive such information.

( b ) Contractor bid or proposal information and source selection information must be protected from unauthorized disclosure in accordance with 14.401, 15.207, applicable law, and agency regulations.

Emphasis added. Unlike FAR 3.104-3, which describes statutory restrictions, FAR 3.104-4 does not just limit disclosure during the period before award. It appears to establish restrictions that go beyond those in the Procurement Integrity statute.

Now see the definitiion of "source selection information" in FAR 2.101:

“Source selection information” means any of the following information that is prepared for use by an agency for the purpose of evaluating a bid or proposal to enter into an agency procurement contract, if that information has not been previously made available to the public or disclosed publicly:

(1) Bid prices submitted in response to an agency invitation for bids, or lists of those bid prices before bid opening.

(2) Proposed costs or prices submitted in response to an agency solicitation, or lists of those proposed costs or prices.

(3) Source selection plans.

(4) Technical evaluation plans.

(5) Technical evaluations of proposals.

(6) Cost or price evaluations of proposals.

(7) Competitive range determinations that identify proposals that have a reasonable chance of being selected for award of a contract.

(8) Rankings of bids, proposals, or competitors.

(9) Reports and evaluations of source selection panels, boards, or advisory councils.

(10) Other information marked as “Source Selection Information-See FAR 2.101 and 3.104” based on a case-by-case determination by the head of the agency or the contracting officer, that its disclosure would jeopardize the integrity or successful completion of the Federal agency procurement to which the information relates.

Emphasis added. See also FAR 45.1503(d):

Agency evaluations of contractor performance, including both negative and positive evaluations, prepared under this subpart shall be provided to the contractor as soon as practicable after completion of the evaluation... The completed evaluation shall not be released to other than Government personnel and the contractor whose performance is being evaluated during the period the information may be used to provide source selection information. Disclosure of such information could cause harm both to the commercial interest of the Government and to the competitive position of the contractor being evaluated as well as impede the efficiency of Government operations.

Now, what "ratings" would you disclose?

Summary proposal ratings, such as "Total score = 100" or "Overall Acceptable".

Factor ratings, such as "Technical = 72" or "Acceptable"; "Past Performance = Relevant/Limited Confidence"?

Subfactor ratings?

Lower level ratings?

Doesn't the disclosure of ratings reveal information in reports and evaluations of boards, etc.? Mightn't it harm competitive position, depending on what a company could discern through analysis of the RFP and the ratings?

Whom may the CO authorize to receive information about ratings? FAR 3.104-4(a) does not say, but see, e.g., the Department of State FAR Supp. at 603.104-4:

(a) The following classes of persons may be authorized to receive contractor bid or proposal information or source selection information by the contracting officer or head of the contracting activity, when such access is necessary to the conduct of an acquisition:

(1) Individuals involved in the selection process, such as the Contracting Officer's Representative, technical evaluators, advisors, consultants, and the Source Selection Official;

(2) Clerical personnel directly involved in the acquisition;

(3) Supervisors in the contracting officer's chain of command;

(4) Contracting personnel involved in reviewing or approving the solicitation, contract, or contract modification;

(5) Individuals from offices who may be required to perform pre-award audits, such as DCAA; and,

(6) Personnel in the following offices: Office of Small and Disadvantaged Business Utilization (OSDBU), Office of the Legal Adviser, Office of Legislative Affairs, Office of the Inspector General, Office of the Procurement Executive, the Small Business Administration, and the Office of Federal Contract Compliance Programs (Department of Labor).

I don't see unsuccessful offerors (or "losers") among those listed.

The point I making is only that you cannot rely just on FAR 15.506 when determining what you can or cannot disclose. You have to research and think more deeply about it. Your determination might be different in different acquisitions. A lot might depend on how much the RFP revealed about the evaluation of factors, subfactors, and below. A rating might reveal more than you think. Having said that, I favor disclosure of offeror factor-level ratings, and I have done it. But you have to be careful that such disclosure will not reveal too much.

You have to assure the offeror being debriefed that you evaluated its proposal properly, and the debriefing should focus on that goal. I don't think you can disclose enough information to assure an offeror that you complied with the selection criteria with respect to every other proposal.

[Weno2]

Actual cite is FAR 42.1503, Procedures, paragraph (d). Probably an 04:43 am typo.

[onliberty]

Retreadfed,

I agree with Don too.

I'm not sure if you've read the entire thread, but if you do you will find the context. Somebody asked what our process was for debriefings. I explained the way I do it and the rationale for why. That's it. I never stated anything was mandatory or law or required or even preferred.

I don't know what agencies people are from, but everything I'm saying is straight out of the FAR, the DoD Source Selection Procedures, GAO case precedent, and my agency's interpretation and implementation of those documents. Nothing novel. Nothing unique. If your agency (or anybody else's agency) interprets and implements them differently, great. I very highly recommend that you follow your agency procedures.

Also earlier in the thread, I stated I provide everything in FAR 15.506, which includes the technical rating of the successful offeror only and any rankings, if any were developed (this is pretty rare). None of the ratings of the other offerors are provided. The FAR explicitly says that the debriefed offeror gets only their own past performance ratings. If you do LPTA or PPT, there's not much else to give them. I'm guessing 90% of the readers of this forum, if they ever do a source selection, will do it either LPTA or PPT.

As for questions of whether or not we complied with the evaluation criteria, if it's LPTA it will be apparent by the price and technical rating of the winning offeror. If it's a tradeoff, regardless if their price is higher or lower than the winning offeror, we explain that any potential tradeoff either was or was not enough to overcome the price premium.

[onliberty]

Vern,

Yes, thanks. There is a lot that goes into it. We as COs have to balance giving the losing offeror as much information as we can with also protecting the proprietary info of the other offerors. My agency chooses to strike that balance by following the processes I've described in my previous posts. If other agencies strike the balance differently, good for them. Regardless, the way we do things today might change by tomorrow.

[Don Mansfield]

onliberty,

Now that you mentioned the DoD Source Selection Procedures, I see where you've been coming from. Appendix B (Debriefing Guide), Section B.1 (Purpose of Debriefing) contains the statement: "The debriefing also provides feedback to offerors to assist in improving future proposal submissions."

Section B.9 (Sample Offeror Questions That May Be Used for “Dry Run”) contains the following "note":

Please provide the evaluated cost or price and technical, management, and past performance ratings for our proposal and all other offerors.

NOTE: Information on the overall evaluated cost or price and technical ratings is not provided for all offerors; only for the successful offeror and the offeror being debriefed.

This note, as well as some other notes in the guide, seems to be based on the authors' preconceived notion of debriefings rather than official DoD policy.

[joel hoffman]

onliberty,

Now that you mentioned the DoD Source Selection Procedures, I see where you've been coming from. Appendix B (Debriefing Guide), Section B.1 (Purpose of Debriefing) contains the statement: "The debriefing also provides feedback to offerors to assist in improving future proposal submissions."

Section B.9 (Sample Offeror Questions That May Be Used for “Dry Run”) contains the following "note":

This note, as well as some other notes in the guide, seems to be based on the authors' preconceived notion of debriefings rather than official DoD policy.

Don, what is official DoD policy regarding debriefings other than what is in the DOD Source Selection Procedures, which are mandatory under the Procedures, Guidance and Information (PGI) Web page? I do realize that use of the debriefing guide in Appendix B is "encouraged" but would seem to be consistent with "DoD policy."

PGI 215.300—Scope of subpart.

See the policy tab for Director, Defense Procurement and Acquisition Policy memorandum dated March 4, 2011, Department of Defense Source Selection Procedures, which provides the procedures to be used within DoD when conducting negotiated, competitive acquisitions utilizing FAR part 15 procedures.

The DPAP Director's MAR 4, 2011 Policy memo is entitled : "Department of Defense Source Selection Procedures" with attached Source Selection Procedures at http://www.acq.osd.mil/dpap/policy/policyvault/USA007183-10-DPAP.pdf

The Memo says the "attached document provides the procedures to be used within the Department when conducting negotiated, competitive acquisitions utilizing FAR Part 15 procedures." It also says that they effective July 1, 2011 "and are mandatory for all competitive acquisitions utilizing FAR Part 15 procedures."

Para. 3.10 D 3.10 Debriefngs

The PCO shall ensure offerors are debriefed, if requested, in accordance with FAR 15.505 and FAR 15.506, as applicable. The PCO shall document the debriefing(s) provided to offeror(s). At the request of the PCO, members of the SST shall participate in debriefings to offerors. The PCO is encouraged to use the debriefing guide provided in Appendix B.

[Don Mansfield]

joel,

I think the official DoD policy on debriefings is in FAR 15.505 & FAR 15.506, DFARS 215.506, and paragraph 3.10 of the Source Selection Procedures, which you quoted above. I don't read the "notes" in the "Sample Offeror Questions That May Be Used for “Dry Run” in a nonmandatory debriefing guide as an official statement of policy.

[joel hoffman]

joel,

I think the official DoD policy on debriefings is in FAR 15.505 & FAR 15.506, DFARS 215.506, and paragraph 3.10 of the Source Selection Procedures, which you quoted above. I don't read the "notes" in the "Sample Offeror Questions That May Be Used for “Dry Run” in a nonmandatory debriefing guide as an official statement of policy.

Yes, but they should be consistent with the DoD Policies that you and I identified above. They are part of the document that the Source Selection Joint Analysis Team (JAT) developed for the Undersecretary of Defense for Acquisition, Technology and Logisitics with the objective to standardize the methodology and approach that the DOD uses to conduct negotiated competitively negotiated source selections.

If onliberty is conducting discussions consistent with the Source Selection Procedures - including the Appendix B - then he/she should be in compliance with Department policy.

[Don Mansfield]

If onliberty is conducting discussions consistent with the Source Selection Procedures - including the Appendix B - then he/she should be in compliance with Department policy.

No argument here.