I remember reading Vern Edwards' Source Selection Book years ago, how he recommended that, for FAR 15 and other procurements as well, if an agency does Phased Approach, then Phase 1 should be a GO/NO-GO, meaning that the agency would eliminate only the technically unacceptable proposals. 

Well, what happens if the agency, using FAR 8.4, issues a solicitation that has Phase 1 and Phase 2.  Phase 1 includes only one factor: an Experience Factor.  The agency can assign only one of 3 adjectival ratings to a proposal: High Confidence, Some Confidence, and Low Confidence.  The solicitation states that there will be a "mandatory down-select" after Phase 1.  The agency rates Offeror X's proposal as Low Confidence.  Offeror X is eliminated from the competition and cannot move on to Phase 2.  Offeror X files a timely protest at GAO.  How do you think GAO would decide?

The way I see it, the agency in this hypothetical didn't put any "technical" factors in Phase 1, so how can the agency find any proposal "technically unacceptable"?  How is "Experience" a technical factor?  And how is "Experience" something that could be deemed "technically unacceptable" if the solicitation's worst adjectival rating category for Experience is "Low Confidence"?  Low Confidence is not No Confidence, see?

This hypothetical is very similar to Central Care, Inc., B-420959.13 et al. (Jan. 11, 2024).  GAO denied the protest, finding that it was ok for the agency to eliminate the "Low Confidence" protester after Phase 1.

 

 

9 hours ago, govt2310 said:

This hypothetical is very similar to Central Care, Inc., B-420959.13 et al. (Jan. 11, 2024).  GAO denied the protest, finding that it was ok for the agency to eliminate the "Low Confidence" protester after Phase 1.

If this I hypothetical is very similar to Central Care, why are you asking? The agency assigned the protestor’s experience factor low confidence , thus not one of the most highly rated phase 1 proposal submissions. 

Who says that “experience” isn’t a technical factor? It’s not a price factor. Experience has been a common technical (non-price ) factor. The Corps of Engineers used it for many years in hundreds of single phase and two-phase competitive acquisitions.

Past performance is also a technical factor, which some agencies have combined with experience for a “confidence” rating.

There are numerous protests denied for lack of or very little relevant experience. This topic has been discussed over the years in this Forum.

The point of two phase processes or multi-step evaluations  is to down select in phase (or step) 1 using discriminators, such as relevant experience. Probability of successful performance…

10 hours ago, govt2310 said:

The way I see it, the agency in this hypothetical didn't put any "technical" factors in Phase 1, so how can the agency find any proposal "technically unacceptable"?  How is "Experience" a technical factor?  And how is "Experience" something that could be deemed "technically unacceptable" if the solicitation's worst adjectival rating category for Experience is "Low Confidence"?  Low Confidence is not No Confidence, see?

This hypothetical is very similar to Central Care, Inc., B-420959.13 et al. (Jan. 11, 2024).  GAO denied the protest, finding that it was ok for the agency to eliminate the "Low Confidence" protester after Phase 1.

I don't totally follow the question.

FAR 8.4 doesn't use the term "technical factors," but it expressly allows agencies to consider "experience," which is what the agency did in Phase 1.  

Further, it appears that the agency clearly set out the ground rules in the terms of the RFQ.  The agency stated that this was a two-phased evaluation with a mandatory down-select.  The agency stated that Phase 1 would solely focus on experience, which would result in one of three adjectival ratings.  

It appears that the agency used the term "technically unacceptable" to mean that the offeror didn't have the level/quality of experience to move forward to Phase 2.  I don't see a meaningful protest ground here just because the protester nitpicks the semantics.

The time to have raised this protest was prior to the date set for the receipt of quotes.  Raising this argument now would likely result in a dismissal on timeliness grounds at the GAO. 

I don't understand the importance of Central Care decision to your hypothetical.   I get that they are both 8.4 competitions using a two-phase mandatory down-select.  Unlike your hypo, however, the Central Care case used two non-price evaluation factors in Phase 1 -- a "staffing plan/approach" and "corporate experience evaluation factors."  The protester contested the agencies evaluation and lost.  So how is that similar to your hypothetical?  How does that decision support your point?

To recap, I don't see the agency error here, and to the extent that any FSS holder had an issue with the terms of the RFQ, the time to have raised that was prior to the date set for the receipt of quotes.  

 

There is no requirement for a phase one evaluation to use only GO/NO-GO factors, meaning that the agency would eliminate only technically unacceptable proposals in that phase.  Rather, the phase one evaluation may be selective and subjective.

For example, if an agency wants to do oral presentations or needs to do technical testing, but only has capacity or appetite for such in-depth evaluations with a small number of offerors, a phase one may be used to narrow the field to the prospective offerors who are most likely to be successful.

Of course, the solicitation needs to be written appropriately.

govt2310, do you disagree with the finding in the case you cited?

12 hours ago, govt2310 said:

FAR 15

 

12 hours ago, govt2310 said:

Well, what happens if the agency, using FAR 8.4,

Just repeating.....they are different!

 

12 hours ago, govt2310 said:

How do you think GAO would decide?

In a post award scenario I believe that if the agency adhered to the evaluation criteria idenfifed in the RFQ GAO would not sustain the protest.   

Since the hypothetical is done using FAR 8.4, an easier approach and alternative to two-phases is conduct market research upfront.  Only solicit sources with known experience.

govt2310, since this is a “hypothetical” situation, did the solicitation state that the basis for the phase one “mandatory down-select” is “go/no-go” and describe a minimum  standard of acceptability (“go”)?

You said the circumstances were “very similar” to the Central Care, Inc. Decision.

 In that Decision, the solicitation stated that it was a best value, trade off process, with only the highest rated vendors in phase one invited to proceed with the submission of phase II quotations.  Those vendors that were not amongst the most highly rated would be eliminated from the competition. 

3 hours ago, formerfed said:

Since the hypothetical is done using FAR 8.4, an easier approach and alternative to two-phases is conduct market research upfront.  Only solicit sources with known experience.

Keeping in mind FAR 8.405-2 which may provide for further action beyond just market research depending on the specifics of the acquisition.

I

1 hour ago, C Culham said:

Keeping in mind FAR 8.405-2 which may provide for further action beyond just market research depending on the specifics of the acquisition.

Good point.  But if market research is done properly, there shouldn’t be surprises.  It’s pretty easy to ensure sufficient responses are received.

Quote

9 hours ago, joel hoffman said: govt2310, since this is a “hypothetical” situation, did the solicitation state that the basis for the phase one “mandatory down-select” is “go/no-go” and describe a minimum  standard of acceptability (“go”)? You said the circumstances were “very similar” to the Central Care, Inc. Decision.  In that Decision, the solicitation stated that it was a best value, trade off process, with only the highest rated vendors in phase one invited to proceed with the submission of phase II quotations.  Those vendors that were not amongst the most highly rated would be eliminated from the competition.    Ah, I see.  Sorry, my mistake.

 

 

12 hours ago, joel hoffman said:

govt2310, since this is a “hypothetical” situation, did the solicitation state that the basis for the phase one “mandatory down-select” is “go/no-go” and describe a minimum  standard of acceptability (“go”)?

For a go/no go, phased down-select, the process should describe the standard for the determination of the minimum for a go, since firms are going to invest their time and resources to respond to your action.

”Keep it Simple” doesn’t mean to say nothing at all.

But the bottom line with this method is that it might not result in a manageable number of vendors for phase 2, rather it would include all “qualified” vendors or at least those  vendors that the government has some stated minimum level of confidence in. 

If the object is limit the phase 2 participation to some manageable range or number, then one should at least explain the range or upper limit (maximum number) and that only the most highly rated vendors, based upon… (relevant experience? Past performance of that experience?,  confidence of successful performance, etc.?) would be invited to participate in phase 2.

“I’ll know it when I see it” level of simplicity isn’t fair to the  universe of a schedule vendors, in my opinion. 

 

 

 

 

On 1/21/2024 at 8:29 PM, govt2310 said:

I remember reading Vern Edwards' Source Selection Book years ago, how he recommended that, for FAR 15 and other procurements as well, if an agency does Phased Approach, then Phase 1 should be a GO/NO-GO, meaning that the agency would eliminate only the technically unacceptable proposals. 

@govt2310I cannot find where I supposedly said that an agency should adopt a Go/No-Go approach in the first phase of a phased evaluation. I have checked both editions of my book. I'm not saying I didn't say it, although I doubt it. But you should not attribute a statement to a person without being able to cite something to which others can refer to verify your assertion.

14 hours ago, Vern Edwards said:

@govt2310I cannot find where I supposedly said that an agency should adopt a Go/No-Go approach in the first phase of a phased evaluation. I have checked both editions of my book. I'm not saying I didn't say it, although I doubt it. But you should not attribute a statement to a person without being able to cite something to which others can refer to verify your assertion.

I apologize.  Vern did not say an agency should adopt a Go/No-Go approach in Phase 1 of a Phased Evaluation.  I was thinking of something Vern said in a 2015 thread on Wifcon.  Here is link.

On "page 2" of this thread, Vern wrote a comment on October 10, 2015:

FARmer:

Phased evaluation is simply progressive winnowing of the field of competitors through phased application of the evaluation factors.

The idea is to start with the easiest factors to evaluate -- generally: legal acceptability, experience, and bottom-line price -- so that by the time you reach more difficult factors to evaluate you won't have as much work to do. There is no reason for the concept to be controversial, except to the eternally blockheaded.

You must consider price in each phase before eliminating anyone. However, if your solicitation says that you are going to award without discussions, then you do not have to consider the price of any offeror whose proposal is ineligible for award as submitted. For example, you don't have to consider the price of any firm whose offer does not conform to any material term of the prospective contract as stated in the solicitation.

 

I think what I was thinking of is not a statement by Vern, but various legal decisions (GAO, courts) where an agency didn't evaluate/consider Price in Phase 1, the agency eliminated an offeror, and it was ok because the offeror was technically unacceptable.  It was ok to eliminate an offeror in Phase 1 without considering Price, based on a GO/NO-GO factor.

On 1/23/2024 at 10:44 PM, govt2310 said:

[Quoting Vern Edwards:] “Phased evaluation is simply progressive winnowing of the field of competitors through phased application of the evaluation factors.

The idea is to start with the easiest factors to evaluate -- generally: legal acceptability, experience, and bottom-line price -- so that by the time you reach more difficult factors to evaluate you won't have as much work to do. There is no reason for the concept to be controversial, except to the eternally blockheaded.”

Phased evaluation processes and multiple phased, down-select quote or proposal submission processes are different, distinct processes.

I believe that govt2310 was initially referring to a phased submission approach, not a phased evaluation approach. One must be careful not to confuse the two, separate processes

In my opinion, if preparing an offer/proposal or quote requires significant time, resources and expense, then requiring full proposals in a phased evaluation with the primary intention to simplify the government’s workload can be onerous and unfair to industry.

This is particularly onerous and aggravating in construction and even more so in design-build (D-B) competitions. Both require significant time, resources and expenditures to compete and prepare technical and price proposals.

A construction or D-B firm must first plan and organize a team, solicit subcontractors (who also have to invest time and resources to participate). There are significant cost estimating efforts required to develop price proposals for both construction and D-B competitions.

In D-B, there is significant effort and expense by both the prime and design firms/teams to develop preliminary designs and price proposals.

Fortunately, due to significant industry involvement (including Professor Ralph Nash, I believe), in conjunction with the government agencies then using the onerous DoD one step design-build approach, the two-phase D-B process (FAR 36.3) was adopted by Congress.

Phase 1 was specifically aimed at using a performance capability based short-list selection process, with no design development  or pricing requirements. Phase 1 was intended to have some similarity to the quality-based, Brooks Act A-E selection process.

The short-listed firms them compete for contract award(s) in phase 2.

Multiple Award Task Order, ID/IQ contracts can offer less expensive competitions for both industry and government for services or construction.

A relatively simple and economical two phase,  down-select task order process could be used where there are large numbers of contract holders (e.g., the various GSA schedules).

The referenced Central Care, Inc. GAO decision is an example of this approach.

Agency construction MATOCs should and Agency design-build MATOCs must purposely have small pools of contract holders to provide attractive, yet effective competition for task orders. Then, there should be no need to use multi-phase evaluations.

 

 

 

On 1/23/2024 at 8:44 PM, govt2310 said:

I apologize.  Vern did not say an agency should adopt a Go/No-Go approach in Phase 1 of a Phased Evaluation.  I was thinking of something Vern said in a 2015 thread on Wifcon.  Here is link.

@govt2310 Thank you. But I don't think I said it there, either. "Easiest factors" didn't mean Go/No go factors.