Eliminating Bid Protests

[formerfed]

One thing that really bothers me is there’s no individual accountability for contract award decisions.  I wonder how many sustained protests occur because no single individual questioned things?  It seems like everyone involved - CS/COs, PMs, technical specialists, technical evaluators, lawyers, reviewers including contract review boards, and others, just things slip by.  instead of critically examining items and highlighting problems, too often issues aren’t brought up.  

If errors occur and especially sustained protests happen, there’s nobody individually responsible.  When I started out as an intern decades ago, the contracting officer had personal accountability.  There were COs that had warrants removed for not taking prudent actions.  In some instances, they lost a grade because their job duties were classified at a lower level without a warrant.   Now blame is so diffused, accountability doesn’t exist.  

[Vern Edwards]

51 minutes ago, formerfed said:

One thing that really bothers me is there’s no individual accountability for contract award decisions.

See FARR 15.303. By the book, agency heads are responsible. When they delegate SSA to someone else, that person is responsible.

Maybe what you mean is that no one is held responsible. No one is reprimanded or punished for process mismanagement when something goes wrong. Avoiding responsibility for that for which you are responsible is a very old, very human thing. And superiors are reluctant to hold anyone responsible because they would be responsible for that person.

[Fara Fasat]

On 3/2/2024 at 7:57 AM, Vern Edwards said:

There is no evidence whatsoever that the current methods of contractor selection and contract award other than sealed bidding produce "best value."

Please note that I didn't assert that without protests the government would be getting less than best value. I said that if that were the case, the cost of eliminating protests should be balanced against it. You have determined that there is no evidence the government was stuck with second-best goods, so eliminating protests outweighs it.

In other words you have considered the reasons for the fence, determined that they don't justify the fence, and recommend tearing it down. Chesterton would approve.

 

[Vern Edwards]

2 hours ago, Fara Fasat said:

n other words you have considered the reasons for the fence, determined that they don't justify the fence, and recommend tearing it down.

Actually, I say that the fence is excessively costly to maintain and does more harm than good.

[Me_BOX_Me]

On 3/2/2024 at 9:57 AM, Vern Edwards said:

Conclusions about "value" are usually described in vague adjectival terms such as: Outstanding, Good, Acceptable, Marginal, or Unacceptable. See W. Edwards Deming's 1975 essay, "The Logic of Evaluation":

What on earth does Good mean when applied to different proposals of various content? What is the margin of difference between Good and Acceptable?

I cannot describe how frustrated engineers get on my SEBs when we have to make decisions based on these wholly-qualitative terms. It takes months and months of writing and re-writing to ensure all the analysis is consistent. The implementation varies from SEB to SEB. It's easily protestable, because the offerors have a much different perspective about what they wrote down. At the same time, it's difficult to win those protests because judges defer to SEBs on those qualitative assessments as long as they are consistent and followed the written RFP. Thanks for the essay link, btw. Hadn't read that one yet. 

On 3/2/2024 at 9:57 AM, Vern Edwards said:

Evaluation documentation is often poor and is often destroyed. At the end of performance no one compares what was described in the winning proposal to what was actually received and writes a comparative assessment. In many agencies the winning proposal is not seen again after the source selection.

And we have known for decades, since before WWII, that proposals are often based on desperate imagination and bull----. That knowledge is well documented.

This seems like the most actionable part of the dialogue. This could easily be a part of an integrated project team as they start market research and acquisition planning. Even better would be collecting this data throughout the contract. That would be true continuous improvement. Vern et al - It would be helpful for wifcon to facilitate sharing knowledge and tools to help professionals establish these initiatives within their agencies. Unfortunately for senior managers to sign on, something would either have to be very well developed already. My group likely has the capacity to take something like this on in the coming years. We have the centralized data, expertise, and willingness to try. 

That isn't the main focus of this discussion, but it's the one we can affect at the deckplate level. Everything else is for policy wonks, hopefully supported by professionals through data calls and surveys. 

[Don Mansfield]

8 hours ago, Me_BOX_Me said:

This seems like the most actionable part of the dialogue. This could easily be a part of an integrated project team as they start market research and acquisition planning. Even better would be collecting this data throughout the contract. That would be true continuous improvement. Vern et al - It would be helpful for wifcon to facilitate sharing knowledge and tools to help professionals establish these initiatives within their agencies. Unfortunately for senior managers to sign on, something would either have to be very well developed already. My group likely has the capacity to take something like this on in the coming years. We have the centralized data, expertise, and willingness to try.

What exactly are you requesting? Guidance on writing better evaluations?

[joel hoffman]

The KO should send the technical portions of the winning proposal applicable to contract performance to the contract admin office/team along with the consensus evaluation minus any ratings.

This is for construction**, design-build construction** and certain service contracts.

In addition, we identified any proposed betterments which exceeded the solicitation requirements and which were accepted and incorporated at award.**

—————————————————

Footnotes:

**We normally incorporate applicable portions of the technical proposal into the contract award for construction and design-build contracts. It usually corresponds to proposed key personnel, specific proposed material or equipment, building systems etc. for construction. For D-B, it also includes proposed design features.

There was an order of precedence clause. Not the FAR Order of Precedence clause, which is only applicable to and appropriate for the UCF format.

The Army Corps of Engineers uses the Construction Specification Institute, CSI format for construction and design-build construction contracts..

The CSI format is more suitable for Federal construction and design-build construction contracting than the UCF.

The UCF is suitable for service and supply contracting.

[C Culham]

1 hour ago, joel hoffman said:

The UCF is suitable for service and supply contracting.

Other than commercial product or service.

[joel hoffman]

17 minutes ago, C Culham said:

Other than commercial product or service.

Agreed.

[Me_BOX_Me]

10 hours ago, Don Mansfield said:

What exactly are you requesting? Guidance on writing better evaluations?

No. I was referring to this part: "At the end of performance no one compares what was described in the winning proposal to what was actually received and writes a comparative assessment." 

Just saying that what Vern has proposed hasn't been attempted, certainly not at scale. So socializing what that comparative assessment might look like would be helpful to people who are interested in improving outcomes or reform similar to what Vern suggests. 

None of what I quoted talked about how to write better evaluations, it was about what we do with those documents post-award, if anything. 

[formerfed]

57 minutes ago, Me_BOX_Me said:

No. I was referring to this part: "At the end of performance no one compares what was described in the winning proposal to what was actually received and writes a comparative assessment." 

There are situations where performance evolved into something different than what the government envisioned and what the contractor proposed.  And it was mutually beneficial as both parties gained insight and experience.

[Don Mansfield]

1 hour ago, Me_BOX_Me said:

No. I was referring to this part: "At the end of performance no one compares what was described in the winning proposal to what was actually received and writes a comparative assessment." 

Just saying that what Vern has proposed hasn't been attempted, certainly not at scale. So socializing what that comparative assessment might look like would be helpful to people who are interested in improving outcomes or reform similar to what Vern suggests. 

None of what I quoted talked about how to write better evaluations, it was about what we do with those documents post-award, if anything. 

I think there is a comparison between what was promised in the contract and what was delivered. That's the purpose of a CPAR. 

I think what Vern is referring to is the kind of BS you find in technical proposals (e.g., technical approach) that tends to weigh heavily, probably too heavily, in source selection decisions. If we were to compare that information to actual results, I think we would find two things. First, there is not much correlation between a proposed approach and the actual one used. Second, it doesn't matter that a contractor didn't use their proposed approach during contract performance. Contract performance depends more on the contractor's ability to adapt to changing circumstances than their adherence to a preconceived approach based on speculation of what the future will be. 

Hopefully, we would draw the conclusion that evaluating technical approaches is a waste of time. 

[Me_BOX_Me]

2 hours ago, Don Mansfield said:

I think there is a comparison between what was promised in the contract and what was delivered. That's the purpose of a CPAR. 

I think what Vern is referring to is the kind of BS you find in technical proposals (e.g., technical approach) that tends to weigh heavily, probably too heavily, in source selection decisions. If we were to compare that information to actual results, I think we would find two things. First, there is not much correlation between a proposed approach and the actual one used. Second, it doesn't matter that a contractor didn't use their proposed approach during contract performance. Contract performance depends more on the contractor's ability to adapt to changing circumstances than their adherence to a preconceived approach based on speculation of what the future will be. 

Hopefully, we would draw the conclusion that evaluating technical approaches is a waste of time. 

That's the conclusion my agency arrived at 5 years ago. Now we either do 1) "management approaches" that are so high level they might as well not exist, 2) a basic technical approach to contract transition, or 3) what I did on my last SEB which was to only evaluate Key Personnel and Past Performance. 

It has worked very well for us so far and I try and strip out the technical/management approach on each SEB I run. Most of the contracts are follow-on contracts anyway so there's nothing groundbreaking the agency wants/needs. Besides all the BS horse and pony show.

Still. There is valuable data in those evaluations and we would do well to mine it for what we can. CPAR evaluators rarely know what was in the proposal. They barely know what's in the PWS, especially as it changes over time. 

Since you need evidence that technical and management approaches are not correlated with outcomes, you have to do the comparative assessment. Otherwise senior leaders and source selection officials will continue to lean on their biases and insist on using their "expertise" to evaluate technical approaches. 

[Me_BOX_Me]

3 hours ago, formerfed said:

There are situations where performance evolved into something different than what the government envisioned and what the contractor proposed.  And it was mutually beneficial as both parties gained insight and experience.

This is very true on multi-year or multiple-year contracts. Understanding how it evolved (and why) would inform the acquisition team as they craft either a follow-on or other multi-year/multiple-year contracts. Gotta do that retrospective, though. 

[formerfed]

4 hours ago, Don Mansfield said:

Hopefully, we would draw the conclusion that evaluating technical approaches is a waste of time. 

Years ago, there were some saying experience and past performance might be all that’s needed for many acquisitions.  One very convincing advocate felt past performance done the easy way - sending surveys to offeror references (or “friends of the offeror) and use of CPARS was mostly a waster of time.  He went on saying the contracting team needs to find instances of offeror performance and personally question those customers to get meaningful data.

1 hour ago, Me_BOX_Me said:

That's the conclusion my agency arrived at 5 years ago. Now we either do 1) "management approaches" that are so high level they might as well not exist, 2) a basic technical approach to contract transition, or 3) what I did on my last SEB which was to only evaluate Key Personnel and Past Performance. 

It has worked very well for us so far and I try and strip out the technical/management approach on each SEB I run. Most of the contracts are follow-on contracts anyway so there's nothing groundbreaking the agency wants/needs. Besides all the BS horse and pony shows.

For some projects such as R&D or responding to a SOO type requirements statement, a technical approach or some description of the offerors way of addressing the need might be necessary if for nothing more than assessing feasibility. 

[joel hoffman]

For routine acquisitions, the Air Force has preferred using price and “past performance” as evaluation factors. The “past performance” incorporates recent, relevant experience as evaluation criteria.

13 hours ago, formerfed said:

Years ago, there were some saying experience and past performance might be all that’s needed for many acquisitions.  One very convincing advocate felt past performance done the easy way - sending surveys to offeror references (or “friends of the offeror) and use of CPARS was mostly a waster of time.  He went on saying the contracting team needs to find instances of offeror performance and personally question those customers to get meaningful data.

I agree with the “very convincing advocate” that “the east way[s] of having the contractor contact references or the government sending the survey to references is mostly a waste of time” and ineffective. 

My preference was to use standardized forms for prime, key subs (and for design-build, the design firm’s) project experience and having them provide customer references. We reserved the right to contact and interview the references, using a standardized question format to verify the claimed experience and claimed past performance quality for those recent, relevant projects..

We also used CPARs as a reference and reserved the right to consider other sources, including personal knowledge of project experience and performance.

i hate requiring references to fill out and return the information especially when it was repetitive for multiple acquisitions.

I found the best information was gained by TELEPHONICALLY INTERVIEWING the references. Never had any protests concerning such reference INTERVIEWS.

We kept the written record of interviews for future project SS references where necessary and applicable.

Of course, using documented, TELEPHONIC, ORAL COMMUNICATIONS isn’t “the easy way”…

[Vern Edwards]

On 3/6/2024 at 3:56 AM, joel hoffman said:

The UCF is suitable for service and supply contracting.

The UCF is one of the best things in the FAR. It was in the ASPR and FPR before FAR. It is an excellent way to organize the content of a solicitation and contract.

On 3/6/2024 at 5:15 AM, C Culham said:

Other than commercial product or service.

FAR 12.303 prescribes a format to be used "to the maximum extent practicable," but its use is not strictly mandatory, and some agencies now use the UCF to organize commercial item RFPs and contracts.

I recommend use of the UCF for large commercial item RFPs and contracts, because it works. Otherwise, RFPS for commercial items are sometimes badly organized, even chaotic, because contract specialists receive next to no training about organizing and writing RFPs and contracts, and important task.

[C Culham]

36 minutes ago, Vern Edwards said:

The UCF is one of the best things in the FAR. It was in the ASPR and FPR before FAR. It is an excellent way to organize the content of a solicitation and contract.

Agreed.  My recollection is that in the move its use was implied to be imperative to provide ease for contractors in viewing solicitations/contracts across all agencies so stuff was essentially in the same place.

 

39 minutes ago, Vern Edwards said:

FAR 12.303 prescribes a format to be used "to the maximum extent practicable," but its use is not strictly mandatory, and some agencies now use the UCF to organize commercial item RFPs and contracts.

And the format of FAR 12.303 is built upon (my terminology) in FAR clasue 52.212-4 paragraph (s) Order of Precedence.

 

42 minutes ago, Vern Edwards said:

I recommend use of the UCF for large commercial item RFPs and contracts, because it works. Otherwise, RFPS for commercial items are sometimes badly organized, even chaotic, because contract specialists receive next to no training about organizing and writing RFPs and contracts, and important task.

Even the small ones by my experience attempt UCF but  are organized poorly and provide in some cases of conflicts that could not/cannot be solved by a simple application of paragraph (s) of 52.212-4.  Consistency is the key and agencies should consider a waiver to paragraph (s) that provides for use of full UCF at the  discreation of the CO. 

I never understood why FAR 12 departed from UCF for commercial product/item as in my experience there is no real uniform contract format in the commercial market place.  I guess I could be wrong but the commercial terms and conditions I have viewed suggest no consistency across the commercial market place.

Overall I have always thought the UCF was good as well but it seems the "to the maximum..." is read as an imperative rather than discreationary.  

[Vern Edwards]

On 3/6/2024 at 1:03 PM, Me_BOX_Me said:

That's the conclusion my agency arrived at 5 years ago. Now we either do 1) "management approaches" that are so high level they might as well not exist, 2) a basic technical approach to contract transition, or 3) what I did on my last SEB which was to only evaluate Key Personnel and Past Performance. 

The use of the word approach in RFP statements of evaluation factors and proposal preparation instructions is quite old. It probably dates back to the design competitions conducted by the Army and Navy under the Air Corps Act of 1926 and was carried forward in the use of negotiated procurements after passage of the Armed Services Procurement Act of 1947. It originally referred to aircraft design approach (two-engine, like the B-25, or four, like the B-17), but thanks to cut-and-paste practice it is now used in all sorts of procurements and means who knows what. "Proposed [technical or management] approach" is a very common evaluation factor and proposal preparation instruction term.

The concept of management approach was adopted in the early 1960's, because preliminary designs were not good predictors of performance. The emphasis shifted to contractor capability. See Peck & Scherer, The Weapons Acquisition Process: An Economic Analysis, pp, 361 - 385..

I just downloaded from SAM.gov the following instruction in a combined synopsis/solicitation for human resources support services, RFP W91CRB-24-R-0014:

 

Quote

 

Technical: (Technical proposals shall not exceed 15 pages)

Offerors shall provide sufficient information for the Government to determine the ability of the Offeror to perform the requirements of the RFP. The Offeror's proposal shall describe in detail its technical approach to performing the requirements of the Performance Work Statement (PWS) in meeting the following:

Note:  The Technical Proposal shall not include any cost or pricing data.

Technical Approach 

The offeror’s proposal shall demonstrate a clear, concise, comprehensive understanding and methodology for the following specific tasks on part 5 of the PWS:

1. Development of a Plan of Action and Milestone (POA&M)
2. Management of Civilian and Army Personnel
3. Leadership Development
4. Management of HQDA workers compensation Program
5. Management of Service Contract Reporting

 

The PWS is 32 pages long (no table of contents). Offerors get 15 pages in which to address five topics, so that's an average of three pages per topic. The second topic is PWS Task 5.2.2, entitled, "Civilian and Military Personnel Management." It is takes up about three quarters of a page and consists of six subtasks. So the space available is about 1/2 page per subtask. Subtask 5.2.2.3 says:

Quote

Review and monitor HQDA organization Request for Personnel Action (RPA) as required using AutoNOA. This task includes validating RPAs against approved organization structures and in compliance with HQDA civilian strength management policy; advising the Director, Human Resources Management Directorate (HRMD), or HRMD Government personnel on appropriate action; and monitoring RPAs to ensure they are processed as approved. a) Provide monthly report on the RPAs reviewed when requested by the HRMD Director. (CDRL A010 – Monthly Report).

What are the offerors expected to say about that in 1/2 of a page?

Can you all see how utterly stupid this is?

 

Whose fault is this kind of thing?

THE CONTRACTING OFFICERS'! THEY ARE SUPPOSED TO BE THE "BUSINESS ADVISORS," THE SOURCE SELECTION EXPERTS. THEY ARE SUPPOSED TO BE THINKERS!

What does approach mean? What do the evaluators want to know? When you ask someone to describe their "approach" to some undertaking, what do you want them to describe? What specific questions do you want them to answer?

According to definition of approach in the Oxford English Dictionary (the OED--the 20-volume dictionary, not some "college" dictionary), definition 5b, approach means:

Quote

figurative. A way of considering or handling something, esp. a problem.

Most contract specialists will say, "We want them to describe how they plan to do the job?"

What does "how" mean? According to the OED it means:

Quote

In what way or manner? By what means?

Approach is not specific. Asking "how" is not being specific. What, specifically, do you want to know? What specific information will affect your selection of an offeror and acceptance of its offer?

 So the next time the "technical people" write a cut-and-paste proposal preparation instruction, sit down with them and say:

"What questions do you want answered?"

And then, instead of writing a "proposal preparation instruction", write a questionnaire. If you want page limits, write an appropriate limit for each question.

Think!

[REA'n Maker]

Just got my first protest, from someone not even in line for award. 

But that still means a CICA stay and hundreds of hours charged by lawyers and others on a protest that is going to ultimately fail.  The only one who wins is the pre-award incumbent (and the protestor's attorneys of course).

How about mediation as the first avenue of a protest? I realize GAO is a pseudo-mediator but they are still a bureaucratic government agency with all the attendant baggage and sclerotic timelines. 

Or, maybe a more formulaic ("hurdle") approach whereby protestors would have to demonstrate competitive prejudice, standing, etc., before they are even allowed to file a protest.

I do agree that the current system incentivizes protests and is wasteful and expensive.  It seems we are more focused on avoiding protests than providing sound business advice. 

 

[Vern Edwards]

25 minutes ago, C Culham said:

I never understood why FAR 12 departed from UCF for commercial product/item as in my experience there is no real uniform contract format in the commercial market place.  I guess I could be wrong but the commercial terms and conditions I have viewed suggest no consistency across the commercial market place.

The idea was to streamline the solicitation process by enabling combined synopsis/solicitation. The UCF did not lend itself to that process. However, the original concept of combined sysnopsis/solicitation has long been laid by the wayside.

[Vern Edwards]

2 minutes ago, REA'n Maker said:

How about mediation as the first avenue of a protest?

It takes time, and a protest afterward is still possible, even likely in hotly contests procurements.

[Vern Edwards]

5 minutes ago, REA'n Maker said:

I do agree that the current system incentivizes protests and is wasteful and expensive. 

I don't know that it "incentivizes" protests. The biggest problem is time and cost, both the cost of the protest process and the costs of delayed contract awards. Protests hamper agencies' ability to get what they need to do their job for the American people. The benefits of the protest system are largely theoretical, rather than established in fact.

[REA'n Maker]

2 hours ago, Vern Edwards said:

I don't know that it "incentivizes" protests.

Maybe I should limit that comment to the CICA stay, which is a huge incentive to file a protest regardless of merit.  The incumbent can make way more money continuing to bill for a month or two on a $100M contract than they will ever spend on a protest.  All they have to do is ask and viola, CICA stay, and the money keeps rolling in!

(I have to say; I got a good laugh out of Vern's post regarding approach. So true.  Everybody uses it as a key term in solicitations and evaluations, but the implication in common English is that to approach something is a precursor to actually doing something.  "OK, now that you've approached it, what do you plan to do when you actually get there?" 🤔😄🤣)