Proposal Evaluation Factors

[ndean]

In the "Source Selection Answer Book" second addition by Vern J. Edwards page 98. It recommends the use of three source selection factors.

1: Offer Acceptability

2. Risk - based on offeror experience and past performance

3. Price

I have two questions.

One has anyone used these in a source selection and if so could you please provide your experience with these evaluation factors?

Second with regard to cost realism did you experience any concerns with completing a cost realism analysis or do you perceive any concerns with regard to completing cost realism based on these factors without receiving a full up technical proposal?

[napolik]

I have used these factors with success to make more efficient, less onerous source selections. In my opinion, the relevance of experience and the quality of past performance are the best indicators of a contractor’s future performance. However, customers oftentimes wish to maximize the number of factors and the volume of paper making up the non-price proposals.

Customers frequently want résumés, management plans, staffing plans, and descriptions of technical approaches. They wrongly associate the skills of a proposal writer with the capabilities of a company to perform a contract. If you are buying a commercial item, this additional information does not help to predict performance. Moreover, I believe there is a direct correlation between the volume of proposal information received and the likelihood of errors when assessing the information, conducting negotiations and in making competitive range and source selection judgments.

If you are not buying commercial items, there can be circumstances when you might want information beyond experience and past performance. This is particularly true for R&D. If you are in one of these circumstances, minimize the number of factors and the amount of information you solicit.

With regard to price or cost realism, you may wish to obtain a sheet demonstrating the types and volumes of labor and material that you will compare to your government estimate or to the cost build-ups of other contractors.

[Don Mansfield]

I've used those factors in source selection, too. I agree with everything napolik wrote.

[joel hoffman]

How one evaluates "offer acceptability" and "price" depends upon the context of the acquisition and your objectives.

What are you planning to purchase? Is it services, supplies, management or consulting expertise, construction, design-build construction, etc. Since you are considering evaluating cost realism, is this going to be cost reimbursable type contract? If to be fixed price, why would you feel it necessary to evaluate cost realism?

What you will evaluate depends upon what you are buying as well as your objectives for the acquisition. Would you evaluate acceptability of a product or service or are there variations of what can be provided that may warranted paying more for additional value?

With regard to your questions on cost realism, what is the purpose of the analysis?

In my opinion, without knowing the purpose and type of acquisition, any answers to your specific questions might or might not be be relevant to your situation. An out of context answer to your broad questions, may cause more harm than good.

[Guest Vern Edwards]

My writing reflected my thinking that many RFPs contain a model contract that reflects the deal that the government wants to make, usually missing only the name of the contractor and a price. What the government should do is seek promises (offers) to do what the model contract says at a proposed price. Evaluation of "offer acceptability" is to be simply a matter of pass/fail verification that the offeror has promised to do that, taken no exceptions, and proposed no alternatives, and has proposed a price. The evaluation of risk is to be based on (1) experience, (2) past performance, and any other considerations (e.g., key personnel qualifications) that would indicate how likely it is that the offeror will keep the promises in its offer. The decisional tradeoff is to be between risk and price.

Such an evaluation can be conducted in a very short period of time depending on the number of offers received and the number of factors subsumed under risk. The procedure is especially useful in the acquisition of services. The acquisition of supples might be more complicated. Based on a review of GAO decisions, the method has been used in some form or other by many agencies. I am not aware of any particular legal problems associated with it. napolik's observations about the wishes of customers are valid, and only a knowledgeable, competent CO can allay such customer concerns.

As for cost realism, the technical information needed to make that analysis can be obtained as part of the cost proposal and need not be detailed or voluminous. The analysis itself is not especially difficult, but an idiot organization can screw up anything.

Finally, no source selection should be conducted by a CO who finds any of this mystifying or difficult to understand.

I no longer teach source selection, explain these things at length, or discuss it, so please do not write to me about it. I have given up on the idea of simplifying (streamlining) source selection. For that reason, I have (thus far) declined to produce a new edition of my book. Sooner or later the publisher will stop selling it or will hire someone else to adapt it in light of regulatory changes.

[ndean]

Acquisition Type:

Cost Reimbursement Contract - Non-Completion Effort – The services would be best described as a “Term” effort or severable services.

[Msutherland]

I have used these factors for several years. My projects are mainly road and building repair. The vendors are typically small construction firms. They are not essay writers. This approach makes it easy for vendors to develop a proposal.

I use cost realism as an evaluation factor. I have worked with larger construction firms that wanted a project to keep their crews busy between other jobs, but typically if I receive an unrealistically low price it is because the offeror missed something.