Keeping price volumes from the technical team

[MAY-D-FAR-B-WIT-U]

Does anyone know the history behind the evaluation process of keeping price proposals from the technical evaluation team (TET)? The price team, generally contract specialist and contracting officer have access to both price and technical volumes, why not vice versa? 

I have always assumed I knew the answer to this question without asking more experienced contracting officer in my agency, unfortunately with almost 6 years as an 1102, I am now one of the more experienced KOs 🤦‍♀️. My assumption was this is done to prevent the technical team from consciously or subconsciously engaging in a trade-off analysis during the technical evaluation. Total evaluated price is generally an objective value and it's easy to see how seeing that bottom number could creep into the technical evaluation.

On the other hand, technical evaluations are very subjective and most KOs performing price analysis never bother to look at the technical volume. Most importantly the total evaluated price is an objective number with the exception of maybe a cost realism analysis where the team uses some subjective input to determine probable cost.

To give context, we have a source selection where there could be a benefit to allowing the TET access to the price proposal, but I want to make sure we have plans to mitigate any pitfall from allowing this approach. Two potential pitfalls I can think of is 1) The technical team performing tradeoff due to seeing the total price 2) The technical team heading down that cost-realism tunnel.

I appreciate all input.

[joel hoffman]

There is nothing wrong with the technical evaluators seeing the pricing information - - after they have completed the evaluation and rating. In my organizations, the concern has been that knowing the pricing might unfairly influence the technical evaluation and rating, due to preconceived notions. It’s more complex than that depending upon the scenario but that’s the short version. As for “history”, I was doing SS since about 1990 and that was a concern then. However, I shared the pricing info with the evaluation board members after the consensus evaluation and ratings were complete. It can often provide a broader perspective to the technical evaluation.

i also let the price evaluation team see the technical proposals, when cost engineers (estimators) were on that team for construction and design-build projects. Cant speak for services acquisitions, but the cost engineers are very good at correlating between the technical and cost or price proposals for construction and design-build.

I won’t apologize for my opinion that limiting the price evaluation to a KO or Contracting specialist - particularly for construction or design-build - doesn’t work very well. That is based upon my experience as a guest or oversight member of acquisition teams in other USACE Districts that I wasn’t personally in charge of. 

[Vern Edwards]

1 hour ago, MAY-D-FAR-B-WIT-U said:

Does anyone know the history behind the evaluation process of keeping price proposals from the technical evaluation team (TET)? The price team, generally contract specialist and contracting officer have access to both price and technical volumes, why not vice versa? 

It's an old practice. I encountered it in my first Air Force source selection, which was in 1975. The idea, as I recall, was that seeing prices might affect technical evaluators' assessments of technical proposals. I don't recall whether it was official policy or just a practice norm.

As for policy today, see FAR 15.305(a)(4).

[joel hoffman]

13 hours ago, Vern Edwards said:

As for policy today, see FAR 15.305(a)(4).

“(4)  Cost information. Cost information may be provided to members of the technical evaluation team in accordance with agency procedures.”

I recommend waiting until after the TET evaluations and all (at least initial) consensus evaluation/ratings have been completed, if agency procedures don’t prohibit sharing the price/cost information.

I have personally encountered TET members whose personal opinion/judgement would be influenced by knowing the prices, particularly comparing prices between the proposers. They acted as though they would favor certain firms with favorable pricing.

Other than the obvious problem of unequal treatment, they didn’t understand how initial pricing can be adjusted through the discussion process, which might also include adjusting the scope, spec requirements or other aspects of the solicitation to achieve the best value within the budget.

It is a lot easier to talk through the pricing and correlation to the technical proposal after the initial consensus evaluation/ratings are all completed.

NOTE that discussions were generally the norm for our construction and especially for design-build construction source selections. Less common for the service contract SS’s I was involved with. So the initial prices may well change anyway from a TET member’s perspective. I would tell them that they shouldn’t be unduly influenced by the initial pricing .

[here_2_help]

At a contractor, we were evaluating several offers for potential time-sensitive subcontract award. We did not have access to the pricing information. We identified the most technically advantageous offer. (We didn't have adjectival ratings, but we had to describe why that offer was better than the others.) It took the evaluators about a week. A couple of days later we were told that our choice had been dropped from the competition because they were too expensive. Next: which offer was Number Two, and why was it better than the other (remaining) offers?

What a waste of time.

Again, this was at a contractor so things are not as formal as with the government. But the evaluators agreed that we are not going to invest a lot of effort into the next evaluation until we have assurance that every offer we are evaluating has already been determined to be within the competitive range.

[MAY-D-FAR-B-WIT-U]

3 hours ago, joel hoffman said:

 

I recommend waiting until after the TET evaluations and all (at least initial) consensus evaluation/ratings have been completed, if agency procedures don’t prohibit sharing the price/cost information.

Thank Joel. That will be my strategy if we decide to execute this approach. My concern is some element of price realism could kick-in from the TET without fully understanding the business decisions that could affect lower pricing. An evaluator could assign different ratings or a higher risk to two offers with similar solution simply because they looked at the total price and made a judgment that the price is too low to execute the solution without diving into possible business/pricing decisions that could explain the low prices.

I also thing providing the proposed hours to execute the solution is what the TET really needs not the prices.

I agree there is still a lot of benefit if price volume is provided after initial consensus evaluation and ratings. Any question from the TET can be addressed via price volume discussions. Have you had a scenario where you adjusted initial ratings or added a strength/weakness/deficiency to the technical evaluation after the TET reviewed the price volume?

[MAY-D-FAR-B-WIT-U]

13 hours ago, Vern Edwards said:

It's an old practice. I encountered it in my first Air Force source selection, which was in 1975. The idea, as I recall, was that seeing prices might affect technical evaluators' assessments of technical proposals. I don't recall whether it was official policy or just a practice norm.

As for policy today, see FAR 15.305(a)(4).

Thank you Vern. Sounds a bit like the Pot Roast Principle but I think there is a consensus here that seeing prices might affect technical evaluators' assessments of technical proposals.

[joel hoffman]

1 hour ago, MAY-D-FAR-B-WIT-U said:

Have you had a scenario where you adjusted initial ratings or added a strength/weakness/deficiency to the technical evaluation after the TET reviewed the price volume?

No, there was never an adjustment to a technical rating to my recollection. But we may have been able to determine why a price was higher than others or even question some pricing that would be an input for discussions.

For example, seeming misalignment between prices and the technical approach or solution. Or a proposer  may have proposed self-performed some work which the estimators assumed would be subbed out, etc. 

[Sam101]

Lately I been instructing offerors/quoters in section L to state how many hours for each labor category they plan to use to perform the SOW as part if their technical approach section. For example:

Offeror A:

Project manager: 100 hours

Senior developer: 200 hours

Offeror B:

Project manager: 125 hours

Senior developer: 150 hours

First of all, this is helpful to evaluating technical approach because it shows the extent to which the offeror understands the requirement, if this information is only seen in the price volume then the technical team will only realize that thier outstanding rated offeror is proposing 1,000 hours for the project manager after they see the price volume... the number of hours is a technical thing just as much as a price thing, so I want to see that number in the technical approach.

Other times I also "release the budget" and tell quoters (never done this for an RFP) that "the estimated level of effort is $100,000.00" and I get quotes around that range give or take, I seen SOWs that unless you disclose the level of effort in terms of dollars or hours to the vendors that some will quote $500,000.00 and some will quote $5 million for the same exact SOW. In my case the IGCE was around $500,000.00 and the $5 million quote could have been avoided if I had stated in the RFQ somewhere that this SOW's level of effort is around $500K.

[Vern Edwards]

1 hour ago, Sam101 said:

First of all, this is helpful to evaluating technical approach because it shows the extent to which the offeror understands the requirement, if this information is only seen in the price volume then the technical team will only realize that thier outstanding rated offeror is proposing 1,000 hours for the project manager after they see the price volume... the number of hours is a technical thing just as much as a price thing, so I want to see that number in the technical approach.

@Sam101 I have three questions for you:

1. What kinds of things do you buy?

2. What do you mean by "approach"?

3. Is the offeror's description of its "approach" supposed to be a promise or set of promises or is it just information?

[Sam101]

4 hours ago, Vern Edwards said:

1. What kinds of things do you buy?

2. What do you mean by "approach"?

3. Is the offeror's description of its "approach" supposed to be a promise or set of promises or is it just information?

Hi Vern, 

1. I usually buy professional services NAICS Code 541611 - Administrative Management and General Management Consulting Services.

2. By approach I mean "what processes/methods will you use to accomplish the requirements of the SOW, to include the description of labor categories and hours for each labor categories that you plan to use." But the exact language will depend on the SOW of course.

3. It can be for information purposes just so that the government understands that the offeror can do the work and understands the requirement or if it's for complex services I'll include the language from the best parts of the offeror's approach in the award version of the SOW. The hours are just for information purposes though because 1,000.00 hours for project manager will be considered a weakness because it's too much level of effort for what the government needs, otherwise the evaluation panel will only find out when when see the price volume (if hours are broken out there).

Edited May 8, 2021 by Sam101
Last paragraph reworded for clarity.

[Vern Edwards]

@Sam101 Thank you Sam101. Your answers are clear (a rarity here at Wifcon Forum), and I appreciate them.

[formerfed]

9 hours ago, Sam101 said:

Other times I also "release the budget" and tell quoters (never done this for an RFP) that "the estimated level of effort is $100,000.00" and I get quotes around that range give or take, I seen SOWs that unless you disclose the level of effort in terms of dollars or hours to the vendors that some will quote $500,000.00 and some will quote $5 million for the same exact SOW. In my case the IGCE was around $500,000.00 and the $5 million quote could have been avoided if I had stated in the RFQ somewhere that this SOW's level of effort is around $500K.

I’m surprised when I hear so many people say their lawyers object to releasing budgets.  I tell them to go back and ask their lawyers to prove why it’s prohibited.  Offerors need some way to understand boundaries for their responses.  Sometimes budgets or at least a range is all you can do.  That’s especially true when a SOO is used.  

[ji20874]

I have done this for an RFP.  

The classic case is contracting for environmental impact statements.  If the program has a reasonable amount budgeted for an EIS, it is best to tell offerors -- it will help offerors scope the content of their proposals and will allow for the best possible selection.

If the budget estimate is too low, offerors will say so and the agency can adapt.  If the estimate is too high, many offerors will propose lower prices and/or lots of value.

In procurements like this, sharing the estimate even shapes the playing field -- some excellent companies will focus on high-end EISs while other excellent companies will focus on economical EISs.

I am not recommending always releasing the estimate (or a range within which the estimate falls).  But sometimes, it makes sense.  It is a business call, not a legal call.

[Vern Edwards]

 

On 5/8/2021 at 4:21 AM, Sam101 said:

Other times I also "release the budget" and tell quoters (never done this for an RFP) that "the estimated level of effort is $100,000.00" and I get quotes around that range give or take, I seen SOWs that unless you disclose the level of effort in terms of dollars or hours to the vendors that some will quote $500,000.00 and some will quote $5 million for the same exact SOW. In my case the IGCE was around $500,000.00 and the $5 million quote could have been avoided if I had stated in the RFQ somewhere that this SOW's level of effort is around $500K.

 

22 hours ago, ji20874 said:

If the budget estimate is too low, offerors will say so and the agency can adapt.  If the estimate is too high, many offerors will propose lower prices and/or lots of value.

Budget or estimate? They are not necessarily the same. A budget is how much money you have, which might be based on a cost estimate or on the amount you were given last year. An estimate is a prediction, perhaps a three or four year old one, of how much it will cost to achieve some end result. At least, that's what I take those words to mean.

While there is nothing inherently improper (as far as I know) about disclosing the government's estimate or budget, you should be very careful about doing it, especially in acquisitions for services. What are you saying to offerors? What's the message?

Why are you disclosing the budget or estimate? In order to "scope" the contract? If so, are you promising to stay within scope? Did the person or people who wrote the statement of work base it on the estimate or budget? Did they write it with the estimate, budget, and future costs in mind, or did they simply cut and paste from someone else's SOW for a similar project or from the last contract?

If the contractor bids to the disclosed budget or estimate, but the requiring activity didn't keep that budget or estimate in mind when writing the SOW, and doesn't keep it in mind when demanding service, and insists upon "full performance" in return for payment, you could end up with a claim for out-of-scope change, and a court or board might consider the estimate or budget when interpreting the SOW. The contractor's lawyer will almost certainly argue that it should.

Disclosing a budget or estimate in an RFP for a service is not the same as telling an auto dealer that you don't want to spend more than $45,000 for a pickup. There might be legal consequences after contract award. It's not a good idea for rookie acquisition teams or for agencies in which the requiring activity and the contracting office are not working very closely with one another and do not share a solid understanding of performance costs.

And it's an especially stupid thing to do, in my opinion, if you are going to award without discussions.

[Vern Edwards]

BTW, see FAR 36.203, Government estimate of construction costs, paragraph (c).

[Vern Edwards]

Some additional thoughts:

If you are thinking about revealing an estimate, are you going to reveal the constituent details, such as the amounts included for materials (including the bill of materials) and subcontracts, labor, other direct costs, indirect costs, and profit? Are you going to reveal the age of the estimate? Would you show how you allocate the estimate by SOW task? Or are you going to reveal just the bottom line?

If you are thinking about revealing a budget (funding), are you going to reveal the bases for its development and when it was developed? (Maybe two years or more before issuance of the solicitation.)

Are you going to include a caveat in the RFP to the effect that the government does not represent the estimate to be reliable or that the budget represents the anticipated cost of performance. Will you say that the government does not indemnify the successful offeror from injuries caused by government estimating errors or budget insufficiency? Will you say that the government takes no responsibility for estimating errors and omissions and budget shorfalls?

ji20874 said, "If the budget estimate is too low, offerors will say so and the agency can adapt."

Maybe. Hopefully. But are you going to provide a way for them to tell you that without the risk of being eliminated from the competition? Are you going to give them time to tell you before submission of proposals, or must they do so only with submission?

If you reveal the budget or estimate in the RFP, are you going to conduct discussions with offerors in the competitive range in order to go over the government's estimate or budget and their proposals and try to reconcile any differences and discuss uncertainties and risks and how they might be resolved during contract performance?

I'm writing these things so the people who come here to learn won't think it's as simple as just putting some dollar amount in a solicitation.

Don't get me wrong—I'm not opposed to the idea, but I think it must be done professionally.

[C Culham]

On 5/8/2021 at 2:22 PM, formerfed said:

I’m surprised when I hear so many people say their lawyers object to releasing budgets.  I tell them to go back and ask their lawyers to prove why it’s prohibited.  Offerors need some way to understand boundaries for their responses.  Sometimes budgets or at least a range is all you can do.  That’s especially true when a SOO is used.  

 

22 hours ago, ji20874 said:

The classic case is contracting for environmental impact statements.

 

Just now, Vern Edwards said:

If you are thinking about revealing an estimate,

 

Just now, Vern Edwards said:

If you are thinking about revealing a budget (funding)

I picked the quotes not to call each out but just to note the thoughts.   My two quick thoughts are 1) Who has the onus - contractor or government? and;  2) What can be divulged. 

The onus?  From the high view there is not much a person could not find out about an agencies budget and intended procurements.  Many agencies have public facing websites that will offer all kinds of information.  Likewise I suspect (little rusty here) that asking for agency acquisition plans is probably allowed and done so with little implication of FOIA.   Like "Can you send me your advance acquisition plan, please?"    When it comes to NEPA activities and processes they are very public and digging around can help find information as well. 

On this point I did some quick research and found a couple of representative references.   It took me like 2 minutes to find them and makes me wonder what I might find in taking 8 hours to do my market research as a contractor for a specific project I am interested in.

https://business.defense.gov/Small-Business/Acquisition-Forecasts/

 https://www.fs.usda.gov/wps/portal/fsinternet/cs/projects/mthood/landmanagement/projects?archive=1&sortby=4

For smaller projects I am reminded of FAR 5.101 and how agencies might be meeting the guidance.   I remember from the old days how I did it but not sure about today but I would darn sure ask.

The What?  Noting the above there is always the push pull of the warning not to release procurement specific information as well as the cautions noted in the thread so far.   But right or wrong here is how I see it.  What is out there already that a person could find in public facing documents released or available from an agency?   In my own experience as a CO I always pushed for (and was willing) to allow information by reference or specific wording information that was already in the public domain.   I know it changes with the type of contract but doing business with the government is risky and hopefully any contractor understands that no matter what or what is not included in a solicitation.

Overall I am in the camp of great interchanges of information with industry is a good practice.   Why?  I do it every day in my personal life when I am contemplating a large procurement.   I learn about "industry" and "industry" learns about me.   Communication of information is a good thing.   And this is a real throwback but a visit to "Myth Buster" memos of the Office of Federal Procurement Policy is always a good read even today!

[formerfed]

Quote

Overall I am in the camp of great interchanges of information with industry is a good practice.

I often hear comments to the effect that “the more industry understands about the government’s requirement, the better proposal responses will be.”  

[Vern Edwards]

2 hours ago, formerfed said:

I often hear comments to the effect that “the more industry understands about the government’s requirement, the better proposal responses will be.”

True in theory. Naive in reality.

[formerfed]

23 minutes ago, Vern Edwards said:

True in theory. Naive in reality.

Maybe.  But providing available information doesn’t hurt and isn’t difficult.  Then there are constant allegations that existing companies embedded in agency operations have an unfair advantage from the knowledge they have. This process might negate that.

[joel hoffman]

11 hours ago, Vern Edwards said:

BTW, see FAR 36.203, Government estimate of construction costs, paragraph (c).

See also this excerpt from 36.204. :

”... In no event shall the statement of magnitude disclose the Government’s estimate.”

For construction contracts, when the government provides the design, defining the scope of work, if basis of award will be lowest price(IFB) or price is heavily stressed in an RFP, there is often no need to identify the available budget. The budget may or may not be the same as the IGE.

However, if there is a best value (trade off) , where the government may allow or encourage better than minimum qualifications or minimum materials or building systems or equipment, controls, installed process equipment, etc. , it is advisable to provide a budget ceiling, This number wouldn’t necessarily equal an IGE, which should be based upon the stated minimum requirements. 

For design-build projects that include performance based criteria for much or some aspects of the design criteria, it is essential to identify the cost ceiling limit (based upon budget). The government will often also identify enhanced features that it desires if  they can be achieved within budget. But industry needs to know what budget limit they must design and construct within. Again, the IGE is simply an estimate of what the full scope at specified, minimum quality might cost. 

When we identified the ceiling limit, we also stated to the effect that offerors are not obligated to use the entire budget amount if they can provide  full scope and high quality for less than the ceiling amount. Competition will usually come into play, as well as the relative importance of price vs. non-price factors. 

[joel hoffman]

3 hours ago, formerfed said:

I often hear comments to the effect that “the more industry understands about the government’s requirement, the better proposal responses will be.”  

For construction and especially for design-build, this is generally true in my experience, perspective and observation of projects across the Corps. The Design-Build Institute of America (DBIA) also concurs with formerfed’s observation.

Edit: not only “the government’s requirement[s]”, including a budget ceiling but also “the government’s desires”

[Vern Edwards]

36 minutes ago, formerfed said:

Then there are constant allegations that existing companies embedded in agency operations have an unfair advantage from the knowledge they have. This process might negate that.

The process can't negate that. An incumbent's advantage is hard-earned.

 

[formerfed]

I meant companies doing work at an agency and not necessarily incumbents.  They get to better understand the agency culture, working environment, perspectives, details of mission needs, funding, etc. They are able to posture themselves to win over outside companies.