Carl, I agree with you. 

12 hours ago, C Culham said:

The given scenerio states that the "most probable cost is flawless' based on a cost realism analysis so tell me what else analytically is there to evaluate?  Seriously!!!!

Don already explained the parameters for both the cost and non-cost factor evaluations and comparisons in this scenario. There was no need for further thoughtful and professional in-depth analysis for this scenario. 

@joel hoffman @C Culham

To say that the MPC analysis was "flawless" means to me that there were no mistakes. But that is not the only analysis to be made of the data reported. Any elementary knowledge of cost estimating would inform a decision maker that there are some questions to ask about the numbers, including a very significant one.

But if you don't know anything about that, and about probability, elementary statistics, and risk analysis, then I don't blame you for your answers.

But, wow: "There was no need for further thoughtful and professional in-depth analysis for this scenario."

Jeez.

https://www.wifcon.com/articles/Edwards, Vernon - Lifelong Learning, Cultivated Curiousity and Self-Interrogation.pdf

Quote

But at the core of every professional life, there should be the haunting expectation that there is more to know and a lifelong drive to learn it so we can do more and do it better. We must never tire of interrogating ourselves, being honest with ourselves about what we really know, inquiring about what seems familiar, and seeking answers to a never-ending stream of thoughtful, carefully crafted, incisive questions. Not “quick” questions directed at colleagues, but tough questions, directed at ourselves.

 

21 hours ago, C Culham said:

The given scenerio states that the "most probable cost is flawless' based on a cost realism analysis so tell me what else analytically is there to evaluate?

While I appreciate Josh's response  I would suggest very strongly that if he were the CO that conducted the cost realism as the scenerio states then I would hope it would include a narrative that would explain its flawless effort.  Such a narrative would be available for the "they" whomever they are and would be the only thing needed to answer why the cost realism was flawless. 

I agree with your suggestion that the contracting officer's determination of the most probable cost is flawless means that a fulsome cost realism analysis was done and is sufficiently documented, to include the reasons for the probable cost adjustments.

My point is the scenario only provides the most probable cost for each offeror. I assert that there are other aspects of the cost realism analysis that, while fully captured in the contracting officer's file, are nevertheless not being considered by the notional contracting officer in their award decision, based on that information being omitted from the scenario. At least that's how I read the scenario. Do they have enough information to determine the best value? I'd argue no, based on what is provided.

On 3/16/2025 at 2:27 AM, Don Mansfield said:

1. Assume Offeror A's adjustment was due to a few dollars apiece over to various labor rates over five years and Offeror B's adjustment is due to a single cost element critical to performance that they forgot to account for. Now what is your answer?

2. If you assume the opposite, does your answer change?

Assuming no variables exogenous to the scenario (e.g., a nonresponsibility decision), Offeror A would likely be best value under the first situation. The cost realism concern I would have would be with Offeror B, whose offer already constitutes the higher probable cost. Factoring in this cost realism concern (the omission of a large, critical cost element) would only serve to weaken Offeror B.

Under the second situation, I think it gets more interesting. Formation of Government Contracts, 4th Ed., discusses the downgrading of an offeror's proposal as the result of the Government's probable cost of performance being significantly higher than the offeror's estimate (p. 1530). How and whether that could or should happen here would, I imagine, depend on the specific facts.

The assertion that the cost realism analysis was flawless has no bearing on the answer to Don's questions, which were:

Quote

1. Assuming the contracting officer's determination of the most probable cost is flawless, do they have enough information to determine the best value?

2. If not, what additional information should they consider?

It should be obvious what additional information an intelligent decision maker would seek.

I hope DOGE isn't reading this thread.

I addressed those questions earlier. 

Edit: I’ll let DOGE know when the answer to the quiz is revealed. 

17 minutes ago, joel hoffman said:

I addressed those questions earlier. 

@joel hoffman What questions? These?

Quote

Missing information here might be past performance and experience track records of the competing firms for completing similar work and for controlling costs on CR contracts,  what type of CR contract and details concerning the proposed fees. Are the “proposed costs” and “most probable costs” inclusive of fees? 

Nope.

Come on! You're an engineer!

I'm as innumerate as they come, positively stupid, and even I would know to ask for at least one more piece of information.

It may seem to the responders that I did not intellectually research the made up story.  I did.  And as such my position remains as stated even in the light of those that want to proclaim or otherwise imply that my response has no intellectual substance.  Here is my detail to support that in fact the story that started this thread is in fact flawed.

Again I must note that the supposed story is that the Determination of Most Probable cost which was based on a cost realism analysis was flawless. 

Flawless in my view as a used in the story reaches to its common use and definition as  neither "flaw" or "flawless" are defined in the Federal Acquisition Regulation.  The common use then provides that the Determination was sound without shortcomings,errors or flaws. As I noted before as a flawless determination it is reasoned that the cost realism analysis was flawless as well becasue it is part and parcel to a flawless determination.  Or stated another way if the cost realism analysis has to be nit-picked for flaws, well then the Determination was not flawless and the story is flawed. 

With regard to a cost realism analysis  a general description of the process that is found easliy via an internet and/or AI research provides that the aim of a cost realism analysis is to determine a probable cost. Most importantly the FAR at 15.404-1(d)(2) provides that a cost realism analysis "shall" provide for a determination of probable cost.   So again with regard to the story the flawless Determination of Probable Cost which is the aim of cost realism analysis is concluded by me to be flawless.  

(Here I would note that  in a post to this thread I seriously posed the question that if the cost realism analysis was flawless what else analytically is there to evaluate?  A question I believe that has not been answered because it presents a conundrum to the story that states the Determination is flawless.  Presenting something that "must" be further analzed suggests a shortcoming, yet if there is nothing to be analyzed then there is no shortcoming and the story falls flat and does not need further dialog as the OP and some of the respondents want to bat about.  Or in other words presumed flaws in a flawless determination. What?)

Most importantly A CO in making an award with regard to a flawless probable cost determination is on solid ground.  As provided by the story it is a flawless probable cost determination that had to be inclusive of a flawless cost realism analysis and the use of the Determination on which to determine cost risk to government for a best value determination is appropriate and allowed by FAR 15,404-1(a)(1).  In otherwords per the story it reasonably based and not arbitrary.  So in the context of the story Offeror A and Offeror B were both rated equal, the non cost/price factors as provided by the OP in a subsequent post are more important than price,  the CO (per the story) considered the offerors’ proposals and documented a flawless cost realism analysis that must be included in the Determination of Probable Cost therefore from a cost perspective the CO has nothing left to consider for a best value decision except consideration of the non-cost/price factors.   

PS - I hope DOGE is reading this thread as well as I believe it demostrates that the junior varisity can and has played better than the varisty 

With the above noted I now want to be direct to those that have replied specifically to my responses in this thread to which I have not yet replied.

@Vern Edwards Remember the story says "Determination" not "MPC" which I view as an estimate, or in other words the cost realism analysis which is different in my view as the MPC is just a part of the (flawless) full Determination.  As flawless Determination that should have addressed and resolved any and all questions about all the numbers but I guess you view using a MPC as a flawless Determination as well.  

@Josh Houston In response to me you have stated "I assert that there are other aspects of the cost realism analysis that, while fully captured in the contracting officer's file, are nevertheless not being considered by the notional contracting officer in their award decision, based on that information being omitted from the scenario."   I am confused.  Are you saying that in a decision of best value (remember the story did not say award) that a CO would have to revisit their Determination of Probable Cost in their narative determination of best value?  I want to suggest strongly that a cost realism analysis is singularly allowed to make a determination of fair and reasonable price and that the Determination (shall) includes the cost realism analysis and to be flawless would have already addressed everything that would have concluded on a fair and reasonable price.  If the notional CO did not do so then once again the Determination is not flawless.  Or in other words the story does not need any substantiation other that it was flawless in the eyes of the OP.  I add that in this story the OP would be left to defending the position of a flawless Determination as being reasonable and consistent with the terms of the solicitation.   But that was not the question posed by the OP.  The OP's sword is that it is flawless.   Think about it, when I posed just one simple question to @Don Mansfield about the terms of the solicitation regarding cost/price being more or less important to other factors I was taken to task for going beyond necessary information, a "sidetrack". 

PS - I hope DOGE is reading this thread as it demonstrates that the varisty has in fact made a confusing mess out of Federal acquisition and its problems for sheer academic purposes.  

 

3 hours ago, joel hoffman said:

I addressed those questions earlier. 

Edit: I’ll let DOGE know when the answer to the quiz is revealed. 

This is not a quiz with a right answer. I really am interested in how people react to the scenario. I think it’s a matter of judgment. I suspect that knowing the most probable cost of each offeror would be enough for most people. I'd like to know if I'm wrong about that.

@C CulhamI didn't mean to criticize your request for the relative importance of factors. I told you that you are free to make an assumption about what the relative importance was when answering.

I agree knowing the most probable cause of each offeror is enough for most people.

Furthermore I don’t think there are enough experienced and knowledgable people in the government now to even think about digging deeper.  I’ve seen a few in DoD and some in a couple civiian agencies but that’s about it.

I started to say something similar to Carl’s post about a “flawless” cost realism analysis but he beat me too it.  He also did a much better job of explaining the position that what I planned.

Regardless, this was a very enjoyable exercise and I hope people do more threads like this.

Don's scenario presents us with two competing "most probable costs.” Don's questions were:

1. Assuming the contracting officer's determination of the most probable cost is flawless, do they have enough information to determine the best value?

2. If not, what additional information should they consider?

My answers:

1. No.

2. How likely is it that each of the competing "most probable" costs will actually be realized? What are the chances?

To put it another way, How probable is each most probable?

FAR 15.404-1(d)(2) says, "Cost realism analyses shall be performed on cost-reimbursement contracts to determine the probable cost of performance for each offeror." But most agency "most probable cost" determinations do not produce probability numbers. They're just the agency's own take on an offeror's cost estimate. The government typically refers to its number as "the most probable cost."

But it's unlikely that any probability has been calculated. Not enough data. And while agencies should meet with offerors to discuss their cost realism analyses, they rarely do, since almost everyone seems to be scared of discussions. So another question a smart SSA might ask is: Did you discuss your MPC numbers with the offerors? If not, why not?

Don's MPCs are just points along ranges of cost possibilities. What are the ranges? What are the probability distributions? Until the decision maker knows that they don't really know enough to make an intelligent best value determination, although they probably meet GAO's standards, which are not very high.

The answer to the question: How probable is the most probable cost? has long been recognized as a key factor in rational decision-making.

 

I think it's reasonable to consider what the cost to the Government is likely to be before awarding a cost-reimbursement contract. However, I don't think the Government does that. In practice, I think the Government is merely coming up with a point estimate that it deems realistic and uses it to make decisions as if it were a fixed price. I can see a contracting officer making an award to Offeror A in the original scenario and claiming that the Government would save $5 million. However, you can't credibly make that claim without mentioning the probability of that happening.

I think the Government has come up with a technique that seems reasonable to lawyers but would seem deficient to a professional cost estimator. If the Government is going to do a cost realism analysis, I think they need expertise in risk and uncertainty analysis to evaluate cost estimates. Contracting officers typically don't have that expertise, nor do price/cost analysts. DoD trains its cost estimating community in risk and uncertainty analysis, but I haven't heard of them ever assisting in cost realism analysis. 

Alternatively, maybe we shouldn't have offerors compete on cost estimates.

1 hour ago, Don Mansfield said:

Alternatively, maybe we shouldn't have offerors compete on cost estimates.

We shouldn't.

Congress's insistence that price be a factor in every source selection, with a couple of exceptions, reflects a 19th Century view of procurement policy. Price competition has likely created more cost overruns than it has prevented. There is no market for the big ticket items the government buys, except the government. "Market" price competition does not work in such buys. Contractors for such buys should be selected based on competence and reputation, and estimated costs and fees and target prices should be negotiated one-on-one with full disclosure of cost and pricing data. The cost realism policy makes sense to auditors, but not to people engaged in high and large scale technology development programs.

Case in point. The USAF KC-X tanker second round competition in 2010 after Boeing protested the initial award to Northrup-Grumman EADS in Feb 2008.

The 2010 re-competition was touted in the news media as “fixed price”   They also reported that it was Best Value with price being very important.

Not mentioned by the news outlets was that the re-competition for the development and the first four aircraft was a Fixed Price Incentive, with 60/40 Government/Contractor share of cost growth exceed target cost.

The re-competition changed the requirements and apparently favored a smaller tanker than the EADS modified A330 tanker version. According to Northrop Grumman (N-G)  there was a clear preference for a smaller aircraft than the EADs A330, which was more expensive than the Boeing 767 version in the initial competition. N-G left the competition. EADS then decided to go it alone during the RFP proposal submission period.

The second phase of the acquisition for up to 179 aircraft would be FFP while subsequent phases would be “not-to-exceed” (not explained in the sources I read). Might be FFP with escalation clauses? 

Boeing won the contract in 2011. Initial delivery to USAF was to be in 2014. It  ended up being Jan 2019 for delivery of the first four aircraft.

There were tremendous cost overruns (60/40 government/Boeing  share) and technical problems with the Boeing KC-46, which became evident relatively soon after award and resulting in significant delays in delivery and USAF acceptance of the first four - FOUR- tankers. There were still significant technical problems after the initial delivery.

The current DoD/USAF  acquisition strategy is to re-compete after the first 179 aircraft... 

See, for instance: this 2020 Congressional Research Service report (which is “polite”): 

https://www.congress.gov/crs_external_products/RL/PDF/RL34398/RL34398.49.pdf

and this 2010 USAF article concerning the post, initial award Protest re-competition:

https://www.af.mil/News/Article-Display/Article/117525/tanker-solicitation-seeks-fair-competition-best-value/

Once upon a time (the mid-1950s), Congress's belief in the benefits of price competition in major system acquisitions was (mostly) sincere. Today it is cynical.

They know that major system acquisitions will cost (much) more than the buying agency's forecasts (calling them estimates shows them more respect than they deserve), and decisions about such acquisitions are governed by politics, not business judgment.

I don't have a problem with that. I have a problem with making agencies use needlessly costly and time-consuming methods to award such contracts and with agency incompetence in the use of those methods.

Of what use is a "flawlessly" conducted process that produces half-baked information for people who haven't been trained to know what questions to ask about it?

1 hour ago, Vern Edwards said:

Once upon a time (the mid-1950s), Congress's belief in the benefits of price competition in major system acquisitions was (mostly) sincere. Today it is cynical.

They know that major system acquisitions will cost (much) more than the buying agency's forecasts (calling them estimates shows them more respect than they deserve), and decisions about such acquisitions are governed by politics, not business judgment.

I don't have a problem with that. I have a problem with making agencies use needlessly costly and time-consuming methods to award such contracts and with agency incompetence in the use of those methods.

Of what use is a "flawlessly" conducted process that produces half-baked information for people who haven't been trained to know what questions to ask about it?

Vern, I don’t know how the USAF could have conducted a competition for the Tanker program or can conduct the new competition for follow-on Tankers without some type of cost considerations or competition.

I don’t really know anything about the next planned Tanker competition round - haven’t bothered to study it at this stage in my life.

The EADS/Northrop Grumman KC-45 was a version of the Airbus A330 Multi Role Tanker Transport (MRTT). The A330 MRTT  is a European aerial refueling and military transport aircraft based on the civilian Airbus A330. Per Wikipedia, “a total of 15 countries [have] placed firm orders for approximately 82 aircraft, of which 63 have been delivered by 28 February 2025.”

I have laughed, seeing some photos of A330 MRTT refueling a USAF fighter

____________________________________

I do know that EADS is thriving and expanding their Airbus production lines in Mobile, AL a few miles from our home at the former Brookley AFB.

After losing the KC-X Tanker re-competition, they instead committed to built the new plant in Mobile for their A-320 commercial aircraft series. The plant became operational in 2015 and delivered the first plane in 2016. A second A-320 line is underway.

Airbus also bought the Canadian Bombardier’s CRJ aircraft line, now being produced as the A-220 in Canada and as a second location here in Mobile.  I think that a new production line for those aircraft in Mobile is also under construction

Interestingly, the Mobile A-220 location was initially developed because Airbus was concerned that tariffs were being considered for planes produced on the Canadian production lines. It was later determined that the Canadian production wouldn’t be subject to tariffs but the new Mobile line is still underway.

28 minutes ago, joel hoffman said:

Vern, I don’t know how the USAF could have conducted a competition for the Tanker program or can conduct the new competition for follow-on Tankers without some type of cost considerations or competition.

Cost considerations (facts and analyses)? Yes.

Cost proposals? No.

Boeing and its political supporters fought tooth and nail to win the KC-X Tanker competition. They were widely critical of EADS in the media in an apparent effort to try to block EADS from establishing a U.S. production presence.

That might have influenced their KC-46 competitive price proposal. Maybe not.

IMO, the real winner was EADS/Airbus. They plan to assemble 25 A-320’s per month plus the new A-220 line in Alabama…

4 minutes ago, Vern Edwards said:

Cost considerations (facts and analyses)? Yes.

Cost proposals? No.

Ah so. 

10 minutes ago, joel hoffman said:

in Alabama…

@joel hoffman  Uh-huh. 😁

4 hours ago, Vern Edwards said:

@joel hoffman  Uh-huh. 😁

Hey now. I didn’t call you a tree hugging Birkenstock wearer And drive a Subaru just because you are an Oregonian. 🤪

1 hour ago, joel hoffman said:

Hey now. I didn’t call you a tree hugging Birkenstock wearer And drive a Subaru just because you are an Oregonian. 🤪

😂

You didn't. Thanks.

But I haven't lived in Oregon for almost a decade, I drive a pickup, and I wear Justin lace-up work boots. (I've never owned Birkenstocks, but I do have some Tevas). I do hug the big Doug Firs on my property (although they're sticky) and the old growth giants in the national forest behind my house. I also hug dogs, of which there are four rowdy ones in the family---a Great Pyrenees, an Australian Shepherd, an American Pit Bull Terrier, and a Coon Hound, and the family kids, of which there are several rowdy ones, five boys and four girls, with another boy due in June. I hug my wife all the time.

 I don't try to hug the cougars, coyotes, ravens, bald eagles, turkey vultures, black bears, deer, wild turkey, skunks, and elk in the neighborhood. The turkeys are downright obstinate. The black bears not so much.

We’ve had multiple Labs over the years but the best dog is our current Pit Bull. Right now he’s laying across my lap.  We also had a Coon Hound.  He was left as a young pup at the vets and the owners didn’t return.  We rescued him and he was so much different than any other dog.  Despite my best efforts in training, he often shot out the door.  I usually found him a half mile away near a stream.  

@formerfed

My Pit is the same. Wants to be right next to me or to two of the grandkids. Friendly. Loves kids. Very affectionate. Friendly with all the other dogs except the Australian Shepherd. Best friends with the Coon Hound. The Great Pyrenees is a loner. Wanders the property. Does not like to sleep inside at night. But good with kids. The Coon is just like you described yours.