Matthew Fleharty

1 hour ago, Vern Edwards said:

@Scrutor

What you have described is a failure of requirements analysis and market research.

Bingo - there is a difference between wanting to understand various approaches during market research and evaluating approaches (which are usually not promises) during source selection.

@Scrutor Had your story occurred during a source selection (not some measly purchase order) and the one offeror wrote that as their approach, received the contract because the evaluators liked what they read, and then started to perform without using the ground protection mats (let's say because using the mats would have cost more money and a dollar saved is a dollar earned for the company), what then?

1 hour ago, Vern Edwards said:

No, it's not stronger than I think.

❤️ 😂

3 hours ago, formerfed said:

Technical approach is just the norm for R&D.

I don’t contest that this is the norm - that said, are our R&D projects generally successful (choose your measure(s))? That’s worth thinking about because, if they’re not, well we all know what they say the definition of insanity is…

2 hours ago, Vern Edwards said:

Well, rather than telling them "no," we should explain why what they want to do would be wasteful of their time and resources and then suggest something different, perhaps like oral presentations and Q&A sessions instead of written "technical proposals".

Most of a CO's clients want to conduct source selections the way they were taught to conduct them by contracting personnel. They need to be counseled, untaught, shown better ways, and persuaded rather than be ordered around. As often as not, if properly advised, their reactions will be, "Can we really do that? That would be great!"

Agreed

Thanks everyone who has contributed to the discussion. I’m working on an article that critiques the common practice of requesting approaches so I’ll have a more fulsome response for your thoughts/feedback soon.

While I rarely say always or never, I have not seen/encountered a situation in my professional career that would have benefited from evaluating approaches in competitive acquisitions - quite the contrary, I’ve only seen the practice make situations worse.

11 hours ago, FrankJon said:

Moreover, even if this were substantively true, I disagree with the notion of "overruling" a customer who insists on requesting such information. I think such a decision rightfully falls within the "advisory" bucket of a CO's responsibilities, even when the CO is also the SSA.

I strongly disagree with this position - if most acquisition “professionals” lack competency, most customers are even more ill informed about how to properly structure a competitive acquisition’s decision making process. In almost all cases, they’re insistent on doing something because that’s what they did last time which certainly doesn’t make it right.

Vern once told me (or wrote) contracting officers don’t have “customers” we have “clients” and unlike the adage “the customer is always right,” part of doing professional work is being able to tell a client “no.”

12 minutes ago, formerfed said:

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

Without details of what?

We can evaluate details of the offer and the offeror.

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

@Don Mansfield Towards the end of the podcast you state you're looking for a word to describe the FAR Part 15 "competitive" negotiations process - I think the appropriate word is "begging" (e.g. we send evaluation notices asking offerors to improve/revise an aspect of their proposal/offer) which contrasts nicely to what we should be doing, and as Vern notes we do under the A&E process, which is "bargaining."

2 hours ago, Vern Edwards said:

Start by saying what you mean by "approach" and "understanding".

What are those things?

Thanks for the prompt Vern - it does make sense to get on the same page first.

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

In this context, for "understanding" I like the framework of Bloom's Taxonomy so understanding means "ability to demonstrate comprehension of something by explaining ideas or concepts."

@formerfed @FrankJon & any others:

I’m genuinely curious and want to continue the discussion, but not further derail Vern’s thread so:

What’s your best argument(s) for why the government should evaluate “approaches” or “understanding”?

2 hours ago, formerfed said:

Offeror’s approach is important in many situations. Multiple ways may exist to achieve a needed outcome especially in R&D. You want to throughly understand how they will perform and assess likelihood of success. It doesn’t have to be an evaluation factor, nor should it be explicitly. But it needs considered.

In almost all cases an offeror’s approach is not a promise/offer - assuming the approach is not a promise, do you still believe the government should evaluate approaches?

13 minutes ago, formerfed said:

I’m curious about risk in your comment. How would you consider an offerors approach is based on something untried or with limited successful results? Past performance and/or experience or something else?

I would not consider an offeror's approach as an evaluation factor.

Why is there such an obsession with evaluating offerors' approaches?

As for "risk" in a source selection, how do you measure it? And, if you try to, how is it distinct from what has already been evaluated (the offer and the offeror's capabilities) as to not create double counting? Moreover, the "risk" might not be attributable to the offer/offeror at all - it may simply be the product of the government's own requirements (consider the risk in recapitalizing the country's intercontinental ballistic missiles...) at which point why waste the time.

1 minute ago, FrankJon said:

I provided 4 examples of things we commonly evaluate for that are not necessarily part of the offer, and you lumped them together as part of the "essay-writing" epidemic. But factors like "experience" and "past performance" are the solution to this epidemic, not the problem. And all non-price factors, in one way or another, address "risk." Surely you didn't mean to state that consideration of these aspects during a source selection is "improper."

I should have been more clear - I don't take exception to evaluating past performance or experience (both fit within the definition I provided as attributes of an offeror). I do however take exception with evaluating things like "understanding" and "risk" as I think they produce noise.

6 minutes ago, FrankJon said:

The original question was:

I interpreted this post to be soliciting opinions on how to define "value" through the lens of the current and future part 15s. Both the current and future part 15s state that: "The goal of source selection is to select the proposal that represents the best value to the Government."

On the other hand, if the question is actually asking how we would define "value" if we were drafting (or re-drafting) part 15 from scratch, perhaps I'd give a different response.

I think both current and future part 15s are wrong for stating that. I'm curious what your difference response would be.

Here's my concern with applying AI to government acquisitions - it's drawing from what we've already done, it is not creating anything new/novel. And what we've done/been doing over the past two decades isn't exactly great. The result is that AI is going to spit out the same or similar products and we're going to get the same or similar results. Is that what we need right now? To get to mediocrity (or worse) faster? I don't think so, but I'm just speaking for myself.

7 hours ago, FrankJon said:

The use of the word proposal was intentional. When we consider value or “best value” we usually consider aspects that go beyond the offer. Past performance, experience, understanding, risk, etc.

Do you disagree?

I disagree as to whether that's proper. When you say "we" if you're talking about the acquisition workforce you're correct in that for some absurd reason many people usually consider aspects that go beyond the offer through insane essay writing contests. This is NOT a good thing. What does it mean to evaluate their "understanding" particularly when proposals are written by capture teams (or today it could even be AI) and then, once awarded, tossed over the fence for a separate team to perform? What "value" does that provide?

Vern has written extensively critiquing the all too common practice of evaluating things/proposals that do not provide value to the government. See the following:
1.      “A Primer on Source Selection Planning: Evaluation Factors and Rating Methods” by Vern Edwards

2.      “Contracting Process Inertia: The Enduring Appeal of The Essay-Writing Contest” by Vern Edwards, Addendum by Ralph C. Nash

3.      “Streamlining Source Selection by Improving the Quality of Evaluation Factors” by Vern Edwards, Addendum by Ralph C. Nash

Nate Silver wrote in his book The Signal and the Noise "Information is no longer a scare commodity; we have more of it than we know what to do with. But relatively little of it is useful. We perceive it selectively, subjectively, and without much self-regard for the distortions that this causes. We think we want information when what we really want is knowledge." (emphasis added)

We can do better - let's not regurgitate the poor source selection practices that have built up over time having been copy and pasted ad infinitum to deliver acquisition outcomes that no sane person would consider respectable.

10 hours ago, Vern Edwards said:

How would you define value in FAR Part 15, as in "best value"?

I would define value as: worth assigned to an attribute of an offer or offeror

2 hours ago, FrankJon said:

Without researching the question, I would define value as used in part 15 as something like:

"The anticipated benefit of a proposal to the Government relative to other competitive proposals based upon an assessment of price and non-price factors."

I don't intend to be mean, but I will be critical: this is an example of one of the biggest problems in our profession today - people don't take the time to research or think things through, they just go along saying or doing things unthoughtfully.

For example, why did you choose the word "proposal"? There is a good discussion going on right now in these forums about the difference between "proposals" and "offers" - might be worth a read and it will likely (hopefully) prompt you to rethink your choice of the word "proposal."

I'm going to heed my own advice and think on Vern's question for a bit before providing my input. Looking forward to the discussion.

3 hours ago, Voyager said:

COs report that most transactions - and that means most of their work hours - use commercial procedures

This is faulty logic - transaction count does not automatically mean most of the work hours are spent on those activities - it just means there are more of those transactions.

• For example, if we have 100 sprinters run 400 meters and it takes each of them 60 seconds to run that distance, the total aggregated time spent running is 100 minutes.

• If we have 1 person run a marathon and it takes that runner 3 hours (180 minutes) to run that distance, despite considerably fewer runners (only one!), the total aggregated time spent running is longer.

You cannot draw any sound conclusions on where most work hours are spent (FAR Part 15 vs. Commercial...to say nothing of contract administration) based solely on the FPDS data you allude to in your post.

1 hour ago, formerfed said:

The clause was removed because it’s not required by statue. The guidance says contracting officers are free to use this clause or any variation they like based on the instant procurement. In others words it’s discretionary and not mandatory.

This is one of the problems with the "statutory" filter they are using for the RFO - even if some language in the FAR is not required by statute, that language could be useful in producing better acquisition outcomes for both parties.

As to whether contracting officers free to use the clause or any variation will do so, I doubt it with the current state of education and training in our profession and the mandate of contract writing systems/clause logic generators that constrain contract writing. Many people will not know to use this clause or others like it (or even that they exist or how to find them post-RFO transition), while others simply will not want to go through the trouble of fighting the system(s) to do so.

Meanwhile, other language not required by statute remains. See RFO FAR 12.304(b)(1) "Before terminating a contract for cause, send a cure notice to the contractor, unless the reason for the termination is late delivery." I had my eye on whether that language (which is in the current FAR at 12.403(c)(1) "The contracting officer shall send a cure notice prior to terminating a contract for a reason other than late delivery.") would stay, go, or be rewritten after a frustrating experience where it constrained our options on an important service contract. I have found no statutory basis for that language/requirement for a cure notice.

At least they're consistent in their inconsistency...

@Self Employed I think you came here with a point of view that you're stuck on and you're not reading what I wrote.

On 7/20/2025 at 7:46 PM, Self Employed said:

The pay system is a problem.

The "promote your best technical leaders to supervisory roles," method is a problem.

I wrote that I agreed with both those sentiments, but those are separate issues worthy of a separate more fulsome discussion.

You also contradict yourself when you state:

On 7/20/2025 at 7:46 PM, Self Employed said:

As in many fields -- the people who are willing to put in the work, the research, the time and effort -- will do so, development program or not.

So why do we need a different pay, development, incentive system if the people who are going to develop are going to do so anyhow, development program or not? (I don't agree with your sentiment btw - some people need to be shown the way and once inspired will thrive)

I think there is a fundamental misunderstanding in the contracting and acquisition community of the power of expectations. That's why I think the 80 CLP standard and the Powerpoint deep CBT nonsense we're constantly exposed to and multiple guess exams to demonstrate proficiency is a serious problem. It sets an expectation that merely going through the motions is enough and in a world full of competing priorities many people are simply going to do what is expected of them. That is why we need to raise the standards - many will work to achieve them, some won't be able to, but we'll recruit and retain those who can.

And yes, with raised standards I think raised compensation is appropriate/necessary too.

On 6/19/2025 at 8:41 PM, Self Employed said:

I think all this talk of "where are the good CO's, why are the bad CO's being bad CO's," misses the reason:

All of your good CO's either (increasingly,) leave or promote.

I partially agree that many good CO's leave or promote out of doing CO work, but that's not the only reason we are where we are (as you assert) - we still have a development problem and I think it's been getting consistently worse since I entered the career field over 15 years ago.

Do I think we need to re-examine compensation and some of the career ladder concepts? Absolutely - that's an article for another time. But if we only did that, we would still have a problem developing the next generation of contracting professionals and have no one to offer those things to...

Why do you think it is a best practice to not have people on multiple teams?

What if you only have a small shop with minimal people?

Or only one person who has any experience leading a source selection?

Since you're referring to the DoD Source Selection Procedures, has the DoD been successful in implementing and utilizing those procedures and following the practice you detail in your post? If not, why would you want to continue to do it, especially when you claim it isn't "explicitly stated in the procedures that they can't overlap..."?

In short, what do you think is best and why?

On 6/3/2025 at 2:45 PM, Vern Edwards said:

Agree. But we need to explore the concept of "professional development".

You'll notice that Lt. Col. Fleharty made a distinction between education and training. The distinction is important. I would say that education is the process of gaining knowledge and training is the process of learning how to apply knowledge in practice.

Education without training is suboptimal. Training without education is useless, even dangerous.

(Emphasis added above) Thanks for highlighting this distinction Vern - the takeaway is spot on.

If readers want to know why training without education is dangerous, just look at our source selection process - all training, zero education. The result is we often end up with copy/paste solicitations without thoughtful evaluation factors or methodologies that cause source selections to take way too long evaluating "essays" that are not important to predicting which offeror and their proposal might be the best value.

On 6/4/2025 at 7:48 PM, Don Mansfield said:

@Matthew Fleharty

What is your opinion of DoD Instruction 5000.66, DEFENSE ACQUISITION WORKFORCE EDUCATION, TRAINING, EXPERIENCE, AND CAREER DEVELOPMENT PROGRAM?

Do you think the instruction is to blame at all for the state of the DoD contracting workforce? Or are the problems all in the execution?

Both

To note a couple items that stood out from the DoDI, Section 5 establishes the "low expectations" I criticized in my piece and Section 3 makes the President of DAU as the Chief Learning Officer... in my 16 years I haven't seen any real leadership from that role, if anything the standards and courses have eroded more and more...whose responsibility is it to say "this is not or cannot produce the acquisition workforce we need to generate the acquisition outcomes we need" if not a Chief Learning Officer?

Execution is a huge problem too. It's apparent from some of the pockets of excellence that people can still excel in this profession and pursue mastery (as Vern likes to say), but we definitely make that pursuit way more difficult than it ought to be.

For one, we don't leverage peoples' most formative moments when they enter contracting to properly educate and inspire them. I mention this in the piece so I won't further belabor the point here.

I listened to the two podcast episodes you recorded (thank you for those btw) and I think in both comments were made that contracting people don't read. I think that's fairly accurate, BUT we ought to consider why they don't read: one of my hypotheses is because the "professional" products the acquisition system currently provides them are mostly garbage. I remember in high school when I went through a stretch in my English class of reading a few assigned books I did not enjoy - it made me not want to read for a bit. That's our contracting people's daily existence - suffering through reading bad policies, Powerpoints, and guides that they don't benefit from so at some point many of them just give up (i.e. people won't go back to try and drink from a well that's dry).

On 5/4/2025 at 10:33 PM, Don Mansfield said:

That jumped out at me, too. I'm going to submit a comment on the importance of that statement.

It's the kind of thing that just instills confidence in what might come out the other end of this process...

Admiral Rickover has a quote that I am fond of (emphasis added): "Organizations don't get things done. Plans and programs don't get things done. Only people get things done. Organizations, plans, and programs either help or hinder people."

FAR reform will not fix our acquisition system because our system relies on people to make decisions. People will still need to read, interpret, and apply the FAR - whether they make sound decisions or bad decisions is based on their knowledge and abilities, little of which has to do with what is or is not in the FAR. A better written FAR could help them, while a poorly written FAR could hinder them, but it's the people that policymakers and leadership have lost sight of. We are not setting our people up for success. None of the people in the government acquisition workforce receive quality education on contracting as a standard practice. None of the people in the government acquisition workforce have access to best in class materials like the Cibinic and Nash Report or Briefing Papers. What our people get instead are Defense Acquisition University (DAU), a littany of computer based trainings (CBT), and on the job training (OJT) that, in most cases, is little more than showing them what was done last time and expecting them to do the same....oh, and calls for AI, "innovation," and buying commercial as if they are panaceas...

If we have a well educated acquisition workforce, I believe they could overcome even the most convoluted FAR (and any other organizations, plans, or programs that might serve as hinderances). Conversely, if we have an uneducated acquisition workforce, I believe we could have the best written FAR and that workforce would still find ways to break it.