@formerfed @FrankJon & any others:

I’m genuinely curious and want to continue the discussion, but not further derail Vern’s thread so:

What’s your best argument(s) for why the government should evaluate “approaches” or “understanding”?

Start by saying what you mean by "approach" and "understanding".

What are those things?

15 minutes ago, Matthew Fleharty said:

@formerfed @FrankJon & any others:

I’m genuinely curious and want to continue the discussion, but not further derail Vern’s thread so:

What’s your best argument(s) for why the government should evaluate “approaches” or “understanding”?

In the context of professional services, which is generally what my branch purchases, I virtually always attempt to dissuade my COs and CSs, and their customers, from requesting written "approaches" as to how the contractor will execute or manage the work. I've read many of the same materials as you and find the arguments against requesting approaches in most instances extremely compelling. There's also the FAR, which in multiple spots (e.g., 12.205(a)) makes clear that a written "approach" often isn't appropriate.

However, my success rate in convincing others to change their habits is pretty low. Most COs, CSs, and customer orgs in my agency were taught that this is "the way" to do acquisition, and feel uncomfortable going without it. Breaking that muscle memory is tough. In most cases, we merely agree to reduce the maximum length of these proposals. A step in the right direction, I suppose.

All of that stated, I wouldn't go so far as to say that such a request is always without merit, as you seem to suggest. There may be acquisitions for which it's necessary to ensure that an offeror understands or intends to utilize a fundamentally sound approach to the work. In such cases, the details may be too numerous, or the scope too great, for less burdensome means such as demonstrations or oral presentations. Even if the CO disagrees with such a plan, the SSA may insist on it. And even when the CO is the SSA, I would make it happen if my customer insisted on it. My rule of thumb: When it comes to compliance decisions, I insist; when it comes to business decisions, I advise.

Finally, if the government reserves the ability in the solicitation to incorporate aspects of the approach into the final contract, I think it's reasonable to assume that the offeror will take greater care than it otherwise might have to ensure the approach is realistic.

I'm realizing I didn't directly respond to your request for "best arguments." I'll wait to see what updates you make in response to Vern's reply and give this some additional thought.

I wrote the attached in 1994 as a guest article for The Nash & Cibinic Report.

STREAMLINING SOURCE SELECTION BY IMPROVING THE QUALITY OF EVALUATION FACTORS.pdf

The commentary that follows was written by Professor Nash.

33 minutes ago, Vern Edwards said:

I wrote the attached in 1994 as a guest article for The Nash & Cibinic Report.

STREAMLINING SOURCE SELECTION BY IMPROVING THE QUALITY OF EVALUATION FACTORS.pdf

The commentary that follows was written by Professor Nash.

This is the article I've shared most with other 1102s. Unfortunately, most remain skeptical that it's usually appropriate to refrain from requesting a technical proposal.

For example, this past October, I asked my office to read this. We then got together -- about 15 of us from the SPE and HCA on down -- and discussed it. I asked for a show of hands on how many agreed with your premise that a technical proposal is usually unnecessary. One person raised his hand.

My impression is that 1102s are most concerned about taking blame from the customer if performance goes awry and we didn't request a technical proposal prior to making award.

I agree that “understanding” isn’t meaningful. I also believe that “approach” isn’t always necessary. But in some situations, evaluators need to examine how an offeror will perform. Some R&D projects is one example where offeror means of achieving results can vary widely. Performance based acquisitions using a Statement of Objectives (SOO) is another example. A SOO can allow for a wide range of competing solutions where you end up with a “competition of ideas.” Evaluators need to know details of how an offeror will perform in order to consider the respective merits. IT systems development often falls into need to see offeror approach’s as well. Issues like how much of the work will be met with existing software, how much results from reconfiguration, what needs custom developed, where are the specialized experts coming from, what methodology will be used, and what security will be employed?

Sure, one can argue that this information from proposal approaches isn’t really necessary. But when the acquisition is supporting critical missions and you need to select one company with one chance to successfully perform, getting approaches is just one way to help with decision making. If nothing more, it’s tradition.

2 hours ago, Vern Edwards said:

Start by saying what you mean by "approach" and "understanding".

What are those things?

Thanks for the prompt Vern - it does make sense to get on the same page first.

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

In this context, for "understanding" I like the framework of Bloom's Taxonomy so understanding means "ability to demonstrate comprehension of something by explaining ideas or concepts."

1 hour ago, FrankJon said:

I asked for a show of hands on how many agreed with your premise that a technical proposal is usually unnecessary. One person raised his hand.

My impression is that 1102s are most concerned about taking blame from the customer if performance goes awry and we didn't request a technical proposal prior to making award.

The term "technical proposal" is empty of meaning until we know what is "technical" and what "proposal" means. You asked for a show of hands about a concept that was needed to be clarified and fleshed out.

1 hour ago, FrankJon said:

This is the article I've shared most with other 1102s. Unfortunately, most remain skeptical that it's usually appropriate to refrain from requesting a technical proposal.

Of course they do. Most don't think things through.

In the month after my article, The Nash & Cibinic Report included a postscript that reported the following responses:

We received the following letter from Bryan Wilkinson, Director, Compliance Guidelines, Teledyne, Inc., commenting on Vern Edwards' guest column at 8 N&CR ¶ 56:

Vernon J. Edwards' column in your October 1994 issue was so clear and sensible that I doubt it will have any impact on the procurement process.

When I was doing my doctoral work in psychology and statistics 45 years ago, the validity and reliability of ratings and rankings were a major concern, both in psychology and statistics. There was much literature published back then on the problem. There are some techniques to improve the process. Factor analysis of the rating elements is one that comes to mind. The conclusion reached by many, including myself, was that ratings or rankings will come out the way the rater wants them to, no matter the rating scheme.

When I got into the Government contracting area, I was astounded and “horrified” at the evaluation schemes used to award Government contracts. The extent of the overruns and product inadequacies should make it obvious that the selection process does not work properly.

Mr. Edwards has it right (from both a statistical and psychological standpoint) when he says use the offerors' records of experience, reputation for past performance, the qualifications of its key people, and price or cost and fee. (Though the first three factors are probably highly correlated.) These appear to provide rating factors which will result in valid and reliable ratings.

As a child of the Depression, I hate to see my tax dollars go for essay contests.

We also received the following letter from Steven Kelman, Administrator of the Office of Federal Procurement Policy:

I read with great interest the Vern Edwards article in the October issue of the report regarding streamlining evaluation factors. I strongly agree with Vern that most of our source selection process takes on the character of an essay writing contest. I also strongly agree with Vern that evaluation criteria should be centered around price/cost and past performance. I am very pleased Vern took the initiative to write this article for you, and I look forward to its arousing great interest in discussions within the procurement community--and to watching some agencies try out his suggested approach.

8 minutes ago, Vern Edwards said:

In the month after my article, The Nash & Cibinic Report included a postscript that reported the following responses:

Yes, I've come across this article before. High praise. I just wish the lesson had stuck to any meaningful extent.

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

5 minutes ago, Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

I presume you mean how well they will perform. Since you used the word "know," I have no choice but to agree.

No, I mean how they will perform the work. The actual "approach" they will use.

In other words: process, procedure, method, technique.

I think the answer to your question depends on what we're buying. If we're buying a product (an item of supply), they might have a well-developed standard manufacturing process, which means we can know. I think we can know in the case of standard services.

9 minutes ago, Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

We anticipate. I buy a can of beans at the market and anticipate what the beans will be like when I get it home. And I used some kind of metrics to buy the can - brand name, past experience, and even if the choices are so limited that I take what I anticipate will be good beans anyways. My thinking when the choice is limited might be a can of beans with so and so brand on it might actually come from the same manufacturing plant that not so and so's bean come from. Not much different in government contracting in my view.

So the question is by another example is what do I need to anticipate how a contractor will perform. I am quite possibly the outlier as I think I would want to know a contractors approach to building my house and their understanding of building houses to anticipate said contractors success in performing the effort of building my house.

The problem is that bureaucratic BS of FAR Part 15 is so intertwined with making a decision of anticipation it has clogged has the decision process.

56 minutes ago, Matthew Fleharty said:

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

1 hour ago, formerfed said:

But in some situations, evaluators need to examine how an offeror will perform.

The evaluators are human beings and we humans have always had a saying: "My word is my bond." The evaluators misinterpret the written word as a bond - an offer/promise. Talk about your ancient concepts. Oaths are sacred.

2 minutes ago, Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

Yes*, but that didn't stop all of humanity from entering into contracts. And you know how they made it work? Hard consequences for oath breaking. A king from ancient antiquity once even allowed the sacrificial killing of members of his tribe because he made an oath with another tribe to serve him in exchange for his protection. When he failed to ensure that protection, the king valued his bond more than his kinsmen’s lives even. A modern-day equivalent is one's guarantee of livelihood, vice an actual life. If we solicit a proposal framework that separates promises from paraphernalia, and explicitly places the hard consequence of partial payment on promise-breaking, we are applying ancient wisdom to our work. This theory is applied in an RFP that requires offers to be filled in for contract insertion, to be sure.

* I answer with 100% certainty in fixed-price contracts, but with less in flexibly priced contracts due to the post-award influence of invoice-by-invoice cost allowability determinations. In particular, terms of the contract would allow an offer's enforcement here.

Edited November 6Nov 6 by WifWaf
Vern points out below, and rightly so, that this was not my best work. Lol. What Vern doesn’t realize, though, is that marijuana use is now legal in 40 states. Just kidding. Daylight Savings had been hitting me like a jet lag every day for four days.

This may well be the goofiest discussion of contracting I have ever read. 😂

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

29 minutes ago, Don Mansfield said:

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

No. Even if the approach were contractually stipulated, the contractor might breach the contract.

1 hour ago, Don Mansfield said:

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

No.

Ok, good. If that's true, then can we agree that statements like this don't make sense?

5 hours ago, formerfed said:

Evaluators need to know details of how an offeror will perform in order to consider the respective merits.

Except my comment is taken out of context.

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

6 minutes ago, formerfed said:

Except my comment is taken out of context.

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

It's the "need to know" that I'm taking exception to. You cannot "know" something about the future.

12 minutes ago, formerfed said:

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

Without details of what?

We can evaluate details of the offer and the offeror.

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

22 minutes ago, Matthew Fleharty said:

Without details of what?

We can evaluate details of the offer and the offeror.

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

I agree evaluation of details of approach isn’t needed. I was referring to details of the offer. Sorry for the confusion - bad choice of wording on my part.