Jump to content
The Wifcon Forums and Blogs

Sign in to follow this  
  • entries
    9
  • comments
    55
  • views
    41,449

Experience is the teacher of all things.

Emptor Cautus

3,574 views

Experience is the teacher of all things.

― Julius Caesar, Commentarii de Bello Civili (Commentaries on the Civil War), (c. 52 B.C.).

In a previous Blog, I said:

Note that the evaluation approach put forth as a solution in West Coast is somewhat different than what I suggest, which has not been addressed by the Comptroller General. For those of you who are concerned about my approach giving an unfair advantage to an offeror with no past performance information let me suggest, as Mr. McGuire did to Benjamin in The Graduate, “ I just want to say one word to you. Just one word.” Experience. But, that is a subject for another Blog.

What am I suggesting by that?

In acquisitions exceeding the simplified acquisition threshold past performance shall be evaluated, unless waived by the contracting officer. (FAR 15.304( c )(3). In addition to, and separate from, past performance the source selection should evaluate “Experience.” Past performance deals with how well an offeror has performed prior efforts. Experience deals with whether the offeror has performed such efforts. Absent past performance information, the Federal Government may not treat the offeror favorably or unfavorably. Absent experience, the Federal Government may evaluate an offeror unfavorably, if the evaluation factors for award in the solicitation so state.

Can you do that? According to the Comptroller General of the United States, you can. Let me provide you pertinent parts from just two Government Accountability Office protest decisions that demonstrate that you can.

Commercial Window Shield, B-400154, July 2, 2008

CWS’s argument, however, fails to recognize that the experience and past performance factors reflected separate and distinct concepts. Under the experience factor, the agency examined the degree to which a vendor had experience performing similar projects; under the past performance factor, the agency considered the quality of a vendor’s performance history. Given the fundamentally different nature of the evaluations, a rating in one factor would not automatically result in the same rating under the other.

Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture; B-401679.4, March 10, 2010

Generally, an agency’s evaluation under an experience factor is distinct from its evaluation of an offeror’s past performance. Specifically, the former focuses on the degree to which an offeror has actually performed similar work, whereas the latter focuses on the quality of the work.

You remember the concern expressed by Frere Two?

To me the Government is disadvantaged by a proposer with no relevant past performance. Why did they write the regulation in such a manner? Just as you would not want to ignore favorable or unfavorable information, why would you ignore the lack of?

The approach described above, addresses his concern. Well, it doesn't address his concern about why they wrote the regulation is written in such a manner. Unfortunately, the answer to that is the same as Inspector Javert gave to Jean Valjean in Les Misérables, “Right or wrong, the law is the law and it must be obeyed to the letter.” But, it does provide a way for him to work within the law to achieve, essentially, the same effect.



10 Comments


Recommended Comments

Thank you for continuing this discussion!

I just had a few questions . . .

I know that the GAO is inclined to allow the consideration of relevance under a past performance evaluation factor:

(Insect Shield Manufacturing, LLC B-408067.3, Aug 8, 2013):

". . .

In this regard, we think that it is both illogical and unreasonable to presume that an agency will pay no attention to the size and similarity of past contracts in its evaluation, since such factors are germane to the relevance of the past performance information. J. A. Jones Grupo de Servicios, SA, B-283234, Oct. 25, 1999, 99-2 CPD ¶ 80 at 7.

. . ."

If you add an "Experience" factor, should all strengths/weaknesses about relevance be moved the "Experience" Factor? Or can they be counted in both evaluation factors?

In the event an offeror with no past performance information submits a proposal, and we have taken your recommendation to not evaluate past performance, does the "Experience" factor serve as a life boat which preserves our ability to consider relevance/currency of the past performance of the other offerors?

Please forgive me if I am not following. Very interesting discussion!

Share this comment


Link to comment

Relevancy can be considered in both factors. Check out the DoD two step procedures for evaluating past performance. As part of the first step, the government determines the relevancy of past experience to the instant task or contract scope. In the second step, in evaluation of past performance information, they assess the level of "confidence" that the firm will successfully complete the contract. Past performance on projects with more relevance to the instant project are weighted more than those with less relevance.

Thus a fantastic baker may not necessarily make a fantastic ship builder in a ship building contract.

Project experience should still be a separate factor.

Share this comment


Link to comment

Well, nonsense, unless this is another educational lesson, yes, there is a difference between past performance and experience but apsofacto still believes your recommendation "not to evaluate past performance" when an offeror has none, ceteris paribus. I already described the reasons why this approach is not correct.

By the way, a comma should be placed after "past performance" not after "separate from,".

Based on the content of Emptor's articles, recommend ignoring. Tnx.

Share this comment


Link to comment

Point of clarification: I will *not* remove the past performance evaluation process if an offeror has none. I will believe it can be done when another CO does this and a protest goes to GAO who then says that action is permissible. I will not be that CO.

The "neither favorable or unfavorable" language is still problematic, but I intend to handle it in the traditional manner. (Put a comment on the previous entry to clarify that, putting one here too)

If anyone does try this, kudos for your courage, and we'll all see how it turns out.

Share this comment


Link to comment

No courage needed, apsofacto. Just state in the RFP what you will do and then do what you say. Zero risk.

physiocrat, your comma criticism comes easy from somebody who hasn't written much of anything at Wifcon longer than 100 words or so. That was a cheap shot. Try posting a blog with this software, smarty pants. We'll see how you do. Look at the first sentence of your last comment:

Well, nonsense, unless this is another educational lesson, yes, there is a difference between past performance and experience but apsofacto still believes your recommendation "not to evaluate past performance" when an offeror has none, ceteris paribus.

You should have put a period after "lesson," started a new sentence with the "yes," and put a comma after "experience." "Ceteris paribus" was just showing off.

Emptor's approach was not "not correct." it was just what he would do based on his interpretation of the language in FAR, and it would not be improper to do it that way. I wouldn't do it his way, but I don't think his way violates FAR. If anybody's comments should be ignored, physiocrat, it's yours in response to this blogpost.

Share this comment


Link to comment

I think the first sentence flows nicely Vern a new sentence is not required but in Emptor's case, the misplaced comma is definitely a grammatical error, not subject to debate according to any rules I am aware of. I suppose you are just defending a friend and I respect that.

I am shocked that you think ignoring neutral past performance against superior past performance when conducting COMPARISONS is "proper" when the regulation says no such thing, or maybe just his interpretation?

I love you anywho Vern.

Of course, recognize your source. I posited that the Civil War and the acquisition of a wife has everything to do with procurement principles but I was banned for a time, myopic views prevailed, and so it is, who am I to disagree?

Tnx.

Share this comment


Link to comment

You think your first sentence flows nicely, Emptor thinks his does, and he, unlike you, did not make a grammatical error: His phrase "and separate from" is parenthetical and merited a comma at each end. Whether he needed to put another one after "performance" is debatable. You, on the other hand, omitted a comma before a conjunction ("but") that separated two independent clauses. That is a grammatical error. Look it up.

As for Emptor's position on evaluation, neither statute nor FAR state how to handle a comparison in which one of two offerors has no record of past performance, and so Emptor's solution is perfectly legal. The regulation does not have to say that he can or may do it that way, as long as it doesn't say, explicitly or impliedly, that he may not. I don't agree with him that it is necessary, but it is certainly not improper.

Really, physiocrat, you ought to check your target and the background before opening fire.

Share this comment


Link to comment

If you add an "Experience" factor, should all strengths/weaknesses about relevance be moved the "Experience" Factor? Or can they be counted in both evaluation factors?

In the event an offeror with no past performance information submits a proposal, and we have taken your recommendation to not evaluate past performance, does the "Experience" factor serve as a life boat which preserves our ability to consider relevance/currency of the past performance of the other offerors?

Maybe I'm bad at following along but I don't think he is advocating for not evaluating past performance; he seems to suggest having another pass/fail criteria by which you can otherwise exclude an bakery from a shipbuilding contract. This way, you have a reason to fail the offeror for not having experience rather than being bound by the Phantom Zone that is "unknown past performance".

EDIT: Formatting.

Share this comment


Link to comment

There is no "Phantom Zone." The so-called "neutral rule" in FAR 15.305(a)(2)((iv) says:

In the case of an offeror without a record of relevant past performance or for whom information on past performance is not available, the offeror may not be evaluated favorably or unfavorably on past performance.

That rule does not preclude unfavorable comparisons of such an offeror with an offeror that has a favorable record of past performance.

Suppose that the evaluation factors are past performance and price, with past performance being the most important. Suppose now that you have two offerors, A and B, and that their past performance ratings and prices are as follows:

A Excellent $1,000,000

B None $ 800,000

The rule in FAR 15.305(a)(2)(iv) does not preclude an agency from finding A to be the better choice because of its excellent past performance. That has been long established in the protest decisions. See, for example, Blue Rock Structures, Inc. GAO B-287960.2, 2001 CPD ¶ 184 (Oct. 10, 2001):

[W]here a solicitation, as here, provides for award on the basis of the best value offer, a determination to award to a higher-priced offeror with a good past performance record over a lower-priced offeror with a neutral past performance rating is not precluded. Such a determination is consistent with making a price/technical tradeoff to determine if one proposal's technical superiority is worth the higher cost associated with that proposal. See Eng'g and Computation, Inc., B-275180.2, Jan. 29, 1997, 97-1 CPD ¶ 47 at 4-5; Excalibur Sys., Inc., B-272017, July 12, 1996, 96-2 CPD ¶ 13 at 3.

Footnote omitted.

The flip side of that would be if the evaluation results were as follows:

A Poor $ 800,000

B None $1,000,000

In such a case the agency might find B to be the better value.

See Kushnir, "Evaluating Offerors Without Past Performance Information: The Good, the Bad, and the Neutral," Public Contract Law Journal (Winter, 2006), 35 PUBCONLJ 317:

GAO has long established that agencies may trade off past performance against price and other evaluation factors as long as their decisions are reasonable and well documented. This discretion extends to situations where an offeror has no past performance information. For example, an agency may select a contractor with a higher price and good past performance history over a contractor with a lower price but no past performance history. Similarly, an agency may determine that an offeror's better past performance is not worth the extra price, even when the solicitation specifies that past performance is more important than price. Finally, and not surprisingly, GAO also has upheld an award to a higher-priced contractor with no past performance information when the alternative was a lower-priced contractor with negative past performance history.

Footnotes omitted.

Emptor knows this, but disagrees with the GAO. He thinks that considering A to be better than B would be evaluating B unfavorably in violation of statute and FAR 15.305(a)(2)(iv). He therefore thinks that when comparing A and B you cannot consider past performance at all. That is not in accord with the case law, but Emptor is free to disagree. Why not? I disagree with the GAO on many things, and as soon as I become Comptroller General I'm going to change those things. In any case, that is why Emptor thinks you need to evaluate experience.

I, too, think that you need to evaluate experience, but for a different reason. To me, experience tells you how many chances an offeror has had to learn about the kind of work to be done under a prospective contract. Past performance tells you how well the offeror has learned from experience. The two factors complement one another.

Share this comment


Link to comment

Vern,

My personal experience in source selections is limited to a mere handful being a Jr. NCO, although the climate AFICA brings will hopefully fix that.

The contention between Emptor and GAO plays itself out at the local level regularly.

The "make things protest-proof" mentality we've adopted results over-engineering of source selection, where exactly the situation Emptor describes comes up. The "Phantom Zone" I alluded to is representative of exactly the reasons Emptor seems to be arguing for. It seems "low price but unknown past performance means you don't get the award" is viewed as treating the past performance unfavorably and attempts to argue otherwise end up "feeling" wrong (certainly not a compelling argument, but it sure sounds like one). I think we ride a seven-legged horse into battle, however, and wonder why it isn't 75% faster than the four-legged variety.

Perhaps my assumption that others encounter the same sort of obscurities and hesitations is an inappropriate assumption.

I agree with what you posted regarding the treatment of neutral past performance; best value is what the Government decides to be the best value and we shouldn't be compelled to make a decision which produces anything less than that.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×