Jump to content

Subjective Past Performance Evaluation


Recommended Posts

Are there any restrictions or requirements on how Past Performance is used as an evaluation criteria?

We are faced with a solicitation that includes Past Performance as one of the two primary evaluation criteria. The agency states that this evaluation will not be based on the information/references we provide in our proposal, or any kind of quantitative analysis, but rather "subjective judgment" of all available and relevant information the agency can find. This seems like a huge variable that would allow the agency to validate nearly any best value judgment it makes.

Frankly, I doubt there is any restriction on including an evaluation criteria like this, but if there are, I would really like to know about them as this solicitation is wired for a particular contractor and this variable is troubling.)

Link to comment
Share on other sites

.

Without actually addressing your question, let me remind you that a protest is due not later than when bids are due. This is a patent defect in the solicitation.

If you ask the agency a question or comment and wait for a reply, and the deadline passes, you are too late.

.

Link to comment
Share on other sites

It sounds like they are using FAR 13.106-2.

(3) If using price and other factors, ensure that quotations or offers can be evaluated in an efficient and minimally burdensome fashion. Formal evaluation plans and establishing a competitive range, conducting discussions, and scoring quotations or offers are not required. Contracting offices may conduct comparative evaluations of offers. Evaluation of other factors, such as past performance?

(i) Does not require the creation or existence of a formal data base; and

(ii) May be based on information such as the contracting officer?s knowledge of and previous experience with the supply or service being acquired, customer surveys, or other reasonable basis.

Link to comment
Share on other sites

Guest Vern Edwards
The agency states that this evaluation will not be based on the information/references we provide in our proposal, or any kind of quantitative analysis, but rather "subjective judgment" of all available and relevant information the agency can find.

I see no problem with that statement. When the agency says that the evaluation will not be based on the information/references provided by the offerors I'm sure they mean that they will not rely on that information exclusively. As for subjectivity, all past performance evaluations are ultimately subjective. What's wrong with that? And there is no requirement for any "quantitative analysis."

I see no patent defects and I see nothing to protest about. You should get on with preparing and submitting your proposal.

Link to comment
Share on other sites

Go back and re-read the solicitation. I'm sure it says the agency will evaluate all other information as well as the information/references you provide. Of course they will consider what you submit. Otherwise they wouldn't be asking for it. But it only makes sense for them to evaluate everything they come across.

Link to comment
Share on other sites

Under this scenario, should the offeror be given the opportunity to respond to subjective adverse past performance information? Because this is a subjective evaluation, can "adverse" be defined as not as good as the next bidder.

Vern already mentioned all past performance evaluations are ultimately subjective. Even if you have a questionnaire type rating system with definitions, it's still subjective. About the only objective or quantitative rating is something like "the company was four days late with delivery." Even then if an offeror was "X" days late, that can be considered adverse and the offeror given an opportunity to respond - maybe's there's a good reason.

Yes, if the past performance information is adverse, offerors should have the opportuntity to respond.

Past performance information is always in the context of immediate work or contract jobs. So you never compare one offerors ratings to another. The information is adverse or it's not.

Link to comment
Share on other sites

So, perhaps I should ask in the transmittal letter of our proposal or somewhere that we be allowed to respond to any past performance information that is anything less than exceptional.

To say, "please let us explain if you hear bad stuff" sounds terrible.

Anything less that exceptional is not adverse. If you want to find out more on the subject, there are several good protest decisions on this subject at this site. Go to the protest section and check under FAR 15.306(B)(4). That shoould be "b" instead of (B)

Link to comment
Share on other sites

I don?t see anything in the protests that deal with a protest being denied because the past performance was found not to be adverse in the first place and therefore did not require the bidder to be given a chance to respond. The protests seem to deal with CO discretion, validity of the past performance information, and responsibility of the bidder to provide information.

I don?t see how if there are two bidders, and everything else being equal, and the award is based upon past performance, if the awarded bidder is subjectively found to have very exceptional past performance and the loosing bidder is subjectively found to have only exceptional past performance that the loosing bidder?s past performance was not adverse.

Where is adverse defined as having to be poor or bad in its own right?

Link to comment
Share on other sites

I don?t see how if there are two bidders, and everything else being equal, and the award is based upon past performance, if the awarded bidder is subjectively found to have very exceptional past performance and the loosing bidder is subjectively found to have only exceptional past performance that the loosing bidder?s past performance was not adverse.

Where is adverse defined as having to be poor or bad in its own right?

I never have considered "adverse" in the context you described. I've always thought "adverse" to be negative and even a "successful" or "satisfactory" rating probably isn't adverse depending on the reason. So if one offeror had only exceptional versus another offeror having very exceptional, that's not adverse. Dictionary.com defines adverse as "unfavorable or antagonistic in purpose or effect."

Aslo I believe the decision of whether a rating is adverse is made on an individual offeror basis and not after comparing offerors. But I understand your opinion. That's interesting and I may be wrong.

Link to comment
Share on other sites

.

Former Fed,

I read the definition you furnished from dictionary.com, and have to wonder what you are thinking.

The definition says "adverse" means "unfavorable in effect." Well, losing a competition is unfavorable, in this situation. And having merely "exceptional" past performance information led to that outcome in the hypothetical situation (exceptional vs. super exceptional.) Thus, for this hypothetical, it satisfies the definition of "adverse" that you furnished.

I suspect "adverse" is defined differently in the relevant part of the FAR. That would be a better place to get a definition.

.

Link to comment
Share on other sites

Brian,

"have to wonder what you are thinking"!!

"Adverse" is not defined in the FAR. I'm not aware of it being defined either in case law.

The FAR says to address adverse past performance information to which the offeror has not previously had an opportunity to comment. The question is whether "adverse" applies in the context of an individual contract such as "the contractor performed in a good manner" which may not be considered adverse, or whether the contractor performed in a "good" manner while another offeror is rated "excellent." Can "good" be considered adverse in the comparative context?

I've never seen it applied before in the latter instance.

Link to comment
Share on other sites

.

Former Fed,

I read the definition you furnished from dictionary.com, and have to wonder what you are thinking.

The definition says "adverse" means "unfavorable in effect." Well, losing a competition is unfavorable, in this situation. And having merely "exceptional" past performance information led to that outcome in the hypothetical situation (exceptional vs. super exceptional.) Thus, for this hypothetical, it satisfies the definition of "adverse" that you furnished.

I suspect "adverse" is defined differently in the relevant part of the FAR. That would be a better place to get a definition.

.

Brian,

Losing a competition is not necessarily "unfavorable" or "adverse". Such a determination is entirely subjective and may not be readily apparent at the time that it occurs either.

Consider this scenario: A contractor loses a competition. Later on, the contractor finds that his or her proposal was significantly flawed and would have resulted in financial losses or substandard contract performance that would have been detrimental to future contract awards.

Is losing that competition adverse or fortuitious? Who gets to decide?

"Adverse" as a measurement itself is very subjective, it depends upon what your are using as a standard. The only reasonable standard in relation to a solicitiation or contract competition is the terms and conditions of the soliciation and the work being solicited itself, not other contractors or a standard unrelated to the solicitation. To not restrict the standard as described opens the judgement to every opinion present, and likely some that may enter the fray at some later time. And as I was told in my very early days in the military, opinions are like ...., everyone has one!

Link to comment
Share on other sites

Take a look at the following two cases. From them, I gather the following general principles. First, an agency may define in the solicitation what it considers to be ?adverse? past performance (and therefore, what contractors will have an opportunity to rebut). Second, if adverse is not defined, the past performance should be considered adverse if it is in any way derogatory or reflects poorly on the contractor. So, unless ?Excellent? or ?Good? is defined in the solicitation in some way that is contrary to how they are commonly understood, I do not think either the GAO or the COFC would find an excellent rating to be adverse past performance.

Serco Inc. v. U.S., 81 Fed.Cl. 463 (2008).

In addition, the court rejects Serco's argument that GSA failed to provide offerors with the opportunity to rebut ?adverse? past performance information. The Solicitation stated that the evaluator was to ?provide adverse information to the PCO [procuring contracting officer],? and that the PCO would ?decide? whether the offeror should be given the ?opportunity to ?rebut? the information.? This language tracks FAR ? 15.306(a)(2), which states that ?f award will be made without conducting discussions, offerors may be given the opportunity to clarify ... adverse past performance information to which the offeror has not previously had an opportunity to respond.? While the regulation does not define what it means by ?adverse? past performance information, the adjectival ratings here defined such information as that which resulted in a past performance rating of ?L/N? or ?1.? None of plaintiffs received such a rating.

Fort Carson Support Services v. U.S., 71 Fed.Cl. 571, 605-606 (2006).

The SSEB member responsible for the initial PPE evaluation recommended that FCSS receive a ?Moderate? performance risk rating for PPE overall and for the elements Managerial and Cost Control. See Tab 62. The Plum Island ratings were mentioned as a factor in these determinations that ?some? doubt existed concerning successful performance. Id. Although it went undocumented, see AR Tab 209 at 4, the SSEB's initial consensus was that FCSS should nonetheless receive a ?Low? performance risk rating for the PPE subfactor and each element. See AR Tab 76 at 33. The SSEB Chairman later explained that ?[t]he caucus determined that the somewhat derogatory reference concerning its Plum Island contract was offset by the labor union issues that existed before its start of work on the contract.? AR Tab 209 at 4. The question *606 here presented is whether this ?somewhat derogatory reference? constituted adverse past performance information that the Army was required to discuss with FCSS, as per 48 C.F.R. ? 15.306(d)(3).

To answer the question, we must first determine whether the information was ?adverse.? For the PPE questionnaire, the worst rating that could be provided was ?Unsatisfactory.? See AR Tab 19 at 3. The rating of ?Marginal? was defined as follows:

Performance does/did not meet some contractual requirements. The contractual performance of the element or sub-element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor's proposed actions appear only marginally effective or were not fully implemented.

Id. Does the failure to meet ?some? requirements, or ?a serious problem? in performance amount to an adverse rating? There does not appear to be any case law on point. One opinion of our Court held that evaluations needed to be ?deemed ?poor? or otherwise deficient? to be adverse for the purposes of this FAR provision. Bannum, Inc. v. United States, 60 Fed.Cl. 718, 729 (2004). A GAO decision suggests that a ?marginal? rating would be adverse past performance information. See NMS Management, Inc., B-286335, 2000 CPD ? 197, 2000 WL 1775243, at *3 (Comp.Gen. Nov.24, 2000). Another suggests that adverse means an evaluation that ?reflected a finding of a history of performance problems.? See Standard Communications, Inc., B-296972, 2005 CPD ? 200, 2005 WL 3078884, at *6 (Comp.Gen. Nov.1, 2005).

The Court considers this a close question. But it appears that the SSEB's PPE evaluator was influenced by the Plum Island ratings in determining that ?some? doubt of successful performance existed. See AR Tab 62. And the record does not contain any document preexisting the discussions that reflects a consensus opinion that the ?Marginal? ratings were mitigated by inherited labor conditions. In light of the above, and the characterization of the evaluation as ?somewhat derogatory?-and considering that the definition of ?Marginal? includes serious performance problems and performance failures-the Court concludes that the Plum Island evaluation contained adverse past performance information.

Link to comment
Share on other sites

I am not sure the cited cases provide much help.

In SERCO the Court?s statement (page 31): ?While the regulation does not define what it means by ?adverse? past performance information, the adjectival ratings here defined such information as that which resulted in a past performance rating of ?L/N? or ?1.?? I feel is limited or constrained to ?the adjectival ratings here? in their particular Acquisition/RFP. In our scenario, there are no adjectival ratings, other than you win - you almost won. So super imposing the courts words to our scenario we could get the following: ?While the regulation does not define what it means by ?adverse? past performance information, the adjectival ratings here defined such information as that which resulted in a past performance rating of ?you almost won? or ?only exceptional??.

The case is a good one to read ? it is a study in how to conduct a flawed past performance evaluation.

http://www.uscfc.uscourts.gov/sites/defaul...opdredacted.pdf

Link to comment
Share on other sites

I found this decision which seems to imply "adverse" is less than satisfactory.

http://www.gao.gov/decisions/bidpro/400829.htm

This is an extract. I looked the the solicitation (actually a synopsis) and it didn't define "adverse."

Finally, SEI argues that it should have been provided with an opportunity to respond to any adverse past performance information during discussions. Protester?s Comments at 5, 8; see FAR sect. 15.306(a)(2). The requirement that an agency raise during discussions adverse past performance information to which the offeror has not yet had an opportunity to respond is set forth in FAR Part 15, which is inapplicable to A/E procurements under FAR subpart 36.6. FAR sect. 36.601-3(;); HydroGeoLogic, Inc., B-311263; B-311263.2, May 27, 2008, 2008 CPD para. 218 at 7. In any event, even though the revised synopsis indicated that firms would be given the opportunity to address adverse past performance information, the past performance questionnaires least favorable to SEI rated its performance as ?satisfactory? and generally provided throughout the questionnaires that all aspects of SEI?s performance were ?satisfactory.? Thus, there is simply no merit to the protester?s contention that the agency was required to raise the matter of SEI?s past performance in discussions. See Symplicity Corp., B-297060, Nov. 8, 2005, 2005 CPD para. 203 at 8; ITT Fed. Servs. Int?l Corp., B-283307; B-283307.2, Nov. 3, 1999, 99-2 CPD para. 76 at 15.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...