Jump to content
The Wifcon Forums and Blogs

dkubis

PP Neutral Rating

Recommended Posts

Hi,

It's widely understood having no Past Performance will give you a neutral rating.  What happens when we sent out the questionnaires and they try to suppress their past performance by not submitting anything in order to get a neutral rating?

Share this post


Link to post
Share on other sites

Read this GAO decision: Menendez-Donnell & Associates, B-286599, Jan. 16, 2001: http://www.gao.gov/assets/340/332405.pdfSee if the facts align with your circumstances.

Quote

To the extent MDA contends that GSA was required to assign a neutral rating to its proposal based on the absence of information relating to its key subcontractors, we disagree. Although FAR §15.305(a)(2)(iv) requires an agency to assign a neutral rating where past performance information is not “available,” here, the protester’s proposal represented that its proposed subcontractors are engaged in projects that would illustrate their performance capability. The information thus was available, but MDA chose not to present the information in its proposal, in direct contravention of the terms of the RFP. In our view, an offeror cannot simply choose to withhold past performance information--and thereby obtain a neutral rating--where the solicitation expressly requires that the information be furnished, and where the information is readily available to the offeror.

Also, include Experience as a separate evaluation factor. Take a look at Vern’s first post on 26 April 2016:

http://www.wifcon.com/discussion/index.php?/topic/3493-interpreting-what-neutral-means-in-the-eyes-of-gao-past-performance/

 

Share this post


Link to post
Share on other sites

The key is not to limit the evaluation of past performance in the solicitation evaluation criteria to those @#$%&! questionnaires that the government loves to require the references or the proposers to submit, depending upon the solicitation language.  Provide for flexibility in what and how you can obtain the relevant past performance information to evaluate.

We used a form for obtaining information concerning the proposer/offeror's* recent, relevant experience under that evaluation factor. We used a similar form for prospective key subcontractors.  On design-build competitions, we also included a form, tailored for the design firm. There was an entry on each form for the applicable firm to indicate and describe how the owner or contracting agency had rated the firm, if any rating had been issued.  We also required the firm to provide an owner's reference and contact information and reserved the right to contact and/or interview the reference, if necessary to verify or clarify the provided information. We stated that this information would be evaluated under the past performance factor and correlated with the recent, relevant project experience. 

We made no promise to interview the references.  We often had other sources, such as the CPARS/ACASS/CCASS past performance rating systems (those are terms for past or present NAVFAC and USACE past performance rating systems) and the Past Performance Information Retrieval System to verify the ratings and the various facets of those ratings.  We also often had seen the same information from the proposers in other source selections, so we had little or no need to re-contact the references, unless there were conflicts between the sources of information. We used a telephone interview format similar to the questionnaires that are now being sent in (or not sent in!) by the proposers or their references. 

We stated that we could or would query the available past performance databases. We stated that verifiable personal knowledge by the evaluator(s) could also be used.

**I use the terms "proposers" and "offerors" interchangeably here because past performance is often used in task order competitions in addition to new contract source selections. Multiple Award Contract holders aren't necessarily "offerors'.  They are already MATOC or MACC contractors, who are competing for award of a task order under FAR 16.505 procedures, not under new contract, Part 15 source selection procedures.  

Share this post


Link to post
Share on other sites

Let's assume that the requirement for submitting past performance information was a "collection of information" as defined at 44 U.S.C. § 3502. Let's also assume that this solicitation requirement did not display a valid OMB control number, as required by 5 CFR 1320.3. Would the agency be justified in rejecting or downgrading the offeror for failure to provide the past performance information? 5 CFR 1320.6 states:

 

Quote

 

(a) Notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information that is subject to the requirements of this part if:

(1) The collection of information does not display, in accordance with § 1320.3(f) and § 1320.5( B)(1), a currently valid OMB control number assigned by the Director [of OMB] in accordance with the [Paperwork Reduction Act of 1995] Act; or

(2) The agency fails to inform the potential person who is to respond to the collection of information, in accordance with § 1320.5(B)(2), that such person is not required to respond to the collection of information unless it displays a currently valid OMB control number.

 

5 CFR 1320.3(j) defines "penalty"--

Quote

Penalty includes the imposition by an agency or court of a fine or other punishment; a judgment for monetary damages or equitable relief; or the revocation, suspension, reduction, or denial of a license, privilege, right, grant, or benefit.

 

Share this post


Link to post
Share on other sites
18 minutes ago, Don Mansfield said:

Let's assume that the requirement for submitting past performance information was a "collection of information" as defined at 44 U.S.C. § 3502. Let's also assume that this solicitation requirement did not display a valid OMB control number, as required by 5 CFR 1320.3. Would the agency be justified in rejecting or downgrading the offeror for failure to provide the past performance information? 5 CFR 1320.6 states:

 

5 CFR 1320.3(j) defines "penalty"--

 

HA!!  A good point for eliminating or at least reducing those @#$%&! questionnaires that the government loves to require the references or the proposers to submit, even for contracts or task orders where there are official past performance evaluations in the .Past Performance Information Retrieval System or other archives.

See also Don's article at https://dap.dau.mil/cop/conmgtblog/archive/2014/03/04/do-you-have-an-omb-control-number-for-that-past-performance-questionnaire.aspx

Share this post


Link to post
Share on other sites

Don, I don't think that the definition of "penalty ", above,  includes downgrading the confidence rating of a firm for not returning those questionnaires. And I don't think that a firm will be successful in protesting a rating based upon an unreasonable requirement to submit the form.  Generally, the GAO will consider such types of protests untimely, right? 

Share this post


Link to post
Share on other sites

joel,

What if the protest is not of the requirement itself, but of the agency's actions in response to failing to comply with the requirement? Wouldn't that be a timely protest?

You may be right that downgrading an offeror would not be a penalty, but a good lawyer may be able to argue that it is. To my knowledge, this argument has never been made in a protest. I'd love to see what a court or the GAO would decide. There are a lot of protests denied because an offeror failed to provide every last piece of paper required by the solicitation. Why not invoke the public protection provision of the Paperwork Reduction Act? I think it's worth a try. I'm probably the only one, though.

Share this post


Link to post
Share on other sites
On 7/25/2017 at 5:22 PM, joel hoffman said:

The key is not to limit the evaluation of past performance in the solicitation evaluation criteria to those @#$%&! questionnaires that the government loves to require the references or the proposers to submit, depending upon the solicitation language.  Provide for flexibility in what and how you can obtain the relevant past performance information to evaluate.

Hear, hear! Use broad, expansive, inclusive language, e.g. "including, but not limited to..." 

Also, don't forget that a "neutral" (actually, neither favorably nor unfavorably) rating for past performance does not provide the same protection during the tradeoff or source selection decision stage. 

Consider firm A and firm B, with evaluation factors of past performance, technical, and price.

Firm A has no past performance, firm B has excellent past performance.

Even if firm A and firm B have the exact same technical rating and underlying strengths and weaknesses, you can determine that firm B is the superior choice, even if firm B has a higher price. Your rationale is that the price premium of $X justifies a lower risk (having excellent past performance instead of no past performance).

Bottom line: Having no past performance shields the rating, but doesn't create "immunity" during the tradeoff and source selection decision.

Share this post


Link to post
Share on other sites
On 7/25/2017 at 9:25 AM, dkubis said:

What happens when we sent out the questionnaires and they try to suppress their past performance by not submitting anything in order to get a neutral rating?

In answer to that question:

What can happen is that if you know that they do, in fact, have past performance, and if you can determine how well they performed, you can evaluate them on the basis of the information that you have, no matter what the source. You are not limited to the information that they provide. Are you familiar with the GAO decisions in that regard? See Al Raha Group for Technical Services, Inc.; Logistics Management International, Inc., B-411015.2, 2015 CPD ¶ 134:

Quote

Where a solicitation contemplates the evaluation of offerors' past performance, the agency has the discretion to determine the scope of the performance history to be considered, provided all proposals are evaluated on the same basis and the evaluation is consistent with the terms of the solicitation. Weidlinger Assocs., Inc., B–299433, B–299433.2, May 7, 2007, 2007 CPD ¶91 at 8. An agency is generally not precluded from considering any relevant information, regardless of its source, and is not limited to considering only the information provided within the four corners of an offeror's proposal when evaluating past performance. FAR §15.305(a)(2)(ii); Paragon Sys., Inc., B–299548.2, Sept. 10, 2007, 2007 CPD ¶178 at 8. Here, the RFP also explicitly notified offerors that the pastperformance evaluation would “not [be] limited to review of the information provided in the offeror's Present/PastPerformance volume,” and that the agency would obtain past performance information from other sources, including CPARs available from government past performance databases. RFP at 146.

Now, read that carefully, because the GAO said that an agency is generally not precluded for considering any relevant information, regardless of its source. It did not say that the statements in the RFP were necessary. They were merely fortuitously reinforcing. Don't let anybody tell you different. In Paragon, cited in the quote, GAO said:

Quote

[A]n agency is generally not precluded from considering any relevant past performance information, regardless of its source. See, e.g., NVT Techs., Inc., B–297524, B–297524.2, Feb. 2, 2006, 2006 CPD para. 36 at 5.  

Period.

Share this post


Link to post
Share on other sites
On 7/25/2017 at 3:33 PM, Don Mansfield said:

You may be right that downgrading an offeror would not be a penalty, but a good lawyer may be able to argue that it is. To my knowledge, this argument has never been made in a protest. I'd love to see what a court or the GAO would decide. There are a lot of protests denied because an offeror failed to provide every last piece of paper required by the solicitation. Why not invoke the public protection provision of the Paperwork Reduction Act? I think it's worth a try. I'm probably the only one, though.

Don:

What do you mean by "downgrading"? If an offeror has no record of past performance, then the appropriate evaluation finding is None. In that regard, see SITEC Consulting, LLC; VariQ Corporation; Logistics Systems, Inc., B-413526.4, 2017 CPD ¶ 124, April 3, 2017:

Quote

Although agencies may not rate an offeror that lacks relevant past performance favorably or unfavorably with regard to past performance, an agency may, in a price/technical tradeoff, determine that a high past performance rating is worth more than a neutral past performance rating. American Floor Consultants, Inc., B–294530.7, June 15, 2006, 2006 CPD ¶97 at 5; see CMC & Maint., Inc., B–292081, May 19, 2003, 2003 CPD ¶107 at 4.

Finding that some good is better than none does not constitute downgrading none.

Also, assuming that the past performance questionnaire is considered to be part of the proposal, wouldn't it be covered under the Paperwork Reduction Act by OMB Control No. 9000-0037, which covers FAR Subpart 15.2, "Solicitation and Receipt of Proposals and Information"? Wouldn't putting that OMB control number in the proposal preparation instruction or past performance questionnaire satisfy the display requirement?

Share this post


Link to post
Share on other sites
4 hours ago, Vern Edwards said:

Don:

What do you mean by "downgrading"? If an offeror has no record of past performance, then the appropriate evaluation finding is None. In that regard, see SITEC Consulting, LLC; VariQ Corporation; Logistics Systems, Inc., B-413526.4, 2017 CPD ¶ 124, April 3, 2017:

Finding that some good is better than none does not constitute downgrading none.

Also, assuming that the past performance questionnaire is considered to be part of the proposal, wouldn't it be covered under the Paperwork Reduction Act by OMB Control No. 9000-0037, which covers FAR Subpart 15.2, "Solicitation and Receipt of Proposals and Information"? Wouldn't putting that OMB control number in the proposal preparation instruction or past performance questionnaire satisfy the display requirement?

The "downgrade" would be for failure to comply with solicitation requirements (i.e., the offeror had references, but didn't have them complete and submit surveys).

The information collection approved under OMB Control No. 9000-0037 is for "Presolicitation Notice and Response." See the last FR Notice. If you are referring to the list at FAR 1.106, it is misleading.

Share this post


Link to post
Share on other sites

Don: Sorry, but I still don't understand downgrade.Downgrade from what to what?

Thanks for the reference to OMB 9000-0037. I was referring to FAR 1.106. Why does it refer OMB 9000-0037 to FAR 15.2, since the pre solicitation notice rules are not in that subpart?

How have agencies gotten away with requesting proposals all this time without an OMB Control No.?

Share this post


Link to post
Share on other sites
5 hours ago, Vern Edwards said:

Don: Sorry, but I still don't understand downgrade.Downgrade from what to what?

"Downgrade" probably isn't the right word. What I'm trying to describe is evaluating an offeror unfavorably under an evaluation factor such as "compliance with solicitation requirements" because they didn't bother to send past performance questionnaires to their references.

5 hours ago, Vern Edwards said:

Thanks for the reference to OMB 9000-0037. I was referring to FAR 1.106. Why does it refer OMB 9000-0037 to FAR 15.2, since the pre solicitation notice rules are not in that subpart?

According to the Information Collection Review, the sections affected are "4.5; 14.205; 15.201(c) and 36.213-2."

5 hours ago, Vern Edwards said:

How have agencies gotten away with requesting proposals all this time without an OMB Control No.?

That's a good question. My guess is that 1) Some agencies don't know what their responsibilities are under the Paperwork Reduction Act, 2) Some agencies do know but it's too hard to comply with, 3) some offerors don't know their rights under the PRA, 4) Some offerors do know their rights but don't enforce them because they're trying to win a contract, and 5) nobody really cares.

This article may shed some light on the topic. Avoidance of OIRA (created by the PRA) is not unique to acquisition.

Share this post


Link to post
Share on other sites
11 hours ago, Don Mansfield said:

"Downgrade" probably isn't the right word. What I'm trying to describe is evaluating an offeror unfavorably under an evaluation factor such as "compliance with solicitation requirements" because they didn't bother to send past performance questionnaires to their references.

If an agency requests information about past performance in the form of completed questionnaires, and if an offeror submits no questionnaires, then the proper response by the agency is a finding of "No record" or "No information" or simply 'None" with respect to past performance. The agency should not evaluate them unfavorably on "compliance with solicitation requirements," which would be stupid. "No record," "No information" or "None" would not be an evaluation--an assessment of value. It would be just a statement of fact under the rules of source selection. The GAO does not think that such a procedure constitutes evaluating the offeror unfavorably. When comparing that offeror to one with good past performance, the agency might find the other offeror to be a better value in that regard. It might consider the offeror with none to be better than one that has poor past performance. Those are matters of opinion. I think it would be a very long stretch to call such an opinion a penalty.

A penalty is "the imposition by an agency or court of a fine or other punishment; a judgment for monetary damages or equitable relief; or the revocation, suspension, reduction, or denial of a license, privilege, right, grant, or benefit." The courts have ruled that there is no private right of action against an agency that violates the Paperwork Reduction Act. A person or an organization cannot sue an agency because it did not seek and obtain OMB approval or display an approval number. A person or an organization can cite an agency's violation only as a defense against a "penalty." The key to the success of what you recommend would be whether an attorney could persuade a tribunal that a finding of "None" or "No record" or "No information," or that a consensus of opinion about which of two offerors is better, would, in and of itself, constitute a penalty.

I have not studied the PRA and the implementing regulations closely at all, but it seems to me that in order to comply agencies would have to seek approval of each and every RFP on an individual basis, since the burden associated with various RFPs will be very different. I doubt that class approval would comply. I think that such individual approvals would be impracticable, and it appears that OMB has interpreted the law as not covering RFPs. That is probably within its discretion in interpreting and applying the law. Congress could take action if it disagrees, and it has not done so. So...

Share this post


Link to post
Share on other sites
5 hours ago, Vern Edwards said:

I think it would be a very long stretch to call such an opinion a penalty.

Just to be clear, I did not call such an opinion a penalty. In my hypothetical, failure to distribute questionnaires to references would have no bearing on an evaluation of past performance--it would affect a factor such as "compliance with solicitation requirements." 

Suppose an agency issues an RFP that requires an offeror to submit past performance information in the form of a list of contracts and a brief description of the work required under each. For each contract identified, the offeror is required to send a PP questionnaire to a POC for each contract. The agency does not comply with the display and notice requirements contained in 5 CFR 1320.3(f) and 1320.3(b) when requesting the information. The RFP contains an evaluation factor for "compliance with solicitation requirements."

An offeror submits an offer with the list, but does not send any of the questionnaires to its POCs. The agency now has to contact the POCs to obtain the information that would have been in the questionnaire. Accordingly, the offeror gets an unfavorable rating for "compliance with solicitation requirements." Putting aside the issue of whether it would be stupid in doing so, it would be justified under the terms of the RFP. I think that an argument can be made that the agency's actions constituted a "penalty" under the PRA. Note that under "Obligation to Respond" in the Information Collection for Past Performance Information: Responses During Source Selection it states "Required to Obtain or Retain Benefits." So, it seems that a contract is viewed as a "benefit".

6 hours ago, Vern Edwards said:

I have not studied the PRA and the implementing regulations closely at all, but it seems to me that in order to comply agencies would have to seek approval of each and every RFP on an individual basis, since the burden associated with various RFPs will be very different. I doubt that class approval would comply. I think that such individual approvals would be impracticable, and it appears that OMB has interpreted the law as not covering RFPs. That is probably within its discretion in interpreting and applying the law. Congress could take action if it disagrees, and it has not done so. So...

I don't agree with that assessment. A given RFP contains dozens of information collections. Some of these have been cleared by OMB, some have not. Most of the entries on the list at FAR 1.106 are provisions and clauses. A representation or certification will typically have an associated OMB Control Number. Most forms in the FAR/DFARS display an OMB Control Number if they require an offeror to complete them. In fact, there is an OMB Control Number for collecting past performance information during source selection--9000-0142--but I doubt anybody is complying with the display and notice requirements contained in 5 CFR 1320.

Share this post


Link to post
Share on other sites
8 hours ago, Don Mansfield said:

I don't agree with that assessment.

With which specific statement(s) do you disagree? I made several. Please clarify.

Share this post


Link to post
Share on other sites
On 7/30/2017 at 1:57 PM, Vern Edwards said:

With which specific statement(s) do you disagree? I made several. Please clarify.

I disagree that OMB has interpreted the law as not applying to RFPs. The PRA regulations expressly apply to RFPs. 5 CFR 1320.3( c )(1) states:

Quote

A “collection of information” may be in any form or format, including the use of report forms; application forms; schedules; questionnaires; surveys; reporting or recordkeeping requirements; contracts; agreements; policy statements; plans; rules or regulations; planning requirements; circulars; directives; instructions; bulletins; requests for proposal or other procurement requirements; interview guides; oral communications; posting, notification, labeling, or similar disclosure requirements; telegraphic or telephonic requests; automated, electronic, mechanical, or other technological collection techniques; standard questionnaires used to monitor compliance with agency requirements; or any other techniques or technological methods used to monitor compliance with agency requirements. A “collection of information” may implicitly or explicitly include related collection of information requirements.

I also disagree that agencies would necessarily have to get OMB approval for each individual RFP. An RFP contains dozens of information collections--some approved by OMB, some may not be. If an RFP contained an information collection that was not approved by OMB, then approval would be required for that specific information collection.

Share this post


Link to post
Share on other sites
On 7/30/2017 at 3:27 PM, Don Mansfield said:

In my hypothetical, failure to distribute questionnaires to references would have no bearing on an evaluation of past performance--it would affect a factor such as "compliance with solicitation requirements."

Wouldn't you have to include an RFQ term to the effect that  "offerors who have been determined to possess relevant past performance, but who fail to submit questionnaires, shall be smote accordingly" in order to address your hypothetical ?   What is to stop them from simply not submitting a list of prior contracts?  No list; no failure to distribute questionnaires.

How a contracting agency would unilaterally determine that relevant PP exists, when the offeror fails to identify it on their own, is a question for another day.

Share this post


Link to post
Share on other sites
45 minutes ago, REA'n Maker said:

Wouldn't you have to include an RFQ term to the effect that  "offerors who have been determined to possess relevant past performance, but who fail to submit questionnaires, shall be smote accordingly" in order to address your hypothetical ?   What is to stop them from simply not submitting a list of prior contracts?  No list; no failure to distribute questionnaires.

I don't think you'd have to be that explicit. The solicitation could just say that respondents who fail to follow the instructions herein will be evaluated unfavorably under "compliance with solicitation requirements".

Share this post


Link to post
Share on other sites

If everyone is doing their jobs correctly the past performance information will be in PPIRS.   The contractor cannot claim he has none.  However, not everyone is getting all their reports into the system.  My Agency is about 22%.   Everryone says they have no time.  Then on the evaluation side, only "relevent" past performance can be considered and their is plenty of protest cases over that.  What if it for work in a different NAICS but there is 5 reports in PPIRS (each lower value) that are sat and below.  Why should that not be "relevent".   Between the CPAR system being too segregated in who can submit reports, password difficulties with the system, and too many protests over the PP evals, the whole system needs to be revamped.  

Share this post


Link to post
Share on other sites
On 8/3/2017 at 2:32 PM, REA'n Maker said:

Wouldn't you have to include an RFQ term to the effect that  "offerors who have been determined to possess relevant past performance, but who fail to submit questionnaires, shall be smote accordingly" in order to address your hypothetical ?   What is to stop them from simply not submitting a list of prior contracts?  No list; no failure to distribute questionnaires.

How a contracting agency would unilaterally determine that relevant PP exists, when the offeror fails to identify it on their own, is a question for another day.

That's why you have a separate factor for recent, relevant experience.  No experience submitted - should lead to a low rating under a factor that addresses "experience". Unlike "past performance", lack of relevant experience can be reflected in a lower rating. 

AND - you don't limit the evaluation of past performation in the solicitation evaluation criteria to those confounded questionnaires.

Ask for a reference for every contract or project submitted under experience - on YOUR form that they use to provide the information in YOUR format.  If you don't otherwise have knowledge of their performance rating, call the reference for an interview - again, using YOUR format.  

If you intend to consider experience and past performance, then you will need to take the time to do it.  I don't think it was that difficult when I was running source selections.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×