Jump to content

Highest Technically-Rated, Reasonable Price/Offer (HTRO) and Responsibility Determination


govt2310

Recommended Posts

Highest Technically-Rated, Reasonable Price/Offer (HTRO), also called HTRFP or HTRRP, is an evaluation method where the offeror must self-score its own proposal.  My question is, how does HTRO work with responsibility-type evaluation factors and small businesses?  If they self-score themselves out of contention, so they are found unacceptable for a factor that would make them also "not responsible," does the agency still have to send it to SBA for COC?

Link to comment
Share on other sites

38 minutes ago, govt2310 said:

Highest Technically-Rated, Reasonable Price/Offer (HTRO), also called HTRFP or HTRRP, is an evaluation method where the offeror must self-score its own proposal.

Self-scoring is not an essential feature of the highest-technically-rated with a fair and reasonable price method of source selection.

Link to comment
Share on other sites

57 minutes ago, govt2310 said:

My question is, how does HTRO work with responsibility-type evaluation factors and small businesses?  If they self-score themselves out of contention, so they are found unacceptable for a factor that would make them also "not responsible," does the agency still have to send it to SBA for COC?

An offeror cannot determine its own responsibility or nonresponsibility. Only a CO may do that.

If a CO determines an offeror to be nonresponsible based on the offeror's own self-scoring sheet, and if that is the only thing keeping the otherwise successful offeror from receiving an award, then the CO would have to refer the matter to the SBA in accordance with FAR 19.602.

I have not yet seen any bid protest in which a CO determined an offeror to be nonresponsible based on the offeror's own self-scoring sheet, but there may be one or more that I have missed.

Link to comment
Share on other sites

If the RFP instructions require the offeror to self-score itself on a point system for the non-price factors (which, for this example, include traditional responsibility factors such as Past Performance), and the agency will only evaluate the proposals that are at or above a numerical cut-off point, could the agency argue that, an offeror that self-scores itself below that cut-off point, that does not mean that offeror's proposal is "technically unacceptable"?  So therefore, the agency has not found the offeror not capable to perform the requirement, which means the agency is not required to contact the SBA for a COC?  I'm saying that, in a HTRO RFP that involves a self-scoring sheet, that is not a situation where the evaluation factors are pass/fail.  Rather, the RFP involves a numerical point system.

 

Link to comment
Share on other sites

If the solicitation specifies that the agency will only evaluate the proposals that are at or above a numerical cut-off point, then every offeror will know before submitting whether they are above or below the line, right?  If an offeror self-scores itself below the line, there seems no reason to even submit its proposal -- maybe it should protest before offers are due if it thinks the approach is unfair -- but if it does not protest and submits its protest, maybe it has forfeited its right to complain later when it is not selected?

These kind of hypotheticals are hard, and maybe even unfair -- no one here has read the text of the real or hypothetical solicitation.  Hard decisions need to be made based on facts.  We don't know your facts.

  • Is this a single-award or a multiple-award scenario?
  • Will the agency validate the scores of the highest-self scored offers, or simply rely on them as submitted?
  • If the agency will validate the scores, and one of the highest self-scores is lowered by the agency's evaluation to where it is lower than another offer who was not initially evaluated, will that lower offer be brought into the evaluation pool?
  • Is there a numerical point cut-off for each factor, or is there a single cut-off for total points?  If the latter, then maybe the agency will posit that it is not treating past performance on a pass-fail basis but rather is using it in its racking and stacking for selection -- In such a case, maybe referral to SBA for a small business unsuccessful offeror is inappropriate.

Part of the problem with COCs is that practitioners sometimes overlook fundamental correct principles.  The fundamental correct principle is that if we select a small business offeror as the winner based on the evaluation factors, and then find that offeror to be non-responsible, we involve the SBA's COC process -- but only for that one offeror --  we do not refer all the other unsuccessful small business offerors.  I hope the contracting officer who issued your solicitation is aware of this basic, fundamental correct principle and that he planned the evaluation and selection process accordingly.

 

 

Link to comment
Share on other sites

Well, if you're doing "highest-technically-rated with a fair and reasonable price" in a multiple-award acquisition, the best way to avoid the COC process is to not find any offeror to be nonresponsible, either on an actual or de facto pass/fail basis. Instead, say that a low score in comparison with other offerors may position the offeror outside the set of the highest-technically-rated and, therefore, not among the prospective awardees.

During source selection planning, you must establish a rational procedure for determining the dividing line between the set of highest-technically-rated offerors and everyone else. But I would not establish a predetermined (before receipt of proposals) cut-off score. That might be considered arbitrary, "mechanical", or noncompetitive.

Link to comment
Share on other sites

 

Yes, assume this is a solicitation for a multiple-award contract.

Yes, what I want to know is how to structure the solicitation to avoid having to refer offerors to the SBA for a COC.

Hmm, so if the agency does not establish a pre-determined cut-off score, but instead, says that there will be a comparison done, that would avoid the need to do a COC?  Ok, thank you.  I will have to think about how to carefully phrase this comparison language.

Link to comment
Share on other sites

  • Is this a single-award or a multiple-award scenario? YES
  • Will the agency validate the scores of the highest-self scored offers, or simply rely on them as submitted? AGENCY WILL VALIDATE THE SCORES
  • If the agency will validate the scores, and one of the highest self-scores is lowered by the agency's evaluation to where it is lower than another offer who was not initially evaluated, will that lower offer be brought into the evaluation pool?  YES
  • Is there a numerical point cut-off for each factor, or is there a single cut-off for total points?  If the latter, then maybe the agency will posit that it is not treating past performance on a pass-fail basis but rather is using it in its racking and stacking for selection -- In such a case, maybe referral to SBA for a small business unsuccessful offeror is inappropriate.  THERE IS A NUMERICAL CUT-OFF FOR THE TOTAL POINTS, NOT FOR EACH FACTOR
Link to comment
Share on other sites

I realize that FAR 19.602 says that the CO only has to get a SBA COC for the "apparent awardee."  But if the RFP used "responsibility-type factors" in the evaluation factors, then doesn't the CO have a duty to get a SBA COC for each of the small business offerors that did not make it to the "apparent awardee" stage because the agency evaluated their proposals as unacceptable regarding a responsibility-type factor?  In essence, the agency has made a finding of non-responsibility for those offerors.

Link to comment
Share on other sites

49 minutes ago, govt2310 said:

But if the RFP used "responsibility-type factors" in the evaluation factors, then doesn't the CO have a duty to get a SBA COC for each of the small business offerors that did not make it to the "apparent awardee" stage because the agency evaluated their proposals as unacceptable regarding a responsibility-type factor?

You're making this too hard.  You cannot submit every unsuccessful small business offeror for a COC -- if you did, you would be obliged to award a contract to each of them for whom the SBA grants a COC -- that would be an absurd outcome in a single-award scenario, and the rules are written for single award scenarios.  You are in a multiple-award situation, so you make it fit.

Correct principle: You refer the apparent awardee for a COC if, after evaluation and selection, you later disqualify that offeror for non-responsibility before award.  You do not refer all unsuccessful offerors, and you do not refer those who don't make it to the "apparent awardee" stage, as you styled it.

The COC process does not apply when an offeror is not selected based on a comparative and subjective analysis, even when a responsibility-type factor is used.  Don't think in terms of pass-fail for your factors -- instead, think in terms of good-better-best -- think in terms of selecting the best (or the highest technically rated) multiple awardees considering all the factors (or all of the non-price factors) together.

If you select the best offerors using some subjective and/or objective comparison of offers, you will avoid the problem you are imagining.  You create your own problem when you try to combine the technical evaluation and the responsibility review.  And you create your own problem when you try to have a mechanical selection process instead of a human and subjective comparison and selection.  An offeror with weak past performance might still be selected by your human SSA if it excels under the other factors.  Weak past performance is not disqualifying in a tradeoff.  There is a difference between an offeror being uncompetitive and that offeror being disqualified or ineligible -- you may be conflating these.

After you select your apparent awardees, you will still need to determine them to be responsible before award.  The responsibility review will be different than the evaluation/selection review.  The responsibility review occurs after the SSA has made the best-value selections.

I used more words than Vern did -- he was right, but it didn't work for you, so I explained more at length.  I hope this is helpful.

You may be carrying some baggage of incorrect principles, such as thinking that all small business offerors not making it to the "apparent awardee" stage must be referred to the SBA for a COC if a responsibility-type factor is used.  Shake off that baggage!  Let correct principles be your guide.

Link to comment
Share on other sites

1 hour ago, govt2310 said:

Then why did GAO rule the way it did in this case: https://www.gao.gov/products/b-413104.10?

The agency set a competitive range.  The protester was excluded.  There is no mention of being in line for award.  Why?

Why? Note that in the case the GAO cited the Small Business Act, 15 USC 637(b)(7)(A), which says:

Quote

[The SBA] is empowered, whenever it determines such action is necessary... (7)(A) To certify to Government procurement officers, and officers engaged in the sale and disposal of Federal property, with respect to all elements of responsibility, including, but not limited to, capability, competency, capacity, credit, integrity, perseverance, and tenacity, of any small business concern or group of such concerns to receive and perform a specific Government contract. A Government procurement officer or an officer engaged in the sale and disposal of Federal property may not, for any reason specified in the preceding sentence preclude any small business concern or group of such concerns from being awarded such contract without referring the matter for a final disposition to the Administration.

and the SBA's implementing regulation, 13 CFR 125.5(a)(2)(ii), which says:

Quote

(2) A contracting officer must refer a small business concern to SBA for a possible COC, even if the next apparent successful offeror is also a small business, when the contracting officer... (ii) Refuses to consider a small business concern for award of a contract or order after evaluating the concern's offer on a non-comparative basis (e.g., a pass/fail, go/no go, or acceptable/unacceptable) under one or more responsibility type evaluation factors (such as experience of the company or key personnel or past performance)....

Note that the GAO did not cite FAR 19.602-1, which makes no mention of that rule. The FAR coverage is inadequate. A CO must be aware of the SBA's rule. See Canfield, "Traditional Responsibility Factor Doctrine: From Interpretive Gloss To Regulatory Enshrinement," Procurement Lawyer, Summer 2021:

Quote

Traditional responsibility factor doctrine appears to have begun as an interpretive gloss on the intent of the Small Business Act's conferring of authority on the SBA to determine whether small businesses are competent to perform work for the federal government. Over the years, that gloss has evolved, but it has remained a fairly fixed principle in federal procurement law, culminating in its enshrinement in the SBA's regulations. There is a wealth of statutory, regulatory, and decisional law that outlines and refines the governing principles, and practitioners in all areas of federal procurement would do well to familiarize themselves with it. As the foregoing discussion demonstrates, however, there still remain questions to be answered about the doctrine's application.

FAR 19.602-1 makes it seem like COs must refer a small business for a COC only when they have determined the apparently successful offeror to be nonresponsible. But that is not true.

Also see FAR 15.101-2:

Quote

If the contracting officer elects to consider past performance as an evaluation factor, it shall be evaluated in accordance with 15.305. However, the comparative assessment in 15.305(a)(2)(i) does not apply. If the contracting officer determines that a small business’ past performance is not acceptable, the matter shall be referred to the Small Business Administration for a Certificate of Competency determination, in accordance with the procedures contained in subpart  19.6 and 15 U.S.C.637(b)(7)).

The problem is that the same is true of any responsibility-type evaluation factor.

Link to comment
Share on other sites

2 hours ago, govt2310 said:

Then why did GAO rule the way it did in this case: https://www.gao.gov/products/b-413104.10?

Because, in the GAO's own words,

Quote

In sum, the agency evaluated the small business offerors on an acceptable/unacceptable basis, as opposed to a comparative basis, with respect to relevant experience, a responsibility-type evaluation factor, and found CRS’s proposal unacceptable under that factor.  See 13 C.F.R. § 125.5(a)(2)(ii).  As such, rejection of CRS’s proposal based on its rating of unacceptable under the management approach, domain-specific capability in a health-related mission subfactor, without first referring the matter to the SBA for a COC determination, was improper.  Accordingly, we sustain the protest. 

That agency evaluated on a pass-fail basis.  If you want to avoid that outcome for your procurement, then don't evaluate on a pass-fail basis.  Instead, consider using a comparative basis.  It's easy.  

From the decision--

Quote

Here, the solicitation provided that in phase 1 of the evaluation, proposals were to be evaluated on a go/no-go basis, i.e., non-comparative basis.  See RFP at M-3, M-4.  The solicitation further provided that if rated unacceptable under any factor in phase 1, the entire proposal would be rendered unacceptable and ineligible for award.

So, you should consider using a comparative basis (instead of a non-comparative basis) -- and instead of finding the proposal unacceptable and ineligible for award, simply observe that it does not provide the best value among the competitors.  Only the best go to the next phase.  The loser is not unacceptable or ineligible, it just isn't among the best.  Problem avoided.

 

Link to comment
Share on other sites

3 hours ago, ji20874 said:

So, you should consider using a comparative basis (instead of a non-comparative basis) -- and instead of finding the proposal unacceptable and ineligible for award, simply observe that it does not provide the best value among the competitors.  Only the best go to the next phase.  The loser is not unacceptable or ineligible, it just isn't among the best.  Problem avoided.

 


You cannot do "comparative" with FAR Part 15.  I know it can be done under FAR 16.505, but not FAR Part 15. 

Link to comment
Share on other sites

1 hour ago, ji20874 said:

You most assuredly can do an evaluation and selection on a comparative basis under FAR Part 15.

More specifically 15.101-2(b)(1)  - I know not on point to HTRO but a can do.

15.305(a)(2) and this quote which I believe others emphasized in this thread - "This comparative assessment of past performance information is separate from the responsibility determination required under subpart  9.1."

15.308!

Link to comment
Share on other sites

3 hours ago, govt2310 said:


You cannot do "comparative" with FAR Part 15.  I know it can be done under FAR 16.505, but not FAR Part 15. 

@govt2310That comment was one comment too many. Your ignorance of the rules proves you to be unqualified for the acquisition under discussion. You are in over your head, and I am embarrassed for you. Do yourself a favor and leave the thread. It's dead anyway.

Link to comment
Share on other sites

7 hours ago, govt2310 said:


You cannot do "comparative" with FAR Part 15.  I know it can be done under FAR 16.505, but not FAR Part 15. 

I believe this comment is based on common guidance from policy offices and other authoritative sources. The Procurement Innovation Lab (PIL) stops short of saying that comparative evaluations are prohibited and states that comparative evaluations are “Not recommended for use under FAR part 15.”

I believe the problem comes from the lack of a standardized definition and usage of the term. After all, it should be clear that comparisons are commonly used if not necessary in any competitive acquisition … FAR 15.308 and the DOD SSP expressly talks about comparative assessments and analyses.

So what do you mean when you say ‘comparative’?

Link to comment
Share on other sites

6 hours ago, Jamaal Valentine said:

So what do you mean when you say ‘comparative’?

I don't know what govt2310 meant, but it doesn't matter. FAR Part 15 does not prohibit comparative evaluation.

Here is one explanation of "comparative":

In general, an agency can evaluate offerors and their offers ("proposals") by comparison to a standard or by direct comparison to each other.

For the sake of illustration, suppose that an evaluation factor is "distance of the contractor's office from the work site," the lesser the distance the greater the value. Suppose further that you decide to rate offerors on a numerical scale of 1 to 100 points, 100 being best.

One way to rate offerors and offers would be to establish a distance/points rating standard (scale), compare each offeror and offer to the standard, assign points accordingly, and then rank them by comparing the ratings and underlying facts.

A different way would be to compare the offerors directly to each other, giving the closest offeror 100 points, and then giving every other offeror fewer points on a proportional basis. So, to use a simple example, assume that you receive offers from five offers, the closet being five miles away and that the farthest being 36 miles away. You could assign the five-mile offeror 100 points and the 36-mile offeror 1 point and score those in-between based on the linear function.

Once upon a time some agencies—most notably the Air Force—prohibited evaluation by direct comparison, because they feared that the best offeror and offer might not be particularly good in relation to the agency's requirement. When you're conducting a source selection based on proposed weapon system design concepts, one design concept might be better than all the others, but not particularly good in terms of desired weapon system performance. (Keep in mind that the process we call "source selection" was developed for use in the acquisition of aeronautical, nautical, and space systems.) So the Air Force developed the concept of evaluation standards and required the evaluation of each offeror by comparison to the standards, not to each other. See Air Force Regulation 70-15, Source Selection Policies and Procedures (1976), para. 3-4b:

Quote

The evaluation of proposals is conducted at the factor level under each item and compared against measurable objective standards, and not by comparing proposals to each other.

That Air Force policy was adopted by some other agencies, but prohibitions against evaluation by direct comparison were eventually abandoned by most as the use of source selection expanded to a wider range of acquisition types after the enactment of CICA in 1984.

The prohibition has never appeared in statute. It did not appear in the Armed Services Procurement Regulation, the Defense Acquisition Regulation, or the Federal Procurement Regulation, and it has never appeared in the FAR.

Evaluation of offerors and offers by direct comparison is a perfectly legal and reasonable method to use for many source selections, especially those for commercial products and services in which evaluation should be based on what is available in the market. Evaluation by comparison to standards is much more challenging, because the development of evaluation standards is challenging.

In any case, every source selection decision ultimately requires the direct comparison of offerors and their offers, whether they were evaluated by direct comparison or comparison to standards.

14 hours ago, govt2310 said:

You cannot do "comparative" with FAR Part 15.  I know it can be done under FAR 16.505, but not FAR Part 15. 

Whatever govt2310 meant by that, it is wrong.

Link to comment
Share on other sites

To Jamaal Valentine: thank you.  Yes, I was just getting ready to post the link to the DHS PIL Boot Camp Workbook that says FAR 15 is not supposed to involve "comparative" evaluation.  You are correct, the DHS PIL does not say that one cannot use it with FAR 15, it just says it is "not recommended."  See pages 13 of 44, 22 of 44 in the Workbook at https://www.dhs.gov/publication/pil.

Link to comment
Share on other sites

11 hours ago, Jamaal Valentine said:

I believe this comment is based on common guidance from policy offices and other authoritative sources. The Procurement Innovation Lab (PIL) stops short of saying that comparative evaluations are prohibited and states that comparative evaluations are “Not recommended for use under FAR part 15.”

This comparison is inapt.  The PIL encourages a technique called comparative evaluation (or something like that) where a direct comparison of proposals without assigning adjectival ratings is appropriate -- that is the key to the PIL technique, no adjectival ratings, just head to head comparison.  What we're talking about in this thread is not that.  Yes, the PIL encourages adjectival ratings for formal Part 15 tradeoff source selections because that is so ingrained and the PIL doesn't want to rock that boat too much -- but for tradeoffs in simplified acquisitions or ordering situations where there are only a few factors and a few proposals, the PIL encourages a direct offeror-to-offeror comparison and ranking without adjectival ratings.

Every Part 15 tradeoff is done on a comparative basis -- usually with qualitative adjectival ratings rather than pass/fail ratings.  LPTA is pass/fail.

Link to comment
Share on other sites

1 hour ago, govt2310 said:

You are correct, the DHS PIL does not say that one cannot use it with FAR 15, it just says it is "not recommended."  See pages 13 of 44, 22 of 44 in the Workbook at https://www.dhs.gov/publication/pil.

The phrase "not recommended" does not appear on PILS Workbook page 13. However, that page seems to suggest that FAR Part 15 requires the use of ratings and precludes the use of evaluation by direct comparison. 

PILS Workbook page 22 says that the "comparative evaluation" technique is:

Quote

Ideal for task/delivery orders under FAR subpart 8.4 and § 16.505, but also for part 13 simplified acquisitions (incl. subpart 13.5 for commercial items up to $7 Million). Not recommended for use under FAR part 15. 

The Workbook doesn't provide any rationale for saying that evaluation by direct comparison without ratings is not recommended under FAR part 15. So no critical thinker would concur with that "not recommended."

FAR Part 15 does not require the use of ratings or scores. The PILS workbook is wrong to suggest that it does. Ratings have never been required by procurement statute or by any procurement regulation in the Code of Federal Regulations. There are several good reasons to discard ratings. One reason is to eliminate protests about the assignment of ratings.

The purpose of ratings is to simplify and facilitate the aggregation of evaluation findings. (The best way to do that is with numerical ratings, not adjectives or colors.) See "Scoring or Rating in Source Selection: A Continuing Source of Confusion," The Nash & Cibinic Report, February 2006. See also "Source Selection Decisions: Ratings Should Not Be Used," The Nash & Cibinic Report, April 2018. If you don't use too many evaluation factors, there is no reason to assign ratings.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...