Jump to content
The Wifcon Forums and Blogs

Recommended Posts

Can you use subjective criteria when conducting an Lowest Price Technically Acceptable (LPTA) acquistion?  For example, could one of the evaluation criteria in Section M be:

Offeror's technical approach will be evaluated on a Pass/Fail basis.  The technical approach will be evaluated to determine whether it is technically sufficient in that it provides an approach that will accomplish the tasks stated in the Performance Work Statement within the timeframe specified. 

As this acquisition is in the pre-solicitation phase I cannot really give any details as to the type of work being performed but suffice it to say that I would like the TEP to be able to review the proposals and evaluate them on a pass/fail basis using this criteria.  My respected legal counsel states that this is too subjective and cannot be used for an LPTA approach.

Thoughts? 

Share this post


Link to post
Share on other sites
16 minutes ago, Desparado said:

Can you use subjective criteria when conducting an Lowest Price Technically Acceptable (LPTA) acquistion?  

Have you seen anything in any statute, regulation, or other official publication that says you cannot? If you haven't, why are you asking? I'm sure you are familiar with the guiding principles of the FAR System.

Did your legal say that you cannot use subjective criteria or did he say subjective but mean unclear or indefinite? If he said subjective he may have meant vague. Then again, he may not have thought before speaking and really doesn't know what he means, in which case you should go back and ask him to clarify. Vague and subjective are not the same.

Share this post


Link to post
Share on other sites

To answer your first question, I haven't and the reason I am asking is that Legal is throwing up a potential roadblock because she feels that in a pass/fail situation the criteria needs to be plainly stated and crystal clear.  I feel that our TEP should be given the lattitude to evaluate the offers with their knowledge and make a determination whether the proposed method is technically acceptable or not (in accordance with the performance criteria of the PWS).  Granted, vague and subjective are not the same, so do you feel it is too vague?

She felt that it was not specified enough as to what the offeror would have to do in order to meet that bar and get a "pass".  She felt that an offeror could come back and say that, "well, I proposed to do it this way and you failed it but I believe it is sound and so I will protest", which of course any contractor can do.  She is concerned that if they did we are at risk because we didn't specify precisely enough where that pass/fail line is. 

Share this post


Link to post
Share on other sites
14 hours ago, Desparado said:

Offeror's technical approach will be evaluated on a Pass/Fail basis.  The technical approach will be evaluated to determine whether it is technically sufficient in that it provides an approach that will accomplish the tasks stated in the Performance Work Statement within the timeframe specified. 

As this acquisition is in the pre-solicitation phase I cannot really give any details as to the type of work being performed but suffice it to say that I would like the TEP to be able to review the proposals and evaluate them on a pass/fail basis using this criteria.

 

13 hours ago, Desparado said:

I feel that our TEP should be given the lattitude to evaluate the offers with their knowledge and make a determination whether the proposed method is technically acceptable or not (in accordance with the performance criteria of the PWS).

Those statements lack details. They're too vague. The problem with them is not that they say you want the evaluators to be free to decide subjectively. The problem is that the statements do not say what attributes (features, qualities, characteristics) of the offerors and of their "approaches" the evaluators will look for when making their decision.

Suppose that you are recruiting for someone to manage a company that provides a specific kind of service and that you are inviting people to submit resumes. Suppose further that you say you are going to evaluate candidates based on whether or not you think they can do the job. One prospective candidate asks: What qualities will you be looking for in a candidate? You say: I'll be looking for whether or not they can do the job.

That statement does not tell prospective candidates whether they must have certain academic credentials, professional certifications, general managerial experience, specific technical experience, a security clearance, a certain kind of reputation, certain mental and physical attributes, a certain health history, etc. The statement does not tell prospective candidates enough to enable them to decide whether their prospects are any good and whether they should bother to apply. It does not tell them enough to prepare a suitable resume.

See FAR 15.304(d).

I think maybe what your lawyer is saying is not that your statements are too subjective, but that they don't provide enough information. Whether you would lose a protest because of it is anybody's guess.

See DM Petroleum Operations Company, GAO B-409004.5, 2014 CPD ¶ 52, Jan. 15, 2014:

Quote

A solicitation must inform offerors of the basis for proposal evaluation by stating all significant evaluation factors and their relative importance. Federal Acquisition Regulation (FAR) §15.605(d)(1); Israel Aircraft Indus., Ltd, MATA Helicopters Div., B–274389 et al., Dec. 6, 1996, 97–1 CPD ¶41 at 6–7. However, a solicitation need not identify each element to be considered by the agency during the course of the evaluation where such elements are intrinsic to, or reasonably subsumed within, the stated evaluation factors. AT & T Gov't Solutions, Inc., B–406926 et al., Oct. 2, 2012, 2013 CPD ¶88 at 7; Marine Animal Prods. Int'l, Inc., B–247150.2, July 13, 1992, 92–2 CPD ¶16 at 6, 15.

See also Gentex Corp. v. U.S., 58 Fed.Cl. 634, 652, Dec. 10, 2003:

Quote

The law requires that evaluation factors and significant subfactors be clearly stated in an RFP, including the statement of the relative importance of such factors and subfactors. 41 U.S.C. § 253a(b)(1); FAR 15.304(d); ITAC, 51 Fed.Cl. at 348; Analytical Research Tech., Inc. v. United States, 39 Fed.Cl. 34, 44 (1997). Further, as this Court has recognized, “making offerors aware of the rules of the game in which they seek to participate is fundamental to fairness and open competition.” Dubinsky v. United States, 43 Fed.Cl. 243, 259 (1999). Here, the rules of the game were muddied, especially for Gentex, by virtue of the lack of clarity in the RFP....

Although I don't know the details, I think your lawyer is giving you good general advice.

Share this post


Link to post
Share on other sites

If price is apparently the most important factor, how about using a trade off with price being significantly more important than the non-price factors? You can then use a technical capability/ risk rating scale(s) in the evaluation. 

You will stress price but then may be a bit more subjective in the capability rating and comparison. 

Share this post


Link to post
Share on other sites
3 hours ago, joel hoffman said:

You will stress price but then may be a bit more subjective in the capability rating and comparison

Joel:

Why did you say that? What do you mean by "a bit more" subjective?

  • How you measure the quantity (or degree) of subjectivity? What's the scale?
  • Do you think that the tradeoff process somehow permits more subjectivity than LPTA? If so, please explain. Can you point to a regulation or decision that supports that notion?

Share this post


Link to post
Share on other sites

 They would not have to use a pass/fail rating system.

They may be able to establish some minimum technical approach requirements for an "acceptable"rating. Then use a rating system that allows the evaluators to discern strengths, weaknesses, degree of risk, etc. - like other trade-off competitions . Then make a comparative analysis between offerors. This provides some flexibility in selecting the winner, if the government justifies why it would pay some more for better performance capability or technical approach.

Unless one can use mechanical distinctions to rate the strength of a factor, there will often be some subjectivity in the evaluation when using a sliding scale rating system rather than simply pass/fail.

The government wouldn't necessarily have to go through the  SBA's Certificate of Competency procedures if it doesn't select a firm where it doesn't have reasonable confidence that the firm will accomplish the tasks in the PWS within the specified timeframe. 

I didn't and don't intend to have to fully describe or justify the differences between go/no-go LPTA and trade-off processes, when making it clear to industry that price is the most important factor. I was simply suggesting that they might consider using the trade off process. 

Share this post


Link to post
Share on other sites

DOD and Congress appear to discourage use of subjectivity in LPTA when conducting DoD source selections. Here is an excerpt from Appendix C of the current DoD Source Selection Procedures:

Quote

C.1 Introduction

The LPTA process is appropriate when best value is expected to result from selection of the technically acceptable proposal with the lowest evaluated price. LPTAs may be used in situations where the Government would not place any value on a product or service exceeding the Government’s threshold technical or performance requirements and these requirements can be objectively defined in measurable terms. Such situations include acquisitions of commercial or non-complex services or supplies which are clearly and objectively defined. When LPTA is used, the solicitation and the Source Selection Plan must clearly describe the minimum requirements that will be used to determine the acceptability of the proposal. LPTA should not be used when the SSA will be required to make a judgment as to the desirability of one offeror’s proposal versus a competing proposal. Well-defined standards of performance and quality of services must be available to support the use of LPTA. When standards of performance and quality are subjective, or the Government places value on higher quality or performance, another approach should be used. The LPTA process does not permit tradeoffs among price and nonprice factors (see FAR 15.101-2).

http://www.acq.osd.mil/dpap/policy/policyvault/USA004370-14-DPAP.pdf

And Congress isn’t too thrilled with subjectivity in LPTA.See Section 813(b) of the FY 2017 NDAA:

Quote

Not later than 120 days after the date of the enactment of this Act, the Secretary of Defense shall revise the Defense Federal Acquisition Regulation Supplement to require that, for solicitations issued on or after the date that is 120 days after the date of the enactment of this Act, lowest price technically acceptable source selection criteria are used only in situations in which—

(1) [DOD] is able to comprehensively and clearly describe the minimum requirements expressed in terms of performance objectives, measures, and standards that will be used to determine acceptability of offers;

(2) [DOD] would realize no, or minimal, value from a contract proposal exceeding the minimum technical or performance requirements set forth in the request for proposal;

(3) the proposed technical approaches will require no, or minimal, subjective judgment by the source selection authority as to the desirability of one offeror’s proposal versus a competing proposal;

(4) the [SSA] has a high degree of confidence that a review of technical proposals of offerors other than the lowest bidder would not result in the identification of factors that could provide value or benefit to [DOD];

(5) the contracting officer has included a justification for the use of a [LPTA] evaluation methodology in the contract file; and

(6) [DOD] has determined that the lowest price reflects full life-cycle costs, including for operations and support.

Section 813(c) further provides that

Quote

[t]o the maximum extent practicable, the use of [LPTA] source selection criteria shall be avoided in the case of a procurement that is predominately for the acquisition of—

(1) information technology services, cybersecurity services, systems engineering and technical assistance services, advanced electronic testing, audit or audit readiness services, or other knowledge based professional services;

(2) personal protective equipment; or

(3) knowledge-based training or logistics services in contingency operations or other operations outside the United States, including in Afghanistan or Iraq.

The DAR Council is working a case to implement Section 813 - Case Number 2017-D017. We should know shortly just what DARsters will do. Here is the status:

Quote

01/11/2017 DARC Director tasked Contract Placement Cmte. to draft proposed DFARS rule. Report due 03/01/2017. Report due date extended to 06/21/2017.

 

Share this post


Link to post
Share on other sites
8 hours ago, joel hoffman said:

 They would not have to use a pass/fail rating system.

They may be able to establish some minimum technical approach requirements for an "acceptable"rating. Then use a rating system that allows the evaluators to discern strengths, weaknesses, degree of risk, etc. - like other trade-off competitions . Then make a comparative analysis between offerors. This provides some flexibility in selecting the winner, if the government justifies why it would pay some more for better performance capability or technical approach.

Unless one can use mechanical distinctions to rate the strength of a factor, there will often be some subjectivity in the evaluation when using a sliding scale rating system rather than simply pass/fail.

The government wouldn't necessarily have to go through the  SBA's Certificate of Competency procedures if it doesn't select a firm where it doesn't have reasonable confidence that the firm will accomplish the tasks in the PWS within the specified timeframe. 

I didn't and don't intend to have to fully describe or justify the differences between go/no-go LPTA and trade-off processes, when making it clear to industry that price is the most important factor. I was simply suggesting that they might consider using the trade off process. 

I asked what you meant by "a bit more subjective" and you gave me the above. I don't understand that response. What's a "mechanical distinction"? Are all pass/fail assessments merely the making of mechanical distinctions?

However, I do understand your last sentence and we can let it go at that.

 

Share this post


Link to post
Share on other sites
3 hours ago, napolik said:

The DAR Council is working a case to implement Section 813 - Case Number 2017-D017. We should know shortly just what DARsters will do.

Yep. But it's not clear to me what the law is supposed to do. When using LPTA, technical acceptability can refer to the acceptability of (a) the attributes of the supply or service being procured, (b) the attributes of the offeror, or (c) to both. In light of that, consider:

Quote

Not later than 120 days after the date of the enactment of this Act, the Secretary of Defense shall revise the Defense Federal Acquisition Regulation Supplement to require that, for solicitations issued on or after the date that is 120 days after the date of the enactment of this Act, lowest price technically acceptable source selection criteria are used only in situations in which—

(1) [DOD] is able to comprehensively and clearly describe the minimum requirements expressed in terms of performance objectives, measures, and standards that will be used to determine acceptability of offers;

[Does "minimum requirements" refer just to the supply or service that is being procured or does it include the qualifications of the offeror? Does it include, say, past performance?]

(2) [DOD] would realize no, or minimal, value from a contract proposal exceeding the minimum technical or performance requirements set forth in the request for proposal;

[Same question, plus:. (1) Does that mean that DOD can never use LPTA when one offeror might provide something better than another? (2) How does that apply to evaluation factors relating to attributes of the offeror as opposed to attributes of the supply or service being procured?]

(3) the proposed technical approaches will require no, or minimal, subjective judgment by the source selection authority as to the desirability of one offeror’s proposal versus a competing proposal;

[What if the agency is not going to evaluate technical approach? What if it is only going to evaluate experience and past performance?]

(4) the [SSA] has a high degree of confidence that a review of technical proposals of offerors other than the lowest bidder would not result in the identification of factors that could provide value or benefit to [DOD];

[I'm not sure what that means, since when using LPTA agencies must ignore differences in any such factors.]

(5) the contracting officer has included a justification for the use of a [LPTA] evaluation methodology in the contract file;

[Okay. No biggie.]

and (6) [DOD] has determined that the lowest price reflects full life-cycle costs, including for operations and support.

[What does that mean with reference to services? What's the life-cycle cost of a service?]

What does this law do? Does it prohibit subjective assessments of acceptability?

The distinction between subjective and objective is that the first is a matter of judgement and opinion about the thing being observed, while the second is a matter of verifiable observation of the attributes of the thing being observed, without consideration of judgement and opinion. Does the law prohibit the use of judgement and the consideration of opinion when determining technical acceptability? If that was the goal, why not just say so?

DOD agencies may use LPTA only when (a) the technical acceptability of the supplies or services offered and (b) the qualifications of the competing offerors are to be matters of only empirically verifiable observation and measurement.

It will be interesting to see what the DAR Council comes up with. My bet is that they will simply repeat the statute.

Share this post


Link to post
Share on other sites

Thanks, Napolik.

Share this post


Link to post
Share on other sites

Thank you all for your comments.  My goal (apparently ill-fated) was to have a PWS that states clearly what the objectives of the project are and for the offerors to provide how they would achieve those objectives.  I do not want to compare one approach to another and am not willing to pay more for one approach versus another, but merely wanted the TEP to be able to evaluate whether an offeror's approach will meet those objectives on a pass/fail basis.  Regardless the approach, if it was technically acceptable, they would get a "pass" and then the award would go to the lowest priced offer that achieved a pass rating. 

Share this post


Link to post
Share on other sites
6 hours ago, Desparado said:

My goal (apparently ill-fated) was to have a PWS that states clearly what the objectives of the project are and for the offerors to provide how they would achieve those objectives. I do not want to compare one approach to another and am not willing to pay more for one approach versus another, but merely wanted the TEP to be able to evaluate whether an offeror's approach will meet those objectives on a pass/fail basis.

I think that's perfectly reasonable. All you need to do is provide some details about what facets of each offeror's approach the evaluators will consider when determining whether it will meet your objectives. Come on--that's not hard to, do and it will facilitate evaluation.

Share this post


Link to post
Share on other sites

I agree. Technical approach is often an evaluated proposal aspect. Since a TEP is going to perform the evaluation, y'all ought to be able to develop criteria, as Vern described, some details of what they will consider.

Share this post


Link to post
Share on other sites

Ok, this is where it gets tricky... again, I can't go into specifics but I enjoy the intellectual conversation...  What about something like:

Offeror's technical approach will be evaluated on a Pass/Fail basis.  The technical approach will be evaluated to determine whether it is technically sufficient in that it provides an approach that will accomplish the tasks stated in the Performance Work Statement within the timeframe specified. Specifically, the following will be evaluated on a pass/fail basis:

A) Technical approach - The approach must demonstrate a methodology that is proven successful as demonstrated in the accompanying projects (we have them submit 5 projects within the last 5 years where this approach was used successfully).

B Timeline - The approach must demonstrate the ability to complete the project within XX days as required in the PWS. 

 

Because really, that's all we are looking for... a proven approach that will get the job done within a specific timeframe.

Share this post


Link to post
Share on other sites

I think you will need to provide a bit more detail as to the contents of the tech approach and the definitions/ characteristics of pass/ fail (e.g. staffing adequacy (i.e. types and volume of labor for tasks; personnel degree/ experience requirements); security clearances; source of resources (i.e. prime or subcontracts).

The elements of the tech approach must be mirrored in the pass/ fail criteria. 

Share this post


Link to post
Share on other sites

How about something this:

In determining the likely effectiveness of an offeror's technical approach on a pass or fail basis, the evaluators will review the offeror's two-page descriptions of each of the following:

  • organizational management and supervision structure,
  • staffing policies and procedures,
  • cost and schedule estimating procedures,
  • work assignment and monitoring procedures,
  • budget and schedule monitoring procedures, and
  • quality control/quality assurance procedures.

The evaluators will consider whether the proposed procedures are realistic and sufficient for the work to be done.

 

Share this post


Link to post
Share on other sites

Thank you Vern.  Although some of those do not apply to the project being acquired, it definitely points me in the right direction. This really isn't about staffing so much as it is about the technical approach (the actual method they will use to accomplish the task).  I appreciate everyone's input. 

Thank you.  

Share this post


Link to post
Share on other sites

How is that performance-based? (I'm assuming that your reference to a performance work statement means that you intend to use performance-based acquisition).

Share this post


Link to post
Share on other sites

Speaking of LPTA evaluations, look at the GAO decision issued today, B-414531, FreeAlliance.com, LLC, June 19, 2017:  

Quote

Proposals were to be evaluated under three factors:  technical capability, past performance, and price.  RFP at 138.  The technical capability factor was further divided into three subfactors:  (1) understanding and compliance with requirements; (2) certification; and (3) management and staffing plan.  Relevant here, under subfactor 1, understanding and compliance with requirements, proposals were required to “address each work area in sufficient detail to demonstrate a clear understanding” of the Performance Work Statement (PWS).  RFP at 138 (emphasis added).  The RFP also noted that, in evaluating proposals, the agency would assess whether the proposal described “a sound technical approach” as to “how the offeror will fulfill each requirement in the PWS.”  Id. (emphasis added).

Subjectivity had to have entered this evaluation.

http://www.gao.gov/products/B-414531

 

Share this post


Link to post
Share on other sites

Don - The contractor is proposing their approach.  We are merely evaluating that it will be technically sufficient.  I see no issue with it still being performance-based simply because we want them to tell us how they are going to do it up-front.

Share this post


Link to post
Share on other sites
22 minutes ago, Desparado said:

Don - The contractor is proposing their approach.  We are merely evaluating that it will be technically sufficient.  I see no issue with it still being performance-based simply because we want them to tell us how they are going to do it up-front.

Yes, but you said that their approach would be binding. Performance-based acquisition means an acquisition structured around the results to be achieved as opposed to the manner by which the work is to be performed. If the offeror's approach is binding, wouldn't the contract require that the work be performed in a specific manner (i.e., using their proposed "approach")?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×