Jump to content
The Wifcon Forums and Blogs

Sign in to follow this  
Deaner

SB Set-Aside LPTA Evaluation Factors

Recommended Posts

I had a brainstorming function the other day where we discussed what evaluation factors we would want to use in a few construction requirements. The usual evaluation factors that seem to be universally used came up such as Résumés, Experience, Past Performance, Bonding etc. The RFP will be set-aside for small businesses and the evaluation method is going to be LPTA.

What’s noticeable about the evaluation factors we came up with is that all but one of them could be tied to one of the general standards of responsibility at 9.104-1, which would require the referral of any lowest priced offeror rated technically unacceptable to the SBA for a certificate of competency.

I don’t take issue referring matters to the SBA, but at the same time I also don’t really like the idea the SBA gets the final say or force me to go through a different process if I disagree.

I’m not really looking to solve any problem but more out of curiosity. Whether actually applied or thought up off the top of your head, what are others using for evaluation factors for LPTA (construction or other) solicitations set-aside for small businesses? I did briefly look at a few solicitations posted to FBO and it makes me believe evaluation factors that amount to matters of responsibility are widely used.

Share this post


Link to post
Share on other sites

Why not try tradeoff with price as your most important factor?  You have some flexibility and don't have to rate a factor acceptable or unacceptable.  

Having said that, what is truly critical for this particular project?  What particular experience, for instance, would be necessary to qualify? 

EDIT : Deleted why one might require several responsibility factors in an LPTA SB set-aside. The question wasn't asked.

Share this post


Link to post
Share on other sites
42 minutes ago, joel hoffman said:

Before busting your noggin,

Please don't hold back. I'm a grown man and can take any butt kicking you have to dish out. 

I understand i could use bids, trade-offs, creative eval factors, etc.But that's not what i'm trying to get at. 

experience, we're talking about 7-15 mil project, which probably equates to 3-7 projects in a lifetime. Idea is experience might be 10 years, which is 3 or so projects from start to completion.

Share this post


Link to post
Share on other sites

Maybe the better question is, why use LPTA instead of sealed bidding in an LPTA environment set-aside for small businesses? Off the top of my head, i want to hold discussions. 

Share this post


Link to post
Share on other sites
57 minutes ago, joel hoffman said:

I don't remember what other factors we used other than telling us what features they would self perform and how they would manage the construction project to meet their limitations on subcontracting. 

We do that too. The regs don't always refer to self perform any more tho, they now mention "similarly situated" which is more or less the same thing telling us in a different way 15%. 

Share this post


Link to post
Share on other sites

If you don't have anything to evaluate other than responsibility type factors, why are you planning to do an RFP? 

Share this post


Link to post
Share on other sites

When conducting a competition involving a set aside, it is not uncommon to use responsibility factors as evaluation factors (e.g. past performance, personnel  résumés). However, one cannot avoid the requirement for a responsibility determination and, if necessary, a certificate of competency, if the factor is evaluated on a “pass/ fail” basis and if the small business contractor’s quote/ proposal is determined to have failed.

Quote

Under the Small Business Act, agencies may not find a small business nonresponsible without referring the matter to the SBA, which has the ultimate authority to determine the responsibility of small businesses under its COC procedures. 15 U.S.C. § 637(b)(7); FAR subpart 19.6; Federal Support Corp., B-245573, Jan. 16, 1992, 92-1 CPD ¶ 81 at 4. Past performance traditionally is considered a responsibility factor, that is, a matter relating to the offeror’s ability to perform the contract. See FAR § 9.104-1(c); Sanford & Sons Co., B-231607, Sept. 20, 1988, 88-2 CPD ¶ 266 at 2. Traditional responsibility factors may be used as technical evaluation factors in a negotiated procurement, but only when a comparative evaluation of those areas is to be made. See, e.g., Medical Info. Servs., B-287824, July 10, 2002, 2001 CPD ¶ 122 at 5; Nomura Enter., Inc., B-277768, Nov. 19, 1997, 97-2 CPD ¶ 148 at 3. Comparative evaluation in this context means that competing proposals will be rated on a scale, relative to each other, as opposed to a pass/fail basis. Docusort, Inc., B-254852, Jan. 25, 1994, 94-1 CPD ¶ 38 at 6. We have cautioned that an agency may not disqualify a small business under the guise of a relative assessment of responsibility-based technical factors in an attempt to avoid referral to the SBA. Federal Support Corp., supra, at 4; Sanford & Sons Co., supra, at 3.

Quote

9 The result would be the same had the agency evaluated the bids on a lowest-priced technically acceptable basis. Where an agency utilizes a lowest-priced technically acceptable source selection process, the FAR provides that past performance need not be an evaluation factor at all. However, when it is included, it cannot be utilized for the purpose of making a “comparative assessment”; rather, past performance is to be determined solely on a pass/fail basis. FAR § 15.101-2. Our Office has long held that pass/fail evaluations of capability issues, such as past performance, are tantamount to responsibility determinations, with the result that a rating of “unacceptable” in these areas is the same as a determination of nonresponsibility. See, e.g., Phil Howry Co., B-291402.3, B-291402.4, Feb. 6, 2003, 2003 CPD ¶ 33 at 6. Consistent with this premise, in the context of a lowest-priced technically acceptable evaluation scheme, where the contracting officer determines that a small business’ past performance is not acceptable, “the matter shall be referred to the Small Business Administration for a Certificate of Competency determination.” FAR § 15.101-2(b)(1). Although the agency states that this procurement was conducted under the provisions of FAR part 13, we think that the COC requirements apply here as well. In this regard, the COC referral requirements of FAR part 19 require agencies to make a referral whenever an agency finds an apparently successful small business offeror nonresponsible. FAR § 19.602-1. As discussed above, our Office views past performance as a responsibility criterion where it is used as a pass/fail factor.

FitNet Purchasing Alliance File, B-410263, Nov. 26, 2014. http://www.gao.gov/assets/670/667235.pdf

See also this decision involving an IFB and an assessment of personnel résumés: Coastal Environmental Group, Inc., B-407563; B-407563.3; B-407563.4, Jan. 14, 2013.

http://www.gao.gov/assets/660/651544.pdf

Quote

Finally, Coastal also argues that in negotiated procurements, our Office has treated personnel experience and qualifications requirements as a matter of technical acceptability, akin to a question of responsiveness, and that therefore we must likewise view the key personnel experience requirements in this IFB as a matter of bid responsiveness. We again disagree. Our Office treats questions of responsibility consistently under sealed bidding and under negotiated procurement procedures. See Docusort, Inc., B-254852, Jan. 25, 1994, 94-1 CPD ¶ 38 (when a minimum management experience requirement, a traditional responsibility factor, is evaluated on go/no go basis, rejection of proposals solely on the basis of the offeror’s failure to comply with such a factor must be referred to SBA); Paragon Dynamics, Inc., B-251280, Mar. 19, 1993, 93-1 CPD ¶ 248 at 4 (failure on the part of the offeror's proposed software development engineer to meet the minimum experience and capability requirements concerned a responsibility-related factor); ASR Mgmt. & Tech. Servs., B-244862, B-247422, Apr. 23, 1992, 92-1 CPD ¶ 383 at 7 (noting that, although personnel qualifications are a “responsibility-type factor,” when they are evaluated on a comparative basis, they may properly be considered in an agency’s selection decision). In addition, the Small Business Act requires referral to SBA of questions of responsibility even where negotiated procurement procedures are used. See, e.g., Phil Howry Co., B-291402.3, B–291402.4, Feb. 6, 2003, 2003 CPD ¶ 33 at 5.

 

Share this post


Link to post
Share on other sites

If the construction is relatively complex, why not use a Most Highly Qualified with a Fair and Reasonable Price (MHQ/FRP) process, a variation of GSA's Highest Technically Rated with a Fair and Reasonable Price process? MHQ/FRP would allow you to evaluate experience, past performance, and other responsibility-type factors comparatively in order to rank offerors based on their qualifications. In that way you could bypass the SBA's certificate of competency procedure, but you wouldn't have to make nonprice/price tradeoffs. You would evaluate price fairness and reasonableness on a pass or fail basis.

Properly designed, the MHQ/FRP process would allow you to analyze the most highly qualified offeror's performance cost estimate in depth in order to determine whether it reflects a sound understanding of the construction specifications and drawings and negotiate appropriate price adjustments in order to ensure a fair and reasonable price prior to award. If agreement could not be reached on a fair and reasonable price with the most highly qualified offeror, you could enter into negotiations with the second best offeror.

MHQ/FRP would be better than both LPTA and sealed bidding, but would require greater professional knowledge and skill.

If the construction is simple, just use sealed bidding.

Share this post


Link to post
Share on other sites

Vern, taking the original post at face value,  it appeared that price was the most important factor. If they are contemplating LPTA, one would think that they don't want to pay one penny more to get a better contractor or better products. 

That seems to be the polar opposite of your proposed approach.

it is difficult to suggest evaluation factors without knowing anything about the cost/quality/schedule objectives,  the complexity, or what they need to evaluate for conformity or minimum qualifications. 

Deaner indicated that this is not necessarily a specific project but a general  discussion about factors to use for LPTA. 

Share this post


Link to post
Share on other sites

Joel:

The OP wanted to evaluate responsibility-type factors without having to deal with SBA's COC program, so I told him one way to do it. End of story.

Share this post


Link to post
Share on other sites

Deleted several reasons for using certain responsibility factors in set-asides for construction using LPTA .The question wasn't asked.

 

Share this post


Link to post
Share on other sites

@Deaner

12 hours ago, Deaner said:

Please don't hold back. I'm a grown man and can take any butt kicking you have to dish out. 

Deaner:

Okay. Based on the above I know that you won't be offended if I say that telling us that you are asking about "a few construction requirements" worth $7 to $15 million isn't telling us much worth knowing. Those could include construction of a fully-specified storage facility, renovation of an asbestos-laden office building with no current as-built drawings, construction of a test facility involving installation of special equipment, renovation of a small but historically important local courthouse, construction of a tactical operations center in Djibouti under unknown subsurface conditions, construction of a tactical aircraft runway or revetments in Afghanistan, renovation of a visitors center at a national wildlife refuge,  construction of a public restroom/shower facility at a national park, etc., etc. The appropriate evaluation factors might vary considerably among such different projects.

@All:

Asking a good question requires as much if not more thought than providing a good answer.

No offense meant, Deaner. Hopefully, none taken. But that will be as it may.

Share this post


Link to post
Share on other sites
5 hours ago, joel hoffman said:

If you don't have anything to evaluate other than responsibility type factors, why are you planning to do an RFP? 

I kind of want to reserve the right to hold discussions.

 

3 hours ago, Vern Edwards said:

Most Highly Qualified with a Fair and Reasonable Price (MHQ/FRP) process

I honestly don't have much confidence trying this with the office I'm working with on these upcoming requirements. The last time we solicited using trade-offs with this office the technical evaluation I received back looked like it was done with my kid's Brick Red Crayola, and the consensus ratings were a struggle where I ended up having to just make the final decision more or less by myself as the SSA.

 

1 hour ago, Vern Edwards said:

No offense meant, Deaner. Hopefully, none taken. But that will be as it may.

None Taken and you make a good point.

The construction requirements range from new buildings, renovations of existing buildings constructed in the 20's with no as-builts. Required to achieve two green globes or silver Leed, some involve demo. They're all different, I don't want to get into too much detail as I think it's getting away from the general question I intended to ask.

I think at the end of the day my question was, is it common to use responsibility type evaluation factors, which Napolik points out, is not uncommon.

Share this post


Link to post
Share on other sites

Yes, it is not uncommon to use responsibility factors in RFP's for the types of projects that you mentioned. 

Share this post


Link to post
Share on other sites
20 hours ago, Deaner said:

What’s noticeable about the evaluation factors we came up with is that all but one of them could be tied to one of the general standards of responsibility at 9.104-1, which would require the referral of any lowest priced offeror rated technically unacceptable to the SBA for a certificate of competency.

Deaner,

Have you concluded that it doesn't make sense to use a responsibility-type factor to determine technical acceptability when using LPTA?

Share this post


Link to post
Share on other sites
1 hour ago, Vern Edwards said:

It's easier to do sealed bidding with special responsibility factors

Probably, but I still don't know that I'm ready to give up the ability to hold discussions if I want to.

 

56 minutes ago, Don Mansfield said:

Have you concluded that it doesn't make sense to use a responsibility-type factor to determine technical acceptability when using LPTA?

I don't know if I've drawn that conclusion or not yet. I was going to post a follow up question for why responsibility-type factors are used when using LPTA with a SB set-aside, but I wanted to ponder the idea a little longer first. My initial thought was if it's not that uncommon to do, why is it done? technical acceptability and responsibility can be based on the same thing was my first answer to myself, but I don't have concrete answer to your question just yet.

Share this post


Link to post
Share on other sites
47 minutes ago, Deaner said:

 for constructionI don't know if I've drawn that conclusion or not yet. I was going to post a follow up question for why responsibility-type factors are used when using LPTA with a SB set-aside, but I wanted to ponder the idea a little longer first. My initial thought was if it's not that uncommon to do, why is it done? technical acceptability and responsibility can be based on the same thing was my first answer to myself, but I don't have concrete answer to your question just yet.

I explained above in two separate posts "why" some responsibility factors are used when using LPTA with an SB set-aside for construction. Oh well,  never mind. 

Share this post


Link to post
Share on other sites
6 hours ago, Deaner said:

Probably, but I still don't know that I'm ready to give up the ability to hold discussions if I want to.

If you want to? Are you that lonely and/or bored?

Share this post


Link to post
Share on other sites

I don’t exactly know how to respond to Joel’s and Gordon’s comments without just pissing them off. They don’t add any value to me or anyone else. This theme I see of the senior guys here complain that people don’t think or ask the right questions or just do what has been done previously, and it's a problem. I guess my new question is why do you think 1102’s knowledge of these types of things have gone downhill? Did you senior guys create that problem? 

I can find a way to get past my problem, but it still doesn’t answer my question. Sure I can move forward, but if I don’t understand the answer to my question, even though I found a way to move forward, I still have the question.

I have some good responses in this thread, but I also got a response that obviously was from someone who got butt-hurt because they think I had gotten a response that should have answered my questions, and that person responded like a child by editing/deleting their response. 

 

Share this post


Link to post
Share on other sites
On 3/6/2017 at 4:05 PM, Deaner said:

Whether actually applied or thought up off the top of your head, what are others using for evaluation factors for LPTA (construction or other) solicitations set-aside for small businesses? I did briefly look at a few solicitations posted to FBO and it makes me believe evaluation factors that amount to matters of responsibility are widely used.

That's the original question. Let's try to approach this analytically.

Evaluation factors are attributes of a thing to be evaluated. In an acquisition there are generally two things to be evaluated: (1) the offer (promises) being made and (2) the firm that's making the offer. There are two categories of evaluation factors that correspond to those two things: (a) offer factors and (b) capability factors.

Offer factors. Generally, the offer (promises) being made has two main components: (a) the "technical" (nonprice) promises about what the offeror will do and refrain from doing and when it will finish doing it and (b) the promise to keep its promises in return for a stated price. In construction, the technical offer is usually evaluated for conformity with the terms of the solicitation, i.e., acceptability. This is done on a pass or fail basis. The price is evaluated for relative amount--it must be the lowest received for acceptable technical promises--and it must also be a reasonable amount.

Capability factors. As for capability factors, they are usually attributes of the firm as a business entity. Such factors traditionally include experience, reputation ("past performance"), financial status (including bonding capacity), sufficiency of labor and capital (e.g., construction equipment), management ability (usually demonstrated through past performance and qualifications of key personnel), licensing, and special capabilities (e.g. hazardous waste management and disposal).

Deaner is asking about capability ("responsibility type") factors to be used in small business set-asides. Traditionally, in sealed bidding, these are evaluated after identification of the prospective contractor as part of a determination of responsibility in accordance with FAR Subpart 9.1. However, in LPTA they can be evaluated as a part of the source selection itself, in accordance with FAR Subpart 15.3. The most commonly used factors are experience, past performance, and management ability. However, for certain kinds of construction, when special construction skills, licenses, or other qualifications are needed, the evaluation includes possession of those licenses, skills, and qualifications. In some cases, possession of a security clearance is a factor.

As pointed out by napolik, the rules established by the GAO are that when such capability factors are evaluated on a pass or fail basis, as they are in LPTA source selection, a determination to rate an offeror as unacceptable on the basis of one or more of them is tantamount to a determination of non responsibility and thus the SBA Certificate of Competency program applies.

With the exception of financial status, the status of a firm as a small business has, in my experience, little to do with the choice of capability factors. I think the presumption sometimes is that a small businesses will have less capability than a large business, and while that may be true in terms of financial and bonding capacity, I do not know of any factual basis for making a general assumption in that regard. In short, I don't think there are special factors to be used in small business set-asides. Others might disagree.

@Deaner:

As I told you in an earlier post, the responsibility-type (capability) factors used in an LPTA will (should) reflect the nature of the project to be completed, the conditions of performance, and the amount of work the agency wants to do during source selection. The more factors and the more complex the factors, the more work. You shouldn't evaluate everything you could evaluate. You should evaluate only those capabilities with regard to which the offerors are likely to differ in meaningful ways. Some factors are very hard for contracting offices to evaluate, like financial status, and thus are generally not used. Others, like experience and past performance, are used in virtually every LPTA source selection for construction.

If I were conducting an LPTA source selection for construction of a 100 mile long 500 kV transmission line across a mountain range, my most important responsibility-type factors would be experience, past performance, and special management know-how, because that is specialized work and in difficult terrain. If I were conducting an LPTA source selection for construction of a nuclear waste containment facility at Hanford, I would look for experience, past performance, special management know-how, and any requisite special licensing and security clearance. If I were building a restroom/shower facility at a national park camp ground I would look for nothing but experience and past performance. Others might make different choices.

If you have looked at solicitations posted to FBO, then you have seen some of the factors used. Keep in mind that some usages are based more on tradition or cut-and-paste than on requirements analysis.

Think of the problem this way: In the construction industry, what are the typical attributes (features, qualities, characteristics) of a firm that does the kind of construction that we want done? On which of those attributes are the firms likely to differ significantly? Which of those attributes are important to me? Those are the ones you'll want to evaluate.

If I still haven't answered your question, above, then I don't know what you're looking for and can't do anything more for you.

Vern

 

Share this post


Link to post
Share on other sites
7 hours ago, Deaner said:

I can find a way to get past my problem, but it still doesn’t answer my question. Sure I can move forward, but if I don’t understand the answer to my question, even though I found a way to move forward, I still have the question.

Perhaps there is a chance that your question will be answered once you move forward. Such experiences, good or bad, stick with us longer than an answer in a message board thread.

Share this post


Link to post
Share on other sites

Vern,

Thank you for the very detailed response. I appreciate you taking your time to provide it.

 

1 hour ago, Gordon Shumway said:

Such experiences, good or bad, stick with us longer than an answer in a message board thread.

I agree with that, but when you have to dig through the whining or smart ass comments, well, for me at least, it will make me think twice before participating in a discussion on a website. I'll turn to other places for discussions. If I want to listen to those types of things I can just home to it.

 

I appreciate everyone's input on this topic. I have nothing further to discuss on this matter at this time.

Share this post


Link to post
Share on other sites
33 minutes ago, Deaner said:

when you have to dig through the whining or smart ass comments

It's an acquired, yet necessary skill as an 1102. God speed in your career, Deaner.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×