Jump to content
The Wifcon Forums and Blogs

FrankJon

Wanted: Critical Feedback for Evaluation Method

Recommended Posts

Wifcon Community - I am seeking critical feedback for my planned evaluation approach. I want to ensure that I have my ducks in a row before approaching Legal with this. This method is based upon a mix of Vern's past posts and the Sevatec holding. Thank you in advance for your assistance.

OBJECTIVE: To utilize an evaluation scheme that is more streamlined, intuitive, and flexible than trade-offs, without increasing protest risk. If this works as intended, I would apply it (or some variation) to most of my future service procurements competed under FAR 8.4, 13, or 16.5.

CONTEXT: Competitive BPA call pursuant to 8.4. Four FTEs to provide consulting and management support services. Utilizes PWS and exceeds SAT.

EVALUATION METHOD:

Factor 1: Quote acceptability (compliance w/ T&Cs, in-scope, etc.). (Pass/Fail)

Factor 2: Personnel qualifications. (Pass/Fail)

Factor 3: Risk - Combination of personnel experience and contractor past performance. (I advised against including PP being that this is a BPA call, but my customer insisted.) Factor three will be evaluated through direct comparisons of quotes. Quotes will be ordered from lowest to highest performance risk.

Price will not be a factor, since the Government is willing to pay up to the ceiling rates.

Award will be made to the contractor that passes Factors 1 and 2, presents the least amount of performance risk, and quotes a F&R price.

Share this post


Link to post
Share on other sites

As a follow-up, my biggest challenge with this has been wrapping my head around the "direct comparison" technique. After years of conducting trade-offs against objective criteria, it's difficult to switch gears and do something that is antithetical to everything I was taught.

I would especially be interested in hearing about the experiences and advice of others in this regard.

Share this post


Link to post
Share on other sites

I am going to only address your specific example.  By my read you are contemplating  multiple GSA-FSS BPA’s and the intent is to place a  process the BPA's to fairly consider quotes to award a  call against the BPA’s.  

 

Also assuming  that your anticipated Calls will be in excess of the SAT.  With this in mind your proposed approach is way beyond FAR 8.405-3(c)(2)(iii)(A)(3) in my view which simply says this (emphasis added)…

 

 “(3) Fairly consider all responses received and make award in accordance with the selection procedures.”

 

            I applaud your attempt to think of applying an innovative idea but quite honestly you have much more leeway to “utilize an evaluation scheme that is more streamlined, intuitive, and flexible than trade-offs” already.   You just need to fairly consider the quotes you would get to finalize the call.

           

            So what do you need to do to fairly consider the quotes?  Well you could use your approach but it in my view does attempt to utilize a process that is almost like FAR Part 15.   You use words like factor, pass/fail and risk suggest this.    Instead of your approach why not this….

 

Go here… http://www.wifcon.com/discussion/index.php?/topic/3940-quotes-vs-proposals/ ….read the thread and then think about the post Napolik made on April 11 and consider how you might easily convert the ideal into a fairly considering quotes for the Call and how you might state selection process in the BPA's.

 

 

Share this post


Link to post
Share on other sites

C Culham - Thank you for your reply. The BPAs are already established; this is a competition I am holding among five BPA-holders. While I don't think I agree that terms like "factor, pass/fail and risk" make this "almost like FAR Part 15," I do agree that there is likely room to make this even simpler, yet that may not be feasible at this time.

I think it might be helpful if I explain that in my office, it's just me (and perhaps a couple of other quiet warriors) trying to convince about 130 other 1102s and several attorneys to do anything other than LPTA or trade-offs. I am not a supervisor, nor have I ever held a warrant. Few in my office would dare to do something that could be seen by OGC as "cutting corners," thereby increasing protest risk (case-in-point: the award decision for the BPA I am competing this requirement under is 200+ pages). Other than Wifcon, I really don't have many outlets for testing ideas, pushing boundaries, or trying to effect cultural change. I am grateful that I was able to convince my current CO to try something new, but this is uncharted territory for him, too. I am doing my best to persuade others, but without any senior 1102s to guide me or provide cover, incremental improvements may be the best I can hope for.

Share this post


Link to post
Share on other sites

Somewhat related tangent: The real shame is that DAU doesn't teach this stuff as part of its core curriculum. The vast majority of their content is focused on non-commercial FAR 15 procedures. I can't tell you how many time I've raised my hand in class to clarify how an issue might change in a commercial environment, and was met with blank stares from the instructors. We are constantly being told by senior DoD leaders and policy directives to "innovate" and to work "smarter not harder," yet our formal education does nothing to reinforce these maxims. Instead, entry- and journeymen-level 1102s are often forced to put themselves out on a limb in order to try something different. 

Share this post


Link to post
Share on other sites
40 minutes ago, FrankJon said:

The BPAs are already established

FrankJon - Noting the quoted fact I am at an impasse in trying to provide my thoughts absent what the language is in the BPA's about selecting contractors for Calls.   I will say I am concerned you might be trying to use an innovative process when something else is already stated in the BPA's as to what the process should be.

Share this post


Link to post
Share on other sites

C Culham - There is nothing in the BPA terms that stipulates the evaluation process for call orders, apart from "IAW FAR 8.504-3."

Share this post


Link to post
Share on other sites
19 minutes ago, FrankJon said:

evaluation process for call orders, apart from "IAW FAR 8.504-3."


FrankJon, is that an error in the original document, or did you mean to type "IAW FAR 8.405-3"?

Assuming the latter, does that mean your evaluation will be constrained by FAR 8.405-3(c)(2)(iii)?

FAR 8.405-3(c)(2)(iii):

"(iii) Orders exceeding the simplified acquisition threshold.

(A) The ordering activity shall place an order in accordance with paragraphs (c)(2)(iii)(A)(1), (2) and (3) of this paragraph, unless the requirement is waived on the basis of a justification that is prepared and approved in accordance with 8.405-6. The ordering activity shall—

(B) The ordering activity shall document evidence of compliance with these procedures and the basis for the award decision.

  • (1) Provide an RFQ to all BPA holders offering the required supplies or services under the multiple-award BPAs, to include a description of the supplies to be delivered or the services to be performed and the basis upon which the selection will be made;

    (2) Afford all BPA holders responding to the RFQ an opportunity to submit a quote; and

    (3) Fairly consider all responses received and make award in accordance with the selection procedures."

 

 

Share this post


Link to post
Share on other sites
57 minutes ago, FrankJon said:

Somewhat related tangent: The real shame is that DAU doesn't teach this stuff as part of its core curriculum.

FrankJon,

What do you mean by "this stuff"? How to compete an order under FSS BPAs?

Share this post


Link to post
Share on other sites

Pepe - That was a transcription error on my behalf. The BPA reference is cited correctly as "8.405-3."

Yes, I will have to follow the procedures for BPA calls in excess of the SAT. Is there a particular sentence within that reference that you want to draw my attention to?

Share this post


Link to post
Share on other sites

Don - I am referring to "streamlined" or "simplified" evaluations under 8.4, 13, or 16.5. I cannot recall ever discussing an evaluation method other than LPTA or trade-offs within a DAU classroom. In my experience, the vast plurality (and possibly majority) of the curriculum, beginning with CON 090, is focused on FAR 15.

Share this post


Link to post
Share on other sites
4 hours ago, FrankJon said:

OBJECTIVE: To utilize an evaluation scheme that is more streamlined, intuitive, and flexible than trade-offs, without increasing protest risk.

What's wrong with trade-offs? Under the source selection procedures for the parts you mentioned, the evaluation process should already be streamlined. If it isn't, perhaps that would be a good place to start with "innovation." This approach might be easier to sell to management, and then if successful, would give you credibility to expand into the area on the continuum outside of LPTA and trade-off.

Also, use caution when structuring evaluation factors as pass/fail when small businesses are involved.

Share this post


Link to post
Share on other sites

Gordon - The problem with trade-offs for an 8.4 procurement is that the method is typically a lot of writing and difficult for the customer to write to (this is the experience within my office). I believe that moving to "direct comparisons" would allow the customers to write in a way that they're more comfortable with, thereby making the process more efficient.

Share this post


Link to post
Share on other sites

Frank, you're wasting your time here. The method you are asking about is called "simple additive weighting," abbreviated SAW. It isn't rocket science. There are a large number of descriptions, explanations, and examples of applications of it online. Some are technical, some are simple. Google it.

See this, which is for high school students choosing a major. http://ijcttjournal.org/Volume10/number-3/IJCTT-V10P126.pdf

Here's a link to a handy PowerPoint presentation. enginfo.ut.ac.ir/keramati/.../QM/.../Multi-Criteria%20Decision%20Making.ppt

You have to think, Frank. You have to work it out in your own head. You have to get clear on this so you can be clear to others. You won't get anywhere asking a bunch of strangers for "critical feedback" about your sketchy description. DSFHW!!!

But it's your time, so do with it as you please.

Share this post


Link to post
Share on other sites
4 hours ago, FrankJon said:

Price will not be a factor, since the Government is willing to pay up to the ceiling rates.

Why don't you want to evaluate price? I'm asking from a very basic level, ignoring what the FAR says and how price is discussed in the Sevatec case you referenced.

Share this post


Link to post
Share on other sites

Gordon - Because the rates have been capped at the BPA level below the GSA level. We don't want to exert any additional price pressure, which might affect quality.

Vern - Thanks for the recommendation, and perhaps you're right. I feel that I have "DSFHW" (that one took me a second...) and I generally feel good about the approach. I'm simply anticipating my uphill slog, and seeing whether I can rebut critiques.

Share this post


Link to post
Share on other sites

The only way to be able to rebut critiques (and they're coming, you know it) is to be ready to provide a coherent explanation for everything. Study, study, study. Get theory and practice down. Anticipate questions and challenges. Practice your responses.

By the way, you said:

5 hours ago, FrankJon said:

Price will not be a factor, since the Government is willing to pay up to the ceiling rates.

Wrong! Wrong! Wrong! Price is always a factor unless you're buying A-E services. You might not compare prices and make quality/price tradeoffs, but in that case you must make it a factor by determining fairness and reasonableness. That was the Sevatec method.

Study, Frank. Get it right. Mistakes like that will sink you.

Now, please, as your friend, get the hell off Wifcon, go find a quiet corner, and start reading and thinking.

Share this post


Link to post
Share on other sites
36 minutes ago, FrankJon said:

Gordon - Because the rates have been capped at the BPA level below the GSA level. We don't want to exert any additional price pressure, which might affect quality.

Real quick, before you head to that corner, remember, the objective is to obtain the best value for the government.

If you can get a contractor that is only slightly more risky than the least risky offeror (assuming no ties), yet offers a significantly lower price, you'll have a hard time convincing leadership and OGC that trade-off wouldn't yield a better value. Like Mr. Edwards said, the key "is to be ready to provide a coherent explanation for everything." It's not about being right or wrong, or being the most knowledgeable. It's about convincing risk adverse people (with no real incentive to take a risk) to put themselves out there for your approach. And yes, mistakes will sink you.

God speed.

Share this post


Link to post
Share on other sites
On ‎5‎/‎1‎/‎2017 at 8:17 AM, FrankJon said:

EVALUATION METHOD: 

Factor 1: Quote acceptability (compliance w/ T&Cs, in-scope, etc.). (Pass/Fail)

Factor 2: Personnel qualifications. (Pass/Fail)

 

You might keep in mind that if you “fail” a small business concern on a go/no-go factor for one or more responsibility type evaluation factors (such as key personnel), as opposed to evaluating the concern on a comparative basis, that you will likely need to refer the concern to SBA for consideration for a COC.   You won't end up with a streamlined approach in that event.

Share this post


Link to post
Share on other sites

Gns & Gordon - I appreciate you pointing this out. I was not familiar with this case or the broad interpretation of "responsibility" within the applicable USC section.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×