Jump to content

Crossover between Evaluation Factors


Sam101

Recommended Posts

Hello,

I typically don't allow an offeror to mention anything about corporate experience in their technical approach narrative however I seen many responses to solicitations where the offeror inserts a short blurb in their technical approach narrative basically validating their proposed approach by saying that they have done this in the past and it worked out well for their clients, but then what is the corporate experience section for? If they wanted to reference that xyz was done for a past customer can they just put "see corporate experience #1"? But then the evaluator will have to look at technical approach and corporate experience holistically (i.e. together) wouldn't they? If so how can the Government say that they gave the Technical Approach an individual rating of Good if they had to read the corporate experience section along side the technical approach section to even get that rating? And even if that "validation of how it was done with past customers" is in their technical approach then wouldn't the Government have to ignore it since it's technically not their technical approach but their corporate experience? Or am I overthinking this? To make a long story short I don't see why an offeror would have to validate their technical approach in their technical approach narrative just so the Government will understand their technical approach. Yes, I know technical approach can be an essay writing contest but still, I don't see how blubs about past experience with xyz are needed to describe xyz.

Link to comment
Share on other sites

I think Offeror A's past experience with xyz is very valuable in developing confidence that Offeror A's technical approach is sound.  Generally, I would have more confidence in the soundness of an offeror's technical approach where the offeror has actually successfully done it rather than where an offeror who has not done it.

16 minutes ago, Sam101 said:

To make a long story short I don't see why an offeror would have to validate their technical approach in their technical approach narrative just so the Government will understand their technical approach.

I do not understand this sentence.

Link to comment
Share on other sites

@ji20874  what I mean by "I don't see why an offeror would have to validate their technical approach in their technical approach narrative just so the Government will understand their technical approach" I mean saying "we will sear the steak on both sides and not just on one side" is already understandable enough to where the offeror doesn't have to write "we will sear the steak on both sides and not just on one side, Agency X liked this when we did it for them 5 times last year and it cut their costs by 50%". Everything past the first comma is corporate experience and maybe some self-proclaimed past performance, the only technical approach is "we will sear the steak on both sides and not just on one side"... I see where you're coming from with you "would have more confidence in the soundness of an offeror's technical approach where the offeror has actually successfully done it rather than where an offeror who has not done it" but isn't that what the corporate experience section is for, perhaps their corporate experience can reference their technical approach for clarity? Why have a separate corporate experience evaluation factor if you're allowing offerors to put corporate experience in their technical approach? Unless you want to combine the technical approach section with the past experience section and call it Technical Approach/Corporate Experience I see how that makes sense but in my scenario I have technical approach and corporate experience being different factors each evaluated separately and the RFP says "only put information that belongs in each factor under that factor and it will be ignored if it is not under the proper factor." If I award to Offeror A then Offeror B will be like "well the 5 strengths you gave Offeror A under the Technical Approach factor was really them talking about their corporate experience when they should have been talking about their technical approach, so you should have ignored that information." 

Link to comment
Share on other sites

Requirement: Food service at the federal Supermax prison in Florence, Colorado, including steak for the special Fourth of July meal, and the non-price factors are corporate experience and technical approach.

OFFEROR A

  • Corporate experience:  We do complete food service for three universities, ten county jails, two state prisons, four corporate cafeterias, and one military base, in four different states.  We have an already-existing system of trucks and warehouses.
  • Technical approach:  We will plan nutritious menus (in half of our corporate experience examples, we plan the menus -- and for the other half, the institution plans them), hire quality employees (we're a leader in food service; we are an employer of choice and have the lowest attrition rates in the industry), and so forth.  We will handle all food purchases and food storage using our already-existing system of trucks and warehouses.  For the special Fourth of July meal, we suppose the agency will want to use plastic tableware so we will make sure to tenderize the steaks before searing them on both sides -- we already do this for all of our jail and prison contracts.

OFFEROR B

  • Corporate experience:  We provide snacks for parties at companies, schools, and so forth, in all fifty states and the District of Columbia, and are expanding into full service.
  • Technical approach:  We recognize the crucial importance of nutritious meals that balance flavor and calories and vitamins, and we will hire a freshly-graduated nutritionist to plan complete meals.  We will hire good employees and develop an incentive and retention program based on state-of-the-art research from Professor Jones-Smythe at Harvard University.  We will subcontract for foodstuffs with Sodexho (Sodexho is a French food services and facilities management company headquartered in the Paris suburb of Issy-les-Moulineaux).  We will tenderize the steaks before searing on both sides as we understand this is a best practice in the industry.

For me, Offeror A gets a higher confidence rating than Offeror B for both factors -- and there is no cross-over from one factor to the other.  Both offerors might be proposing an identical technical approach (nutrition, employee retention, foodstuffs, tenderizing, searing on both sides), but I will have far more confidence in Offeror A than Offeror B -- to me, they are not identical in technical approach (they will all get a checkmark on a checklist, but Offeror A's experience adds real value for me).

Link to comment
Share on other sites

@Sam101, you likely are thinking too much about this and caught up in details unnecessarily.  Corporate experience generally asks about similar work performed.  It may not get into specific details about whether or not the proposed technical approach was used or another approach.  It often just says “we’ve done this before many times.” In many instances, technical evaluators just look at the extent of experience and leave it at that unless the solicitation requires details. 
 

Mentioning experience in technical approach validates that this isn’t the first time.  It’s good.

Link to comment
Share on other sites

3 hours ago, Sam101 said:

I seen many responses to solicitations where the offeror inserts a short blurb in their technical approach narrative basically validating their proposed approach by saying that they have done this in the past and it worked out well for their clients, but then what is the corporate experience section for?

@Sam101I have a different take on this than the others who have responded to you. 

If you think there is value in requiring each offeror to describe its proposed "approach" to doing the work (whatever that is—process, procedure, method, technique) then the attributes of the approach itself are what matter and what you should measure or assess, based on appropriate standards.

"Blurbs" about experience are just part of a sales pitch and should have no bearing on the determination of the strengths and weaknesses of the approach. It's the inherent soundness of the approach that should matter. The experience of the user is a separate consideration.

The merits of a horse are determined by the horse's attributes. The merits of the rider are something else.

Link to comment
Share on other sites

59 minutes ago, Vern Edwards said:

The merits of a horse are determined by the horse's attributes. The merits of the rider are something else.

Now this could lead to a very intense discussion.  My attempt is to make it as simple as possible.

One could argue based on experiences that the merits of a horse and a rider are both attributes and something else.

Link to comment
Share on other sites

2 hours ago, Don Mansfield said:

@Sam101What do your proposal instructions say regarding technical approach and corporate experience?

It says this:

L.1 The offeror shall provide the requested information within the designated volume evaluation factor. Any requested information placed outside of the designated location shall not be considered or evaluated under that evaluation factor regardless of whether it is located somewhere else in the proposal. As an example, the offeror shall not refer to any key personnel qualifications, the staffing plan, corporate experience, or past performance in the Technical Approach factor section of its proposal. If the offeror does so, that content will not be considered by the government during the evaluation of the offeror’s Technical Approach.

L.2 Factor 1 - Technical Approach: Offeror shall describe how they will perform the SOW's requirements.

L.3 Factor 2 - Key Personnel and Staffing Plan: Offeror shall provide resumes of the key personnel listed in C.15 and describe their staffing plan.

L.4 Factor 3 - Corporate Experience: Offeror shall describe three past contracts of similar size and scope.

L.5. Factor 4 - Past Performance: Offeror shall provide the attached PPQ to the customers of the three contracts mentioned in the Corporate Experience Factor.

Edited by Sam101
Added Staffing Plan to L.3
Link to comment
Share on other sites

29 minutes ago, Sam101 said:

L2 Factor 1 - Technical Approach: Offeror shall describe how they will perform the SOW's requirements

What are the stated evaluation criteria for this factor? 

Link to comment
Share on other sites

4 hours ago, C Culham said:

One could argue based on experiences that the merits of a horse and a rider are both attributes and something else.

On a ranch, they are components of something else—a working team, which has its own attributes. You can evaluate the horse, or the rider, or the team. Which is it?

In an RFP you must state what it is that you are evaluating. The answer to Sam101's question is that if you state that you are evaluating an "approach", then evaluate it based on its own attributes—the characteristics, features, properties, and qualities that make it what it is.

Link to comment
Share on other sites

On 4/22/2021 at 1:54 PM, Sam101 said:

But then the evaluator will have to look at technical approach and corporate experience holistically (i.e. together) wouldn't they? If so how can the Government say that they gave the Technical Approach an individual rating of Good if they had to read the corporate experience section along side the technical approach section to even get that rating? And even if that "validation of how it was done with past customers" is in their technical approach then wouldn't the Government have to ignore it since it's technically not their technical approach but their corporate experience?

Not according to your proposal submission instructions (bolded question above).

”Any requested information placed outside of the designated location shall not be considered or evaluated under that evaluation factor regardless of whether it is located somewhere else in the proposal. As an example, the offeror shall not refer to any key personnel qualifications, the staffing plan, corporate experience, or past performance in the Technical Approach factor section of its proposal. If the offeror does so, that content will not be considered by the government during the evaluation of the offeror’sTechnical Approach.”

What are the stated evaluation criteria for the L2Factor1 ? 

Edited by joel hoffman
Added details for clarification
Link to comment
Share on other sites

1 hour ago, joel hoffman said:

What are the stated evaluation criteria for the L2Factor1 ? 

The evaluation criteria for technical approach is the government will evaluate the offeror's technical approach to determine the extent to which the offeror demonstrates that thier proposed technical approach will meet the requirements of the SOW, then I have lettered bullet points such as:

a) Serve steaks which are not messed up.

b) Provide steaks on time and within budget.

c) Ability to season the steaks good without too much salt.

If the offeror doesn't have experience with doing this and they rate Outstanding on their technical approach then so be it, they will rate Marginal on Corporate Experience and wouldn't win the contract anyways.

I may do a a combined technical approach/corporate experience factor in the future and see how that works out.

Link to comment
Share on other sites

1 hour ago, Vern Edwards said:

Source selection should be considered a medical specialty. Don't go to podiatrist for brain surgery.

Agree. 

Link to comment
Share on other sites

1 hour ago, Sam101 said:

The evaluation criteria for technical approach is the government will evaluate the offeror's technical approach to determine the extent to which the offeror demonstrates that thier proposed technical approach will meet the requirements of the SOW, then I have lettered bullet points such as:

a) Serve steaks which are not messed up.

b) Provide steaks on time and within budget.

c) Ability to season the steaks good without too much salt.

If the offeror doesn't have experience with doing this and they rate Outstanding on their technical approach then so be it, they will rate Marginal on Corporate Experience and wouldn't win the contract anyways.

I may do a a combined technical approach/corporate experience factor in the future and see how that works out.

There doesn’t appear to me to be any need to describe or evaluate how the firm will perform prescriptive tasks. That’s just regurgitating the requirements. 

Link to comment
Share on other sites

It sounds to me like the OP is focused on process and not outcome.   How is compartmentalizing evaluation factors in the best interests of the government?  If including past experience  in a discussion of technical  approach helps demonstrate understanding  of the requirement, it is a good thing; if it doesn't, the offeror is just wasting valuable proposal space.  That's on them.  A CO's job is to practice good, consistent, business judgement, not just memorize regulations.  As Duke Ellington used to say, "If it sounds good, it is good".  

It seems to me that many procurement professionals are practicing Source Eliminations, not Source Selections.

Link to comment
Share on other sites

19 minutes ago, joel hoffman said:

There doesn’t appear to me to be any need to describe or evaluate how the firm will perform prescriptive tasks. That’s just regurgitating the requirements. 

You don’t need a technical approach factor to evaluate how to perform tasks such as those described.

Simply evaluate whether they have recent, related experience and how well they performed those tasks/contracts.

Link to comment
Share on other sites

Quote

 I don't see how blubs about past experience with xyz are needed to describe xyz.

 

By "Compartmentalizing" I mean treating the factors as being completely distinct and separate, e.g., expecting an offeror to discuss an approach without referring to their past experience with that approach.  That's not a mortal sin. (Edit: including past experience to validate your approach is not a 'mortal sin'.)

You should certainly evaluate PP and technical approach separately, but anything which imparts understanding should not be ignored or discouraged; that's not the same thing as relying on a PP reference in a technical approach to determine the PP factor rating.

"We found that the approach of using the strongest horse  was not conducive to good performance, because big horses tend to be obnoxious jerks who throw their riders without warning."

Link to comment
Share on other sites

9 hours ago, Vern Edwards said:

On a ranch, they are components of something else—a working team, which has its own attributes. You can evaluate the horse, or the rider, or the team. Which is it?

In an RFP you must state what it is that you are evaluating. The answer to Sam101's question is that if you state that you are evaluating an "approach", then evaluate it based on its own attributes—the characteristics, features, properties, and qualities that make it what it is.

I am sorry but I just can't get to your original thought and the further explanation as dipositive example whereby a horse's merits are determine by its attributes but a riders merit has something else.   After all you now have thrown in ranch and team as well.  Each has characteristics, features, properties, and qualities.  If the example was refined to a trained horse, untrained horse, horse for food, show horse, competitive horse for dressage, competitive horse for rodeo then an approach to getting the horse of one of these colors would have its attributes as well.

Have a great day!image.png.6355bd6e4c39286c00f215c7db203cc7.png

 

Link to comment
Share on other sites

1 hour ago, REA'n Maker said:

By "Compartmentalizing" I mean treating the factors as being completely distinct and separate, e.g., expecting an offeror to discuss an approach without referring to their past experience with that approach.  That's not a mortal sin.

@REA'n MakerIt's not a sin of any kind, mortal or venial. But it's not sound practice.

As in all decisions involving multiple objectives, the task is to (1) identify the attributes of the alternatives you are evaluating that will be useful to the pursuit of acquisition objectives (have "value"), (2) assess each alternative's performance on each attribute, (3) integrate the findings on diverse attribute scales to a common value (utility) scale, (4) compare and rank the alternatives.

Decision scientists have worked on that task for decades and developed sound, widely-practiced methods. They go by various names, such as SMART (Simple Multiple Attribute Rating Technique), SAW (Simple Additive Weighting), and Analytical Hierarchy Process (AHP).

You can read about them in books like

  • Decisions with Multiple Objectives: Preferences and Value Tradeoffs (a classic);
  • Advances in Decision Analysis: From Foundations to Applications;
  • Decision Analysis: Introductory Lectures on Choices Under Uncertainty;
  • Foundations of Decision Analysis;
  • Decision Analysis and Behavioral Research (based on Navy-funded research in decision making; another classic, but technical);
  • Value-Focused Thinking: A Path to Creative Decisionmaking (good discussion of attributes);
  • A Science of Decision Making; 
  • Smart Choices: A Practical Guide to Making Better Decisions;
  • Decision Analysis for Management Judgment (very clear and practical); and
  • Multiple Criteria Decision Making: From Early History to the 21st Century;

all of which are on the bookshelf behind me as I sit typing, and many more, not to mention countless and widely-available research and practical-application papers, such as SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement. The government funded much of the research that went into those books, but it didn't tell their contracting people about it or provide them with decent education and practical training.

Ignorant contracting officers—who haven't, don't, or won't read such books, because they don't know about them or because they're hard reads—speculate and pontificate based on half-baked OJT (training by rumor and innuendo), and then spread their ignorance to trainees entrusted to them.

And so it goes.

 

Link to comment
Share on other sites

1 hour ago, C Culham said:

I am sorry but I just can't get to your original thought...

@C CulhamI'm sorry about that. I thought it was a simple illustration. I'm not sure I can dumb it down any further. But try this:

Bucephalus was a magnificent warhorse, large, strong, brave, and fierce.

Alexander was a daring tactician, fearless warrior, great horseman, and the only man able to break and ride Bucephalus.

The objective was victory. Each was valuable in their own way in that pursuit. The value of the whole was equal to the sum of the values of the components.

Together, they were an unequaled victory machine. Never defeated.

Alexander on another horse, or Bucephalus under another rider, would not have been as great.

If that doesn't add up for you, don't bother to tell me, because I can't do better.

Link to comment
Share on other sites

12 hours ago, Sam101 said:

It says this:

L.1 The offeror shall provide the requested information within the designated volume evaluation factor. Any requested information placed outside of the designated location shall not be considered or evaluated under that evaluation factor regardless of whether it is located somewhere else in the proposal. As an example, the offeror shall not refer to any key personnel qualifications, the staffing plan, corporate experience, or past performance in the Technical Approach factor section of its proposal. If the offeror does so, that content will not be considered by the government during the evaluation of the offeror’s Technical Approach.

L.2 Factor 1 - Technical Approach: Offeror shall describe how they will perform the SOW's requirements.

L.3 Factor 2 - Key Personnel and Staffing Plan: Offeror shall provide resumes of the key personnel listed in C.15 and describe their staffing plan.

L.4 Factor 3 - Corporate Experience: Offeror shall describe three past contracts of similar size and scope.

L.5. Factor 4 - Past Performance: Offeror shall provide the attached PPQ to the customers of the three contracts mentioned in the Corporate Experience Factor.

You are communicating what you want, but it seems some offerors aren't reading this. Maybe bold and capitalize?

It's common to include evidence of using an approach in a technical approach. I've seen training for proposal writing that says to write that way.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...