Jump to content

Evaluation Bias


lotus

For those of you with a history on the buyer side of the fence . . .  

23 members have voted

  1. 1. Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

    • Yes
      10
    • No
      13
  2. 2. If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

    • Yes
      4
    • No
      19


Recommended Posts

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

Link to comment
Share on other sites

1)  The golden rule is to evaluate as you said you would in the solicitation, and your evaluation is impartial.  This is very explicitly stated many times in the FAR.   Favoring some offerors over others is a big no-no, and the subject of bazillions of protests.  There are lots of controls in place to prevent that from happening, but it still does happen.

A good Contract Specialist will ensure that the *legitimate* reasons a vendor is a 'favorite' are factored in.    Say a vendor is favored because they have an excellent track-record of performance.  Then a solicitation could heavily weight past performance.  That's legit.  Tweaking specifications so that only a single brand name product could be acceptable, without a corresponding 'Brand Name Only' justification, not so legit.  Deciding in advance who gets the contract, and then backward planning the acquisition procedures and source selection - not legit at all.

2) No.  In my experience, we don't mind hard questions.  I consider them a type of credible positive signaling (Someone competent spent valuable time reading and thinking seriously about the solicitation).

We don't like dumb or pointless questions.  To the extent possible, I try to make questions anonymous so that the folks answering the questions don't know which offeror asked the questions.  I do this to try to avoid setting expectations, because questions definitely do establish some prior expectations about the offerors, no matter what anyone says to the contrary. 

Link to comment
Share on other sites

OK, we have 14 responses:  12 votes and 2 posts, as of now, with contrasting results.  Bias has been raised before in the discussions here.  The OP wrote 2 thoughful questions. 

The OP is looking for responses from 

Quote

For those of you with a history on the buyer side of the fence

Let's reach at least 20 responses by the end of today.

Link to comment
Share on other sites

22 hours ago, lotus said:

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

1.  No.  From my experience, the general or common practice is to diligently treat all offerors fairly.

2.  No.  From my experience, the general or common practice is to diligently treat all offerors fairly.  Generally, substantive questions are appreciated.

Link to comment
Share on other sites

Guest Vern Edwards
On 9/5/2018 at 3:35 PM, lotus said:

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

1. No.

2. Yes, in some cases, especially if the questions fall into the category of "dumb" or the person who asked the questions is a known gadfly.

Link to comment
Share on other sites

Quote

1. Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

I think this question is worded to assume some bad faith on the part of the evaluator.  Let me assume good faith just for a contrast:

A "favorite" firm may be a favorite because they have a good record of past performance, good personnel, etc. and therefore earn a higher score.  The higher score may be a result of those superior features, rather than some emotion on the part of an evaluator.

Then,  you as an outsider may interpret that as a lenient evaluation, even when the "favorite" is evaluated to the same standard as everyone else. 

Link to comment
Share on other sites

I think, like apsofacto said, you might have to look at this from a different direction.

For question 1: are you talking about individuals with FAR knowledge on the procurement team, or just an average buyer/member? 

  • From my experience most procurement agents with FAR knowledge and who work with compliance personnel will avoid "purposely" evaluating to favorite vendors.   However if you get an engineer, or non-professional, involved this if more frequent: where they have a reason for wanting a vendor and will skew towards them where possible. Hence having a diverse team will tend to lead to a more impartial view of the situation.  ex: Lead-time, technical knowledge or past performance  vs just finding a vendor easier to work with.

For question 2: 

  • This again depends on the nature and frequency of the questions.  While lots of procurement agents may not love seeking answers to the questions, if the are well thought-out and not overly burdensome to the procurement process they won't affect the award.  However a vendor that questions every solicitation and specification may be somebody that will slowly be  viewed poorly if those questions are deemed frivolous by the customer.
Link to comment
Share on other sites

  • 2 months later...

For question 2:

- If you are a jerk in your interactions with the contracting officer you could run into problems.

- I've had vendors ask me so many questions that I felt like they wanted me to write their proposal for them. Even the most clearly stated RFQ/P will be interpreted in different ways so I think prospective offerors just have to live with a certain amount of ambiguity. The CO is probably not going to be able to make it perfectly clear for everyone.

- I dislike answering questions that can be answered with publicly available data. FPDS and FBO are your friend. Questions about the incumbent and the previous price paid can almost always be found there (Not always, but most of the time). So don't ask the CO to do your research for you.

Link to comment
Share on other sites

Question 1: The key word here is "purposely".  I believe that some are more lenient with the incumbent but not intentionally.  They just know their work better as they have spent the last several (in many cases) years with them. 

Question 2: If the questions are intelligent questions, no problem.  If the questions are antagonistic or repetitive, then that becomes annoying and one starts to question the intelligence of the company asking them.  If they can't ask intelligent questions, how can we be confident they can perform the work.  However, I attempt to mitigate this as I never divulge to the TEP who the companies are that are asking the questions, thereby trying to keep the playing field as level as possible.  

Link to comment
Share on other sites

  • 6 months later...
On 11/16/2018 at 2:11 PM, Desparado said:

Question 1: The key word here is "purposely".  I believe that some are more lenient with the incumbent but not intentionally.  They just know their work better as they have spent the last several (in many cases) years with them. 

Question 2: If the questions are intelligent questions, no problem.  If the questions are antagonistic or repetitive, then that becomes annoying and one starts to question the intelligence of the company asking them.  If they can't ask intelligent questions, how can we be confident they can perform the work.  However, I attempt to mitigate this as I never divulge to the TEP who the companies are that are asking the questions, thereby trying to keep the playing field as level as possible.  

As an offeror myself, I tend to ask a lot of questions.  I am fairly new to gov contracting but have been awarded two projects.  On both of them, I asked a ton of questions post award.  Both project CORs admitted later on that they didn't think I could do the jobs (I assume because of my questioning practice).  Upon completion of both projects, the CORs and the contracting officers heaped praise upon me for jobs well done.

I tend to be detail oriented and am not eager to accept answers that are not clear.  Not to cause offense, but I feel like many of the projects I have bid were very poorly written and the responses to my questions are too often vague.  I get the impression that many contracting officers are busy with many things and my questions are an additional burden so the answers are lackluster.

I have also read the questions that other bidders submit on published Q&As and I have to cringe sometimes.  I do understand your position on those who appear less than "intelligent".  I see the same thing but I don't think the qualifiers you've laid out are proper indications of intelligence.  Perhaps repetition comes from lack of clarity in answers or probably more so that the asker and the answerer are not on the same page and don't quite realize it.

I think most potential bidders are careful or afraid of aggravating the CO, especially prior to award lol.  Perhaps this too is a reason their questions aren't as pointed as need be?

My biggest pet peeve are non-responsive COs during the bid phase and even worse, non-responsive COs after the bid date.  I have had at least two projects that I have bid that I never heard the result of and can only assume were abandoned without notice to any bidders.  I think it is important to realize that we bidders do typically put a lot of time into our bids and it that needs to be taken into account. 

FYI, this is my first post here.  Don't take anything as criticism but as honest input from the bidding side.

Link to comment
Share on other sites

As for Question #2, I'll add a perspective not mentioned heretofore.

It is possible for the incumbent to pose technical questions, the discourse of which can only be known by the customer and the incumbent.  Potential purpose is to frighten other offerors by revealing how much must be known to operate the effort, or another purpose to derail other offerors.

What I don't know is how evaluators respond if they suspect the incumbent is sowing bad seeds.

Link to comment
Share on other sites

  • 4 weeks later...

Corduroy Frog, yes that happens often.  And it’s not just the incumbent but lots of other offerors. Companies make pretty solid guesses on who their competition is.  Then they ask questions that bring out weaknesses in others.  These can include questions on needed corporate experience, qualifications of key personnel, corporate certifications, methodologies and tools, etc.  The best companies hope for is the agency amending the RFP to tighten things up.  The minimum is it raises doubts for the evaluators and selection officials. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...