Jump to content
The Wifcon Forums and Blogs
Sign in to follow this  
lotus

Evaluation Bias

For those of you with a history on the buyer side of the fence . . .  

20 members have voted

  1. 1. Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

    • Yes
      8
    • No
      12
  2. 2. If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

    • Yes
      4
    • No
      16


Recommended Posts

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

Share this post


Link to post
Share on other sites

I've added a poll using the OP's questions in the hope that it may be answered anonymously.  The OP stated:

For those of you with a history on the buyer side of the fence, ...

 

You can respond verbally, if you like, as the OP wrote it.

Share this post


Link to post
Share on other sites

1)  The golden rule is to evaluate as you said you would in the solicitation, and your evaluation is impartial.  This is very explicitly stated many times in the FAR.   Favoring some offerors over others is a big no-no, and the subject of bazillions of protests.  There are lots of controls in place to prevent that from happening, but it still does happen.

A good Contract Specialist will ensure that the *legitimate* reasons a vendor is a 'favorite' are factored in.    Say a vendor is favored because they have an excellent track-record of performance.  Then a solicitation could heavily weight past performance.  That's legit.  Tweaking specifications so that only a single brand name product could be acceptable, without a corresponding 'Brand Name Only' justification, not so legit.  Deciding in advance who gets the contract, and then backward planning the acquisition procedures and source selection - not legit at all.

2) No.  In my experience, we don't mind hard questions.  I consider them a type of credible positive signaling (Someone competent spent valuable time reading and thinking seriously about the solicitation).

We don't like dumb or pointless questions.  To the extent possible, I try to make questions anonymous so that the folks answering the questions don't know which offeror asked the questions.  I do this to try to avoid setting expectations, because questions definitely do establish some prior expectations about the offerors, no matter what anyone says to the contrary. 

Share this post


Link to post
Share on other sites

OK, we have 14 responses:  12 votes and 2 posts, as of now, with contrasting results.  Bias has been raised before in the discussions here.  The OP wrote 2 thoughful questions. 

The OP is looking for responses from 

Quote

For those of you with a history on the buyer side of the fence

Let's reach at least 20 responses by the end of today.

Share this post


Link to post
Share on other sites
22 hours ago, lotus said:

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

1.  No.  From my experience, the general or common practice is to diligently treat all offerors fairly.

2.  No.  From my experience, the general or common practice is to diligently treat all offerors fairly.  Generally, substantive questions are appreciated.

Share this post


Link to post
Share on other sites
Guest Vern Edwards
On 9/5/2018 at 3:35 PM, lotus said:

For those of you with a history on the buyer side of the fence, ...

1.  Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

2.  If a prospective offeror asks many questions, or questions that are hard to answer or inconvenient to answer, do evaluators become biased against that prospective offeror as someone they don't want to work with?

1. No.

2. Yes, in some cases, especially if the questions fall into the category of "dumb" or the person who asked the questions is a known gadfly.

Share this post


Link to post
Share on other sites
Quote

1. Is it common in practice for the procurement team, especially those who are professionals in fields other than procurement, to purposely evaluate proposals from their favorite vendors more leniently than they evaluate proposals from others?

I think this question is worded to assume some bad faith on the part of the evaluator.  Let me assume good faith just for a contrast:

A "favorite" firm may be a favorite because they have a good record of past performance, good personnel, etc. and therefore earn a higher score.  The higher score may be a result of those superior features, rather than some emotion on the part of an evaluator.

Then,  you as an outsider may interpret that as a lenient evaluation, even when the "favorite" is evaluated to the same standard as everyone else. 

Share this post


Link to post
Share on other sites

I think, like apsofacto said, you might have to look at this from a different direction.

For question 1: are you talking about individuals with FAR knowledge on the procurement team, or just an average buyer/member? 

  • From my experience most procurement agents with FAR knowledge and who work with compliance personnel will avoid "purposely" evaluating to favorite vendors.   However if you get an engineer, or non-professional, involved this if more frequent: where they have a reason for wanting a vendor and will skew towards them where possible. Hence having a diverse team will tend to lead to a more impartial view of the situation.  ex: Lead-time, technical knowledge or past performance  vs just finding a vendor easier to work with.

For question 2: 

  • This again depends on the nature and frequency of the questions.  While lots of procurement agents may not love seeking answers to the questions, if the are well thought-out and not overly burdensome to the procurement process they won't affect the award.  However a vendor that questions every solicitation and specification may be somebody that will slowly be  viewed poorly if those questions are deemed frivolous by the customer.

Share this post


Link to post
Share on other sites

For question 2:

- If you are a jerk in your interactions with the contracting officer you could run into problems.

- I've had vendors ask me so many questions that I felt like they wanted me to write their proposal for them. Even the most clearly stated RFQ/P will be interpreted in different ways so I think prospective offerors just have to live with a certain amount of ambiguity. The CO is probably not going to be able to make it perfectly clear for everyone.

- I dislike answering questions that can be answered with publicly available data. FPDS and FBO are your friend. Questions about the incumbent and the previous price paid can almost always be found there (Not always, but most of the time). So don't ask the CO to do your research for you.

Share this post


Link to post
Share on other sites

Question 1: The key word here is "purposely".  I believe that some are more lenient with the incumbent but not intentionally.  They just know their work better as they have spent the last several (in many cases) years with them. 

Question 2: If the questions are intelligent questions, no problem.  If the questions are antagonistic or repetitive, then that becomes annoying and one starts to question the intelligence of the company asking them.  If they can't ask intelligent questions, how can we be confident they can perform the work.  However, I attempt to mitigate this as I never divulge to the TEP who the companies are that are asking the questions, thereby trying to keep the playing field as level as possible.  

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...