Jump to content

Recommended Posts

I am a contracting officer assisting in the planning of a source selection for an ACAT system. As part of the source selection process, the customer is adamant about including significant field testing as part of the evaluation process. The system is highly sensitive in that just about everything could affect its performance including weather, time of day, harsh language, etc. As to be expected, everyone on my end, especially legal, is worried about risk of a protest from the losing offeror.

Any ideas, useful guidance, prior examples... anything would be helpful and appreciated.

Link to comment
Share on other sites

Guest Vern Edwards

I presume that by ACAT you mean Acquisition Category and that you're working on a major system of some kind.

By field testing I presume you mean some kind of demonstration or prototype competition. If so, there's nothing new or unusual about having offerors demonstrate their systems as part of source selection. I don't know what kind of ideas or guidance you're looking for. It would be impossible for me to offer specific ideas without knowing a lot more about the system, which I'm not interested in knowing. Basically, you develop tests for specified key performance parameters and instruct offerors about how you'll run the tests and how to set up their facilities for testing; who, what, when, where, how stuff. You need to establish some procedural rules. You incorporate the results into your proposal evaluations. The tests can be either pass/fail or head-to-head competitive for the best score. They can be simple additive or tradeoff. Blah, blah, blah. You have to think to pull it off, but people in your kind of organization ought to be able to do it.

 If you know what you're doing, you needn't be especially nervous about protests. If you don't, you should contact DAU for help. They've got some people there who know about these things and can consult. Contact me privately on Wifcon and give you names.

It's pathetic for an attorney supporting a major system house to worry about a protest, and you can tell him or her I said so. His or her job is to help you not worry.

Link to comment
Share on other sites

Aaron, when you say significant field testing, do you mean testing it in an operational environment and using it as if it were government property?  II don't know why it could not be used for other types of equipment particularly if you have a DT/OT evaluation requirement.

Link to comment
Share on other sites

Correct. Offerors would set-up & connect into the system. Certain parameters & attributes would be quantified. The biggest challenges are: (1) the systems high performance sensitivity to environmental factors; (2) relatively tight award target; (3) rolled up nature of the system. It is being formed from multiple smaller systems that were originally sole sourced awards- so in a way there are two semi-"incumbents" likely to be the main competitors. Lastly, the customer has dwelled in sole source land for a very long time and this is one of their first major forays (commendable/we've been urging them...) into the competitive environment. 

Link to comment
Share on other sites

Guest Vern Edwards

For purposes of your source selection, I recommend that you refer to the test as a system verification demonstration, not a field test.

You will need the following from your customer: a system specification, a test specification, a test plan, and a test protocol for how you'll deal with competing offerors (lateness, second chances, etc.) You'll also need language for your RFP Section M.

Do you have any specific questions or requests for information that anyone can answer?

Link to comment
Share on other sites

Guest Vern Edwards

There are some GAO decisions about acquisitions in which the government conducted a product (hardware or software) verification demonstration or test as part of the source selection evaluation. See, e.g., Potomac Electric Corp., B-409710, 2014 CPD ¶ 216; Carahsoft Technology Corporation; Allied Technology Group, B-311241, 2008 CPD ¶ 119; Imaging Systems Technology, B-278112, 97-2 CPD ¶ 161; SCI Systems, Inc., B-258786, 1995 WL 64167; and Northrup Grumman; ITT Gilfillan, B- 274204, 96-2 CPD ¶ 232.

Link to comment
Share on other sites

Thank you VERY much Vern. All of the above is extremely helpful, especially the process guidance such as this:

22 hours ago, Vern Edwards said:

Basically, you develop tests for specified key performance parameters and instruct offerors about how you'll run the tests and how to set up their facilities for testing; who, what, when, where, how stuff. You need to establish some procedural rules. You incorporate the results into your proposal evaluations. The tests can be either pass/fail or head-to-head competitive for the best score.

And even more so this:

18 hours ago, Vern Edwards said:

I recommend that you refer to the test as a system verification demonstration, not a field test.

You will need the following from your customer: a system specification, a test specification, a test plan, and a test protocol for how you'll deal with competing offerors (lateness, second chances, etc.) You'll also need language for your RFP Section M

I am looking through the GAO cases right now, and I can already tell they will also be quite useful.

When you ask what questions/information I am looking for- it is just this sort of thing. Essentially, I am trying to get my arms around the process and get a bit more focused on the thinking part.

As strange as it sounds, I have had challenges finding others near me who have ever incorporated a system verification component into the source selection process. At least where I am, the large systems world is dominated by a combination of enormous MATO vehicles (with plug and play cookie cutter task order processes) and large sole source contracts. We have also been subjected to perpetual turnover, which of course makes it much harder to obtain peer support.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...