Thanks
Gene

By Vern Edwards on Monday, August 19, 2002 - 07:56 pm:

The best way to conduct an oral presentation is as an interview to determine how well the offerors understand the government's requirements.

A good oral presentation session is conducted in three parts:

In the first part, the offeror should be given 10 to 15 minutes to introduce itself as a company and each of its attendees.

In the second part, the government evaluators should ask a set of preselected questions about the government's requirement. This set should be the same for all offerors. The evaluators should be permitted to ask a reasonable number of follow-up questions based on the offeror's answers. The questions should take the form of: How would you... ? How would you respond to... ? What would you do if... ? What would you do if that did not work? This Q&A session should last no more than one and one-half to two hours.

In the third part, the offeror should be given 10 to 15 minutes to say why it thinks the government should select it to be the contractor.

An oral presentation should not last longer than two to two and one-half hours. The RFP should describe the process much as I have done above. The offerors should not be asked to develop a briefing or presentation materials and they should not be given the government's questions in advance. The government should make a video or audio tape of each session.

By Anon2U on Monday, August 19, 2002 - 08:08 pm:

As a survivor of a protest, I must recommend that a court reporter be hired and a transcript of the orals made. They will give you the transcript on floppy disk (insist that they do before you hire them). This saves hours of research time every time a question arises about what was promised at the orals by using the find feature on Word. In addition, GAO enjoyed being able to go to the meat of the presentation versus hunting the information all through a video tape. We got 45 hours of recording (over 9 separate days), two hard copies of the transcripts, and the floppies for about $5K. An quick dismissal of the protest was worth that easily.

By Vern Edwards on Monday, August 19, 2002 - 08:45 pm:

GeneJ:

Although I respect his opinion, I disagree with Anon2U for two reasons.

First, agencies should not obtain promises in an oral presentation [in this regard, see FAR § 15.102(f)], and thus there should be no need to review oral presentations to find out what was promised. Promises (offers) should be obtained in writing. Oral presentations should be used only to assess offeror capability.

Second, a court reporter is a needless expense. Evaluators should take good notes on a checklist form developed for that purpose, and a videotape or audiotape will satisfy the GAO.

Keep it simple and focused and you won't have a problem.

By Chuck Solloway on Tuesday, August 20, 2002 - 03:29 pm:

Gene,

No question in our business seems to have a simple answer. In your case, it depends on when you have oral presentations and why you are having them. If you intend to make award without discussions, then Q&A at the oral presentation may seek clarification of contractors proposals but may not evolve into "discussions" as defined by the FAR. If oral presentations are being used to determine a competitive range prior to holding discussions, then it appears that "communications" may be held, but not "discussions". If you are holding oral presentations only with those in the competitive range, then you may hold "discussions". Traditionally, the extent of discussions may vary significantly among competing contractors so long as there is no "favoring" (such as in technical leveling).

If you are using the "Pop Quiz" technique where contractor key personnel are all asked the same questions to determine how well they understand the government requirement or how expert they are in the particular field of endeavor; then you do not have to concern yourself with issues of "clarification" "communication"and
"discussion" - at least as far as orals are concerned. The Pop Quiz does not give any of the contractors a chance to change their proposal and thus is never "discussions'. And since all contractors are asked the same questions there is no "favoring".

I agree that there is no need for a court reporter and recommend video tape. Use two cameras. There has been at least one instance where a contractor had to be called back because the camera didn't work.

By K.C. on Wednesday, August 21, 2002 - 08:50 am:

First, having been through a major oral proposal effort (very large program, 4 offerors), I also recommend 1) evaluator notes and 2) a transcript. The videotape clearly satisfies the requirement to document the oral props BUT it's quite a burden on the evaluators to go back through a few hours of videotape/offeror to find something. And while it would be lovely for every evaluator to remember everything about every offeror, it's just not realistic. It's not a matter of seeing "what was promised" but a matter of verifying and validating during the deliberation process. (And a court reporter is not necessarily more expensive than video production.)

Second, Vern, on what do you base your prescription for "the best way" to conduct oral proposals? Specifically, I like that you don't prescribe 6 hour presentations by the offeror, but is 10 minutes really enough? Also, what about clarifications?

By Vern Edwards on Wednesday, August 21, 2002 - 09:20 am:

K.C.:

My prescription is based on several things. Most importantly, prepared presentations allow offerors to hire consultants and to rehearse. Unscripted interview responses should provide better insight into what an offeror really knows.

Yes, 10 minutes is enough for introductions and 10 minutes more for concluding statements. As for clarifications, the interview approach to oral presentations provides ample opportunity for offerors to clarify what they know about the government's requirement.

I still disagree about the need for transcripts. If oral presentations are designed and managed properly, there should ordinarily be no need for evaluators to review videotapes. Evaluators don't need to remember everything, and they should score a presentation within two hours after it has been made. Moreover, there is no need for a videotape "production." Put a camera on a tripod in the corner of the room, aim it at the offeror's table, and turn it on.

Keep them simple; keep them short. There should be no such thing as a "major" oral presentation effort.

By K.C. on Wednesday, August 21, 2002 - 09:28 am:

Vern ~ Does your prescription anticipate written proposals (other than the basic contract/terms and conditions)?

By GeneJ on Wednesday, August 21, 2002 - 11:42 am:

Hi

I'd love to work on an oral presentation as described by Vern. Unfortunately we have been getting what I call "written proposals in oral format". Given the obvious scripting and preparation I can't believe that that type of implementation saves money.

One story I did run across was as a result of our encouraging Sub-contractors to attend. Issues that were raised as questions with first offeror teams became well rehearsed charts and talking points the second time that the sub-contractors were in the room.
It didn't effect any thing in this procurement because we were doing a much more formal (less effective) approach than Vern spelled out. My little contribution is to advise folks to use care on sub-contractor attendance if you plan on asking the same set of questions.
Gene

By Pearse McDade on Wednesday, August 21, 2002 - 01:52 pm:

KC/GeneJ et al,

I expect that Vern is too modest to mention it and/or doesn't want to use WIFCON for advertising and apparent self-aggrandizing, but he has written two excellent books "Source Selection Answer Book" (Management Concepts www.managementconcepts.com) and "Streamlining Source Selection with Oral Presentations" (CCH Publishing) that I would heartily recommend to anyone who has questions regarding oral presentations.

His logical approach and clear writing style anticipate all of your concerns and provide answers and rationale supported by regulation and the case law.

Pearse McDade
Federal Employee and Confirmed "Ed" (as in Edwards) Head

By Vern Edwards on Wednesday, August 21, 2002 - 07:28 pm:

Pearse:

Thanks.

KC:

There could be written proposals. You always want a written offer to perform in accordance with the terms of the RFP at stipulated prices. You also need written certs and reps. For services, I think no other written proposal is necessary in most cases. For supplies, depending on the nature of the government's specification, you may need a written description of the specific supplies that the offeror promises to deliver.

I have long argued against proposals that describe the offeror's "technical approach." I don't think those are worth the paper that they're written on.

By GeneJ on Thursday, August 22, 2002 - 06:29 pm:

Vern

If memory serves one of your posts or even a whole thread from a while ago had an explanation of the concepts of "offer language" and "proposal language". I can't find anything on the topic. Do you know if the electrons have been totally recycled? Forgive me if I'm misattributing words to you.

GeneJ

By K.C. on Friday, August 23, 2002 - 08:40 am:

Thanks, I understand the concept of offer. I was more curious if you were recommending written and oral proposals -- so that not only does the offeror have to present information orally but has to write it as well (and the evaluators have to listen/read). Glad to hear that's not what you meant.

BTW, for those of you who don't like scripted oral presentations, my thoughts:

1) Use Vern's method, which limits the amount of presentation.

2) Consider that the person who is speaking almost certainly has salary, perhaps promotion or job, on the line -- not to mention some measure of accountability for overall Firm revenue. The stress is incomparable to what the Government evaluators face.

3) Most RFPs now require that the person who will be the PM (and/or Tech Lead) present the majority of the proposal. Unless you're hiring them to make speeches, give them a break on speaking style.

3) Consider that what they say has to go through corporate contracts/legal, much the same as everything on the Government side. Imagine how scripted an oral SSEB/TEB report would be.

4) And finally -- if you find that presentations are scripted carefully, but the offeror is unable to clearly answer basic questions (either interview style re. the requirement or clarifications re. the proposal), for heaven's sake give them that feedback in the debrief. That's how firms improve, increasing competitive environment for the Government.

By Vern Edwards on Friday, August 23, 2002 - 09:56 am:

GeneJ:

I don't know what happened to that discussion, and I'm not sure whether it took place at Wifcon or at the old OFPP Watercooler.

I discuss those concepts in some detail in my book.

K.C.:

I heartily endorse your comments. I especially hope that government people will attend to your comments about the pressure that an offeror's presenters are under during an oral presentation. COs should design oral presentations to keep that pressure to a minimum.

Ideally, an oral presentation should be conducted in a comfortable conference room, around a conference table, like a business meeting rather than a staged event. The number of government attendees should be kept to an absolute minimum, to avoid creating an atmosphere of appearing before a large audience.

The contracting officer should greet the presenters upon arrival, introduce the government team members, and ask the offeror's team leader to introduce his or her companions. It wouldn't hurt to engage in some idle small talk over coffee or juice for a couple of minutes before starting the official proceedings, in order to give the offeror's representatives time to catch their breath and relax just a little after looking for parking and getting through security. Treat the presenters like you would like to be treated if you were in their place.

By Anonymous on Friday, August 23, 2002 - 12:08 pm:

K.C. and Vern Edwards' posts above provide some excellent advice. Much more will be gained when the presenters are at ease and the focus is on what they actually know about the contract problem and their corporate view of the solution.

The question about what written material of a proposal sort that should be provided should, in my opinion from some experience, be minimal to zero. My impression is of agencies initial attempts here often clinging to a comfort blanket of substantial supplemental written material. It usually isn't worth the time spent to evaluate in terms of providing a discriminator.

The only exceptions I can think of to a zero goal are in highly technical solutions involving either numbers, perhaps creatively meeting multiple standards or such other dreary, yet important information. Some sort of matrix may assist in understanding how the company understands and approaches the problem. This can be required and examined in advance of the oral presentation thus avoiding wasted time on the subject and providing material for questions. Offerors should be on notice that they may be asked questions about this material as part of the oral presentation's questioning period, but they should not waste their time in repeating the material except as needed to illustrate other points in that presentation.

Viewing the oral presentation as a means to measure the offeror's understanding of the problem and solution should mean that testing questions are utilized. Those putting a muzzle on that aspect out of fear of "discussions" and such are missing the point and misusing oral presentations in my opinion.