Jump to content
The Wifcon Forums and Blogs
  • entries
  • comments
  • views

Part II - A Contracting Trainee and Her Supervisor Discuss Source Selection Evaluation Factors

Vern Edwards


Alice Wonderly, a contracting trainee, is walking back to her cubicle after talking with her OJT supervisor, Mr. Sagesse, a contracting officer with an unlimited warrant. She has been tasked with developing a source selection plan and has been given an old one from which to cut and paste. But she wants to understand what she’s doing. She especially wants to understand the concept of “evaluation factor,” instead of just having "samples" and learning the rules in FAR about their use. She wants to understand what kind of things they are and what they all have in common. She thinks that may be the key to understanding the evaluation process. She asked Mr. Sagesse about evaluation factors, but did not get a helpful answer. All he said was that they are “criteria” and “standards.” To Alice, that was nothing more than referring to evaluation factors by a different name, without explaining what they are.

Discouraged, Alice is walking down a corridor when she runs into her friend and fellow trainee, Patrick Moore, who works in the Optimum Momentum Program. They stop to chat.

Patrick: Hi Alice, what’s going on with you?

Alice: I’ve been told to prepare a source selection plan. I’m working with the typical cut-and-paste “template” they give you, but I’d sure like to understand what I’m doing for once.

P: Wouldn’t we all. How’s it going?

A: Not well. I just went to Sagesse to see if I could get some answers, but I got nothing.

P: [Chuckling.] Yeah, well... I did a training rotation with Sagesse. Good luck getting anything from him.

A: Who’s your OJT boss?

P: Mr. Ewing. He’s great. I was pretty lucky to draw him at the last trainee assignment lottery.

A: Good OJT?

P: The best. If you have questions and do your homework before going to him, you’ll get a clear and concise explanation and plenty of references for stuff to read. But heaven help you if you didn’t do your homework first.

A: Tough?

P: Demanding. He expects you to think. Half the time, when I go to him with a question, he asks me questions before he answers my question. He’s checking to see if I did my homework before asking for help. He reads my documentation closely and asks me questions to test my thinking and conclusions. He insists on good writing. He says people think in words and sentences and when a person writes poorly it's either because he thinks poorly or is lazy. I graduated with honors and thought I was a good writer until I went to work for him. He makes me rewrite things a lot. He will pick out words that I’ve used in a memo and ask for a definition. If I don’t spit it out right away he says that a person who can’t define a word he’s used literally doesn’t know what he's talking about. And he’s a punctuation fanatic. It made me angry at first. He saw the anger in my face once and he said, “I’m here to teach you. You’re here to learn. Work at learning, or go to the training office and tell them you want to work for someone else.”

A: Yikes!

P: Yeah, he intimidates some of the trainees. You know, some people are always talking about being intimidated by this or that. Fragile flowers. But I can see that my thinking and writing have improved a lot since I came to work for him, and I feel like I’m really learning the business. A lot of his former trainees are GS-14s and 15s now. There are even a couple of generals that were former trainees of his. They stop in to visit from time to time.

A: So Ewing is actually training you.

P: You bet. He says that training is one of a manager’s most important personal responsibilities. He personally conducts weekly one-hour training sessions every Wednesday during lunch. He calls them “brown baggers.” Attendance is voluntary, but I know he keeps track of who comes and who doesn’t. He’s looking for commitment. Every week he covers a topic chosen by someone in the office, like data rights, cost analysis, cost estimating, performance-based payments, or cost allowability. He explains concepts, principles, rules, and practices. He doesn’t just tell you. He shows you. But you’ve got to put in the time to study on your own, too. He won’t hand-hold you.

A: You’re lucky. I think.

P: I'm lucky. Look, let’s go see him, and you can ask him your question.

A: But I haven’t done much research. I’ve read the FAR and our supplement and the source selection manual. I’ve looked in dictionaries and a couple of contracting books, but I haven’t found anything helpful. I don’t want to get trashed for not being ready.

P: A fraidy-cat? You? Look, he can kill you, but he won’t eat you. Come on. I think he’ll be okay with it, since you don’t work for him.

They go to Ewing’s office. One wall of the office is covered with small picture frames, 5 by 7 inches, in each of which there is a piece of rag paper on which a quote has been typed on a manual typewriter. Some of the sayings are:

“Reflective thinking is always more or less troublesome because it involves overcoming the inertia that inclines one to accept suggestions at their face value; it involves willingness to endure a condition of mental unrest and disturbance. Reflective thinking, in short, means judgment suspended during further inquiry; and suspense is likely to be somewhat painful. As we shall see later, the most important factor in the training of good mental habits consists in acquiring the attitude of suspended conclusion, and in mastering the various methods of searching for new materials to corroborate or to refute the first suggestions that occur. To maintain the state of doubt and to carry on systematic and protracted inquiry—these are the essentials of thinking.”
John Dewey

“Never be afraid to sit awhile and think.”
-- Lorraine Hansberry, playwright

“Knowledge of the product or service, and its use, is essential to sound pricing.”

-- Armed Services Procurement Regulation 3.802-1, “Product or Service”

“Managing is getting paid for home runs someone else hits.”
-- Casey Stengel

“I get angry about things, then go on and work.” -- Toni Morrison, Nobel laureate

The best advice I ever got was that knowledge is power and to keep reading."

-- David Bailey, fashion photographer

“Improvise. Adapt. Overcome."
-- Unofficial motto of the U.S. Marine Corps

“The question isn’t who’s going to let me; it’s who’s going to stop me.” -- Ayn Rand, novelist and philosopher

“Consider yourself an enemy, and be on the lookout for ambushes.”
-- Epictetus, philosopher, 2d Cent. C.E.

“Don't feel entitled to anything you didn't sweat and struggle for.”
-- Marian Wright Edelman, social activist

“In this outfit, officers jump first and eat last.” -- Maj. Gen. James M. Gavin, Commander, 82d Airborne Division, WWII.

“He that wrestles with us strengthens our nerves and sharpens our skill. Our antagonist is our helper.” -- Edmund Burke, politician, statesman, and philosopher

“We are all born ignorant, but one must work hard to remain stupid.”
-- Benjamin Franklin

“If you obey all the rules, you miss all the fun.” -- Katherine Hepburn

There is a bookcase. Alice sees the Cibinic and Nash series and about 100 other books, shelved alphabetically by title. Some of the titles are:

The Acquisition of Defense Systems

The Cambridge Dictionary Of Space Technology

The Cambridge Grammar Of The English Language

Challenges in Managing Large Projects

Cost Estimating

The Craft of Argument

Decision Analysis for Management Judgment

Defective Pricing Handbook

Dictionary of Modern Legal Usage

Dictionary of Statutory Interpretation

Elements of Contract Interpretation

Elements of Spacecraft Design

Garner’s Modern American Usage

Government Contract Changes

Government Contract Cost & Pricing Handbook

Guide To Federal Agency Rulemaking

Handbook Of Technical Writing Practices

How Spacecraft Fly

How To Read A Government Contract

Informal Logic

Introduction To Decision Analysis

It’s Only Rocket Science: An Introduction in Plain English

Licensing Software And Technology To The Government

The Mind And Heart Of The Negotiator

The Negotiation Game


The Polaris System Development

Probability Methods For Cost Uncertainty Analysis

Rights In Technical Data and Computer Software

The Science and Technology Dictionary

Tacit & Explicit Knowledge

Thinking Mathematically

The U.S. Air Force And The Culture of Innovation

The Visual Display Of Quantitative Information

There is a notebook on top of the bookcase bearing a label that says:

“If you want to borrow a book, write (1) your name, (2) your office symbol, (3) the date that you’re checking a book out, and (4) the date that you will return it. You can’t borrow any book for more than two weeks.”

“These books are mine, not the Government’s. You can borrow a book, but the penalty for not returning one of my books is death. Don’t write or highlight in one of my books. If I have to call you to get a book back, you won’t be able to borrow any more books.”

Ewing is sitting with his back to the door at a small table that is separate from his desk and computer. There is nothing on the table but a pad of blank paper and a pencil. He appears to be thinking.

P: Mr. Ewing?

Ewing: [Turning and looking up.] Ummm? Oh, Hi, Patrick.

P: I’d like to introduce a friend, another trainee, Alice Wonderly.

A: Hello, Mr. Ewing.

E: Nice to meet you, Ms Wonderly. What are you doing here in the Optimum Momentum Program Office? Visiting Pat?

A: Well, not actually. I’ve been looking for an answer to a question, but I’m having a little trouble, so Patrick suggested I come see you.

E: [Hesitates.] Well, who do you work for? Have you spoken with him or her?

A: My boss is Mr. Sagesse.

E: Oh, I see. Well, Jack Sagesse has a lot of experience.

A: Yes, I know. Sorry to bother you.

E: You’re not bothering me. Out of curiosity, what is your question?

A: I’m working on the Foochi Minooli Program and I have to prepare a source selection plan. I want to understand the concepts, not just cut and paste.

E: Well, that’s good, but please state your question.

A: [Hesitating… thinking.] What kind of thing is an evaluation factor? I know that they’re also called criteria and standards, but I want to know what kind of thing they are.

E: Ah. You don’t want a lexical definition. You want a stipulative definition.

A: I’m not sure what those are.

Ewing reaches to the bookcase, pulls out a slim volume, and hands it to Alice. The title is Definition by Richard Robinson.

E: Read in this. Before you leave, put your name and office address in the notebook and read the lending rules. Return the book by the end of next week.

According to the author, a lexical definition tells you the words to which a word or phrase refers. A stipulative definition explains the thing to which a word or phrase refers.

A: Oh. Well, that’s exactly what I want, an explanation of “evaluation factor.”

E: What research have you done?

A: I’ve read FAR Part 15. And I read in the Cibinic and Nash books, Formation and Competitive Negotiation. But they tell you the rules and the case law about evaluation factors. That’s not what I’m looking for.

E: It’s rare to meet someone who asks about concepts. Are you sure you want to know?

A: Uhhhh, yes. Why do you ask?

E: I ask because knowing the answer might make your thoughts clearer, but it won’t make your job easier. It would be much easier to let your question lie and just cut and paste. That’s why people do it. You know what they say: “Ignorance is bliss,” and “Why reinvent the wheel.”

A: How can knowing more make my job harder?

E: Because once you know the answer, you and your team will have to choose between doing a professional job and thinking very hard or doing a cut and paste job and not thinking much at all. I consider that a moral decision. Your technical clients might enjoy thinking hard about technical matters, but they might not like having to think hard about mere contracting matters.

A: I still want to know.

E: Ever taken a class in decision analysis?

A: No.

E: Okay. Do you have total recall, or do you want to take notes?

A: Oh, yes, I’ll take notes. [Hurriedly takes a spiral notebook and a pen out of her brief case.] Ready.

E: An evaluation factor is an attribute of an offeror or of its offer--a feature, property, or characteristic of an offeror or of its offer that provides value to the government by its presence or its absence.

A: [Writing furiously.]

E: Anything else?

A: [Hesitating.] Well, could you elaborate just a little on “attributes”?

E: Sure. All things and ideas have features, properties, or characteristics that are either inherent in them or that are ascribed to them on the basis of subjective judgments. Some of those features, properties, and characteristics are desirable because they are useful or, as we say, “valuable” and so the more of them the better. Some are undesirable, because they’re damaging and detract from value, so the less of them the better. The word “attributes” refers to those features, properties, and characteristics. When we in the government decide to evaluate offerors and their offers on the basis of such attributes, we call them “evaluation factors,” which is a term of art in government contracting.

A: [Writing furiously.]

E: [Continuing.] We choose contractors by comparing offerors and their offers on the basis of the value provided by “evaluation factors.” “Value” is a general attribute that we ascribe to offerors and their offers on the basis of evaluation factors. Consideration of individual attributes yields various measures of value, which can be weighted for importance to our objectives and “synthesized” or “integrated,” as we say, to yield the summary attribute that we call “total value.” The competitor that ends up having the highest total value will be what we call the “best value.”

Formally, what we call “source selection” is a version of what is called “multiple attribute decision making,” which is a collection of several formal methods for making rational choices from among alternatives.

A: [still writing] Could you provide some specific examples of attributes?

E: Of course. Let’s say we’re buying hammers and we decide to evaluate offers for hammers. Hammers have certain attributes: functional design features, such as the claw on a claw hammer, which is for extracting nails, and the ball on a ball peen hammer, which is for working metal; material composition—some have wooden handles, others have graphite core or fiberglass handles; weight; size; durability; et cetera. They also have properties that buyers ascribe to them subjectively, such as “balance,” “heft,” and “feel.” And they come with “assigned” properties, such as price, availability, and warranty.

In order to evaluate offers, we must determine which attributes are important to us. We must then somehow determine how much of each attribute is found in each offered hammer. Then we must compare and rank the offered hammers to determine which is best.

A: So that’s how we evaluate proposals.

E: Actually, we don’t evaluate proposals.

A: Huh? I don’t understand.

E: We don’t evaluate proposals. We evaluate offerors and their offers. Think of a proposal as a package that contains (1) information about the company that submitted it and (2) statements of the promises that the company is offering. You don’t evaluate the proposal, per se. You evaluate the offeror and the offer described in the proposal based on offeror attributes and offer attributes.

A: But everybody says we evaluate proposals.

E: Yes, well, "everybody" says all kinds of things. That’s not what matters. What matters is what's true and what's not. Saying that we evaluate proposals is just a careless way of saying that we evaluate offerors and their offers based on the information in their proposals. When we say that a company submitted an excellent proposal, we’re really saying that the offeror and the offer described in the proposal were excellent. At least, that's what we should be saying.

A: That’s an entirely new way of thinking. No one has said that to me before.

E: Think it over.

A: I will. Can I ask more questions?

E: You've got enough to think about for a while. When learning concepts, it’s best to proceed methodically. You do want concepts, right? You’re not looking for a manual of source selection, are you?

A: No, I’ve read the manual. I want to understand the underlying concepts.

E: Good, but are you willing to struggle a little?

A: Yes, sir.

Ewing opens his desk drawer, flips through some files, pulls out a paper, and hands it to Alice. The 20-page paper is entitled, "SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement," by Ward Edwards (1994).

E: Read this. Think. Then write down any questions you have and come back in a week.

Patrick: May I have a copy, too?

E: [Hands a copy to Patrick and smiles.] If you hadn’t asked, I would have wanted to know why not.

A: Mr. Ewing, thank you so much. I really appreciate your time and answers.

E: The best way to thank me and to show your appreciation is to read the paper, think hard about it, and come back with some good questions. Okay, you two, I have to get back to work now.

Ewing turns back to his table, pencil, and blank pad of paper. Alice and Patrick leave.

P: [As they walk.] Alice, if you come back, those questions had better be good ones and well-written. He has high standards for questions. Topic and query had better be clear and show that you did some thinking and made reasonable presuppositions.

A: Huh?

P: He conducted a “brown bagger” on questions and questioning strategy and tactics, and he gave us assigned readings. Come to my office, and I’ll give you copies of the readings.

To be continued…

1 Comment

Recommended Comments

Thought provoking story. I like the imbedded suggested reading. Here's a link to the article (hopefully I'm not violating any community rules by posting it). Funny, all convention in government acquisition I've seen so far (along with Formation of Government Contracts, Cibinic/Nash) seems to make the case that general evaluation criteria are easier to defend than numerical scores or formulaic methods, no matter how well conceived. Also it seems like I've seen some service-specific prohibitions against using numerical scoring on ratings, but I've found nothing broadly outlawing it.

I'm not against something like what's laid out in the article, (I'll definitely need to read through it a couple times, It's kind of academic in tone) but I'm having a hard time getting my head around how one would implement it. Maybe part three will help.


"But I don't want to go among mad people," Alice remarked. "Oh, you can't help that," said the Cat: "we're all mad here. I'm mad. You're mad." "How do you know I'm mad?" said Alice. "You must be," said the Cat, "or you wouldn't work in Government Acquisition." - Lewis Carroll (with some liberties).

Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now