What is price evaluation?

[Guest Vern Edwards]

What does it mean to say that we are going to evaluate price in a source selection?

Are we really going to determine the value of each price?

If so, then what is the value of a price? Is it the amount of the price itself or can a price have a value that is more or less than its own amount?

If you say that we evaluate a price on the basis of what we'll get for it, then are we evaluating the price or the thing we'll get in return for it?

Probably dumb questions, but it's early in the morning and I'm waiting for a ferry. So...

[here_2_help]

Evaluate. "form an idea of the amount, number, or value of; assess"

Synonyms: assess, judge, gauge, rate, estimate, appraise, analyze, examine, get the measure of

Now, to think about the question ...

 

[Jamaal Valentine]

Any source selection or those involving tradeoffs?

What type of price evaluation - e.g. realism, risk, affordability (e.g. build to budget), or fair and reasonableness?

[Todd Davis]

13 hours ago, Vern Edwards said:

What does it mean to say that we are going to evaluate price in a source selection?

I believe it depends on what is stated in the solicitation.  Price evaluation normally concerned with determining whether or not the price to be paid is reasonable (not too high).  If stated in the solicitation, it could also include an evaluation of whether or not the price is too low (price realism analysis) and represents a risk to successful performance.  Other times it could include evaluating whether or not the prices proposed are imbalanced.   If the price is unreasonable or unrealistic, and discussions are conducted and the offeror is in the competitive range, would discuss this and the reasoning so the offeror can determine whether or not to revise its proposal.  The appropriate techniques from FAR 15.401(b) would be used to conduct the evaluation/analysis.  The price evaluation must always be documented.

13 hours ago, Vern Edwards said:

Are we really going to determine the value of each price?

Not certain I fully understand what you mean by value.  Some of the definitions of "value" from Merriam-Webster include:

1:  a fair return or equivalent in goods, services, or money for something exchanged
2:  the monetary worth of something :  market price
3:  relative worth, utility, or importance <a good value at the price> <the value of base stealing in baseball> <had nothing of value to say>
4:  a numerical quantity that is assigned or is determined by calculation or measurement <let x take on positive values> <a value for the age of the earth>

If the definition of value under the first part of the definition is used, the I would say, yes.  The reason price evaluation is done is to determine if it is fair and reasonable.  If the price in a contract was too high (unreasonable), I would not get a fair return (goods or services provided) in exchange for the payment I would have to make.  While the goods or services may have been acceptable, I would not be happy that I overpaid for them and would not think what I got was worth what I paid.

13 hours ago, Vern Edwards said:

If so, then what is the value of a price? Is it the amount of the price itself or can a price have a value that is more or less than its own amount?

If you say that we evaluate a price on the basis of what we'll get for it, then are we evaluating the price or the thing we'll get in return for it?

Based on the definition above, it can certainly be just the amount (numerical quantity) itself, but knowing that number is not enough.  We also assign value to what we expect to obtain in return.  Normally, the numerical value should be proximate to the value (benefit) to be obtained in return for payment.

I believe it is both.  The price (numerical quantity) is evaluated and compared using one or more of the price analysis techniques.  For example, when the CO is comparing the prices received in response to the solicitation, they are just focused on the numerical amount. Normally, adequate price competition establishes a fair and reasonable price.  However, it is possible that all numerical values are too high.  This is where the CO needs to considered the value of what is being procured.  Is it really worth that much?  How reasonable is the price when compared to the Government cost estimate (was the Government thinks it is worth) or what was paid for it in the past.

[Guest Vern Edwards]

Proposal evaluation is about (1) determining what we call the "value" (mission usefulness or utility) of each competitor (offeror and offer) and then (2) comparing their values and ranking them from best to worst.

Evaluation establishes a functional relationship between a specified range of measurements of each of a set of specified attributes ("evaluation factors") and the pan-attribute "value." Some persons call that relationship the "value function."

Suppose that we want to buy a device. Suppose further that its weight it an essential factor in determining its utility or usefulness (value), the less weight the better. We select an attribute measurement unit (lbs.). We establish an attribute scale by specifying the weight range of interest (from lowest feasible to highest acceptable).

We establish a scale for value (say 0 to 100 points). We specify a functional relationship each attribute measurement scale and the value scale. For weight, we establish the function between weight and value (1 point for the highest acceptable to 100 points for the lightest feasible and a straight-line function in between). We then determine the weight of each proposed device weight and determine its value. (Although I'm talking in terms of numbers, there are ways to do this, albeit less precisely, with nouns and adjectives.)

We do that for each evaluation factor, then aggregate the individual measures of value in order to obtain the measure we call "overall" or "total' value. We then compare the total value associated with each competitor (offeror and offer) to the values of the other competitors and rank them from best to worst.

So I guess what my ferry-wait question boils down to is:

How do we establish the functional relationship between price or cost-and-fee and value, especially when the amount is not all that we consider when we consider price?

or, putting it another way:

What is the nature of the relationship between price or cost-and-fee and value? How do establish the value function?

[here_2_help]

Vern,

Are you asking, "How do we evaluate value?" because that's what it seems like to me?

[Guest Vern Edwards]

Hi Help:

Quote

Are you asking, "How do we evaluate value?" because that's what it seems like to me?

No, that's not what I was asking.

We evaluate competitors (offerors and their offers) on the basis of what we call "evaluation factors." Evaluation factors are attributes of the things we evaluate--of the offeror (breadth and depth of experience; quality of past performance; qualifications of key personnel, etc.) and of its offer (assent to solicitation terms, proposed produce quality, proposed service quality, etc.). Those attributes provide value, i.e., mission usefulness (utility), to the degree that they are present.

In order to determine value we must establish a value function (conceptually, with a factor measurement or description as the "abscissa" and a value measurement or description as the "ordinate") for each evaluation factor, which states how much usefulness (utility) we will attribute to an offeror and to its offer based on the degree to which the evaluation factors are found to be present in them. Total or overall value is the sum or aggregate of the weighted values of the individual factors.

In order to evaluate competitors we: (1) (a) measure, (b) determine value, and (c) aggregate values of each competitor and then (2) compare and rank the competitors on the basis of their aggregated values.

One part of an offer is the proposed price (or the estimated cost and fee). We usually evaluate that part of an offer on the basis of the attribute we call "reasonableness" of amount and sometimes on the basis of the attribute we call "realism." The question is, what is the nature of the functional relationship between value and reasonableness and/or realism of price? Is reasonableness simply a matter of highness or lowness? Does value go up as price goes down? Does it go up as it becomes more realistic? And is the effect on the value of price when we're interested in both reasonableness and realism? What do we do when more is less, but less is also less? Is reasonableness a matter of "rightness"? If so, what is rightness? What makes a price "right"?

Price (or estimated cost and fee) is something we evaluate. We assign a value to it in some way. What is the relationship between price (or estimated cost and fee) and value? How can we think clearly about and talk clearly about this in terms that makes sense to ourselves and to others so we can explain and justify our competitive price evaluations?

I was just thinking about something while waiting for the ferry, and dropped a thought into the Wifcon Forum. Let me know if I'm not making sense, and I'll drop it.

[jwomack]

It sounds like you’re contemplating the calculations of opportunity cost.

[joel hoffman]

1 hour ago, jwomack said:

It sounds like you’re contemplating the calculations of opportunity cost.

 

Quote

 

Opportunity Cost Definition | Investopedia

www.investopedia.com/terms/o/opportunitycost.asp

1.  
2.  

Investopedia

An opportunity cost is the cost of an alternative that must be forgone in order to pursue a certain action. Put another way, the benefits you could have received by taking an alternative action. 2. The difference in return between a chosen investment and one that is necessarily passed up.

 

jwomack, It doesn't "look" to me like he is contemplating the calculation of "opportunity cost" to me, based upon the definition above and other similar definitions found during a Google Search.  What do you mean by opportunity cost?

[jwomack]

36 minutes ago, joel hoffman said:

What do you mean by opportunity cost?

Comparing something’s relative worth to its alternatives.

 

As related to evaluating price in a source selection, it would be the process of determining if the quid pro quo is a relatively good decision or not. 

 

[Matthew Fleharty]

 

On ‎5‎/‎7‎/‎2016 at 4:30 PM, Vern Edwards said:

One part of an offer is the proposed price (or the estimated cost and fee). We usually evaluate that part of an offer on the basis of the attribute we call "reasonableness" of amount and sometimes on the basis of the attribute we call "realism." The question is, what is the nature of the functional relationship between value and reasonableness and/or realism of price? Is reasonableness simply a matter of highness or lowness? Does value go up as price goes down? Does it go up as it becomes more realistic? And is the effect on the value of price when we're interested in both reasonableness and realism? What do we do when more is less, but less is also less? Is reasonableness a matter of "rightness"? If so, what is rightness? What makes a price "right"?

Price (or estimated cost and fee) is something we evaluate. We assign a value to it in some way. What is the relationship between price (or estimated cost and fee) and value? How can we think clearly about and talk clearly about this in terms that makes sense to ourselves and to others so we can explain and justify our competitive price evaluations?

I might add that value is also dependent on the buyer's circumstances and needs, not just what prospective contractors offer to provide.  For example, if an individual is starving, the value/utility of a cracker to that individual is much higher than for someone who is well fed.

[WhoKnows]

On ‎5‎/‎6‎/‎2016 at 8:24 AM, Vern Edwards said:

What does it mean to say that we are going to evaluate price in a source selection?

Are we really going to determine the value of each price?

If so, then what is the value of a price? Is it the amount of the price itself or can a price have a value that is more or less than its own amount?

If you say that we evaluate a price on the basis of what we'll get for it, then are we evaluating the price or the thing we'll get in return for it?

Probably dumb questions, but it's early in the morning and I'm waiting for a ferry. So...

Vern,

Is what you are contemplating the Labor, ODC/Material obtained for a proposed total price(cost + fee)?

For example with labor: Company A bids $100,000 and will provide a total of 100 hours of labor while Company B bids $100,000 and will only provide 95 hours of labor. Would seem pretty straight forward on the surface but then you could start weighting the value of the Prices and Labor to be performed at a more granular level (i.e. LCats, amount of hours assigned to those LCats, etc).

 

[Guest Vern Edwards]

1 hour ago, WhoKnows said:

Vern,

Is what you are contemplating the Labor, ODC/Material obtained for a proposed total price(cost + fee)?

Nope.

[apsofacto]

In my personal life I'll get a "requisition" from my daughter for an expensive toy (approx. one time per week).  I don't often say "no", but I do say something to the effect of "That's a quarter of our mortgage!".  I'm trying to teach her about opportunity cost.

This is easy since I am the contracting officer, budget officer, project officer and Secretary of the household.  At work, these functions are split between different people.  I'm not saying the Government or a large firm cannot think in terms of opportunity cost- it can in theory.  However, the Contracting Officer cannot cancel a project and apply that budget toward paying down debt, or funnel the money to another use.  You get a requisition for a computer, you tend to buy a computer, you are locked out of alternatives like paying off debt.  I think for this reason opportunity cost may not be a good answer to the OP.

Quote

What does it mean to say that we are going to evaluate price in a source selection?

Are we really going to determine the value of each price?

If so, then what is the value of a price? Is it the amount of the price itself or can a price have a value that is more or less than its own amount?

If you say that we evaluate a price on the basis of what we'll get for it, then are we evaluating the price or the thing we'll get in return for it?

When performing a price evaluation, I find myself making comparisons to other prices (e.g. other prices received in response to the solicitation, other prices paid by my agency, or other agencies, etc.). 

I think mean to use the word "evaluate" in the sense that we are assessing the prices we receive.  Perhaps performing a price comparison is a better way of stating it, and we are using the wrong word when we say "evaluation"?  I think the value of the price is its own amount, but this all feels murky for some reason.  I probably need a better finance background.

Quote

One part of an offer is the proposed price (or the estimated cost and fee). We usually evaluate that part of an offer on the basis of the attribute we call "reasonableness" of amount and sometimes on the basis of the attribute we call "realism." The question is, what is the nature of the functional relationship between value and reasonableness and/or realism of price? Is reasonableness simply a matter of highness or lowness? Does value go up as price goes down? Does it go up as it becomes more realistic? And is the effect on the value of price when we're interested in both reasonableness and realism? What do we do when more is less, but less is also less? Is reasonableness a matter of "rightness"? If so, what is rightness? What makes a price "right"?

Hi, Vern,

Would you say that the following is true?

Value=Utility/Price Paid

[Guest Vern Edwards]

1 hour ago, apsofacto said:

Hi, Vern,

Would you say that the following is true?

Value=Utility/Price Paid

apso:

It's not a matter of what I say is true. What I'm interested in is what you (and others) say is true and why you say it.

In your formula: Value = Utility/Price Paid, what does the solidus (forward slash, fraction slash, stroke) / represent? The arithmetic operation of division? If so, what number do I use for "Value"? What is the quotient? What am I supposed to infer about value from the quotient?

If not division, what operation does the solidus represent, what result do I get from the operation, and what am I supposed to infer from that?

 

[C Culham]

On ‎5‎/‎6‎/‎2016 at 5:24 AM, Vern Edwards said:

What does it mean to say that we are going to evaluate price in a source selection?

No doubt making this too simple but it means we are going to follow the process to apply the procedures stated in the FAR  regarding a review of proposed pricing to award a contract.   

Many of the thoughts posed in further posts such as doing price realism or cost analysis may come into play in doing so but it seems the guiding principle, and dare I say definition, of what is price evaluation is contained in FAR 15.305(a)(1).

[Guest Vern Edwards]

2 minutes ago, C Culham said:

No doubt making this too simple but it means we are going to follow the process to apply the procedures stated in the FAR  regarding a review of proposed pricing to award a contract.  

Carl:

What procedure in FAR, exactly? Please cite section or subsection, so I can understand. Does the procedure yield a value function, as I described above?

[apsofacto]

Quote

In your formula: Value = Utility/Price Paid, what does the solidus (forward slash, fraction slash, stroke) / represent? The arithmetic operation of division? If so, what number do I use for "Value"? What is the quotient? What am I supposed to infer about value from the quotient?

Hi, Vern,

I was trying to convey that value has a direct relationship to utility, and an inverse relationship to the price you pay (division in a way, I suppose) and not much else. 

I don't know if you ever really get a number for value, but we try to assign one at work in the form of a numerical technical score which reflects the strengths and weaknesses of the proposal. 

A straight quotient feels arbitrary unless every procurement considered price and non-price factors to be equal.  If otherwise, you have to put a weighting in there somewhere: V=wU/P

Once you have a this weighting, why not disclose it precisely?  Why not disclose the entire formula?

I don't think the operation should be performed, which I suppose echoes the concern of a certain amphibian:

http://www.wifcon.com/discussion/index.php?/topic/3455-dod-source-selection-procedures-march-31-2016/&page=2#comment-30990

I'd add that utility has no "unit" like pounds or yards because it doesn't have a physical form.  Therefore value can't either if the relationships I described make any sense.  If utility were measures in "yoots", value would be measured in "yoot dollars".  But there are no yoots.  Just youths from New York.

 

[Guest Vern Edwards]

6 hours ago, apsofacto said:

I was trying to convey that value has a direct relationship to utility, and an inverse relationship to the price you pay (division in a way, I suppose) and not much else. 

As I use the word, value is just another name for utility. The relationship between price (or cost) and value is inverse (value goes up as price goes down) only if you are not concerned about price or cost realism.

6 hours ago, apsofacto said:

I don't know if you ever really get a number for value...

You do if you use numerical scoring of value.

6 hours ago, apsofacto said:

A straight quotient feels arbitrary unless every procurement considered price and non-price factors to be equal. 

I don't know what you mean by "straight quotient." A quotient would be the product of an arithmetic operation. As such, it wouldn't be arbitrary. I don't understand what you mean.

6 hours ago, apsofacto said:

Once you have a this weighting, why not disclose it precisely?  Why not disclose the entire formula?

I'm confused. What does the disclosure of weighting have to do with the value function of price?

Quote

I'd add that utility has no "unit" like pounds or yards because it doesn't have a physical form.  Therefore value can't either if the relationships I described make any sense. 

Utility has a unit if you assign a unit to it. It doesn't require a physical form. You could measure utility on a 0 to 100 point scale. It has been done that way by decision analysts for decades. As for an example of the use of the relationship that you described -- Value = Utility/Price Paid -- see TESCO, B-271756, June 24, 1996:

https://www.gpo.gov/fdsys/pkg/GAOREPORTS-B-271756/html/GAOREPORTS-B-271756.htm

That approach was once used by the Navy and some other agencies. It has fallen out of favor.

[apsofacto]

Quote

As I use the word, value is just another name for utility.

I think we are talking past each other because I'm thinking of value and utility as two different things. 

It may not be the proper use of that word, though, so perhaps I should stop.  I must be abbreviating 'value per dollar' or 'value for money' to just 'value'. 

Is there a lot of that going on?  You often hear people say 'This car is a great value', meaning its a good car at a low price. 

[Guest Vern Edwards]

Well, among economists and such, there is a distinction between value and utility, but not the one I think you are making.

See, for example, "Distinction Between Value and Utility" http://chestofbooks.com/finance/Amasa-Walker/The-Science-of-Wealth/Chapter-IV-Distinction-Between-Value-And-Utility.html

See also http://praxeology.net/WS-ITV-E2-3.htm.

That's not easy stuff. Basically, to an economist the value of a bushel of wheat is its price; the utility of that bushel is how much bread you can make with it. In the FAR phrase, "best value," value seems to refer to utility in the economist's sense.

In decision analysis the distinction is based on considerations of risk, but the terms are often used interchangeably. See Edwards and von Winterfeldt, Decision Analysis and Behavioral Research, Chapter 7, "Value and Utility Measurement," pp. 211-215: "In our opinion, the distinction between value and utility is spurious...." Prepare yourself--it's very complicated.

I think that when people say that a car is a great value, they mean that you get a lot for your money. Some people would say that about a Rolls Royce or a Maserati. I think it's true of my Subaru Outback. My wife thinks it's true of her BMW. I disagree.

[C Culham]

Vern – How about a GAO view rather than a FAR view as to the value function.

“In a "best value" procurement, it is the function of the source selection authority to perform a tradeoff between price and non-price factors, that is, to determine whether one proposal's superiority under the non-price factor is worth a higher price. Even where, as here, price is stated to be of less importance than the non-price factors, an agency must meaningfully consider cost or price to the government in making its selection decision. e-LYNXX Corp., B-292761, Dec. 3, 2003, 2003 CPD para. 219 at 7. Specifically, before an agency can select a higher-priced proposal that has been rated technically superior to a lower-priced but acceptable one, the award decision must be supported by a rational explanation of why the higher-rated proposal is, in fact, superior, and explaining why its technical superiority warrants paying a price premium. ACCESS Sys., Inc., B-400623.3, Mar. 4, 2009, 2009 CPD para. 56 at 7.”  (Reference - Coastal Environments, Inc., B-401889, December 18, 2009)

[Guest Vern Edwards]

Carl:

4 hours ago, C Culham said:

Vern – How about a GAO view rather than a FAR view as to the value function.

That's interesting. What that quote suggests to me is that we don't "evaluate" price (or cost and fee) in the sense of assigning it a value function. Here is FAR 15.305(a)(1):

(1) Cost or price evaluation. Normally, competition establishes price reasonableness. Therefore, when contracting on a firm-fixed-price or fixed-price with economic price adjustment basis, comparison of the proposed prices will usually satisfy the requirement to perform a price analysis, and a cost analysis need not be performed. In limited situations, a cost analysis (see 15.403–1(c)(1)(i)(B)) may be appropriate to establish reasonableness of the otherwise successful offeror's price. When contracting on a cost-reimbursement basis, evaluations shall include a cost realism analysis to determine what the Government should realistically expect to pay for the proposed effort, the offeror's understanding of the work, and the offeror's ability to perform the contract. Cost realism analyses may also be used on fixed-price incentive contracts or, in exceptional cases, on other competitive fixed-price-type contracts (see 15.404–1(d)(3)). (See 37.115 for uncompensated overtime evaluation.) The contracting officer shall document the cost or price evaluation.

 

To me, evaluation means to determine the value (usefulness, utility, worth) of something. In order to assign a value to something, we have to establish a value function. But maybe price does not have a value function, because it has no value to the buyer. Rather than being a source of value, it is an unavoidable detriment. Although FAR uses the word evaluation, and the GAO often does the same, maybe what they really mean is just analyze, adjust, and consider. Maybe all we do with price is consider the extent to which it offsets nonprice value. Maybe price, an unavoidable burden of greater or lesser weight, diminishes value in our eyes. We don't evaluate price in the strict sense of assigning it a value function; we only consider it.

 

Quote

[A]n agency must meaningfully consider cost or price to the government in making its selection decision. e-LYNXX Corp., B-292761, Dec. 3, 2003, 2003 CPD para. 219 at 7.

 

 

 

 

[C Culham]

Vern - I agree.  I would add that "to me"  is thrown out for us all we it comes to the doing a Federal acquisition, it is instead is "to the FAR it means".  Consider further FAR 15.002 and the result of a competitive acquisition being best value as defined by FAR 2.101 - “Best value” means the expected outcome of an acquisition that, in the Government’s estimation, provides the greatest overall benefit in response to the requirement.

[Guest Vern Edwards]

49 minutes ago, C Culham said:

I would add that "to me"  is thrown out for us all we it comes to the doing a Federal acquisition, it is instead is "to the FAR it means". 

Carl: I don't understand what that means.

 

 

Please explain.