V. Regulatory Standard for CE Newspaper
Contracting
This procurement is governed by 32 C.F.R. pt. 247 (2010),
“Department of
Defense Newspapers, Magazines and Civilian Enterprise
Publications.”4 It is
important to note that contracting for CE newspapers is not
governed by the
Federal Acquisition Regulation (FAR) or the Defense Federal
Acquisition
Regulation Supplement (DFARS). See 32 C.F.R. pt. 247 App. B(J)(2)
(stating
that “CE contracts are not subject to the FAR or DFARS, because
they do not
involve the expenditure of appropriated funds”). For this
reason, the arguments of
the parties that reference specific FAR provisions, particularly
those that set
standards for communications between offerors and a procuring
authority, are largely unhelpful.6 In general, the government
must “ensure that the [procurement]
process is objective and fair.” 32 C.F.R. pt. 247 App. B(J)(8)(c).
The [Letter Request for Proposal] LRFP in this procurement references not the applicable
regulation, but
Department of Defense Instruction (DODI) 5120.4 (dated June 16,
1997), a
document which largely replicates the standards and information
contained in 32
C.F.R. pt. 247. The governing regulation and DODI 5120.4 each
include an
attached sample source selection plan and a sample evaluation
scoring sheet, which
are similar if not identical in content. Compare 32 C.F.R. pt.
247 App. B Atts. 1-2
with DODI 5120.4 Enc. 4 Atts. 1-2. The parties disagree as to
whether the
numerical scoring system set forth in the sample source
selection plan and scoring
sheet is required by the regulation, or is simply provided as an
example. The court
agrees with defendant and TVP that the numerical scoring system
offered as an
example by the regulation provides guidance to the Army, but
does not impose
this, or any other, numerical scoring system as a requirement
for this procurement.
VI. The Source Selection Plan Contained Two
Conflicting Sets of
Evaluation Criteria, Conflicting Weighting Schemes, and Was Not
Approved in Accordance with the Governing Regulation
According to the introduction to the
[Source Selection Plan] SSP,
this document “describe[d] the
organization, responsibilities, evaluation process, and
instructions to the evaluation
committee for the competitive award of a civilian printer for
the civilian enterprise
(CE) installation newspaper, hereinafter referred to as the
Redstone Rocket.” AR
Tab 4 at 3. The purpose of the SSP was to “to ensure an
impartial; equitable, and thorough evaluation of proposals;
ensure that the technical
evaluation findings provide for the selection of the offer most
advantageous to the
Government; and document all aspects of the evaluation and
decision process.” Id.
According to the governing regulation, the [evaluation
committee] EC must follow the SSP, because the
SSP “serve[s] as a guide for the personnel involved and ensure[s]
a fair and
objective process and a successful outcome.” 32 C.F.R. pt. 247
App. B(J)(8)(b)-
(9).
(sections deleted)
In the end, this analysis of the
differences between Evaluation Criteria Set A
and Evaluation Criteria Set B is disturbing. The evaluation
team, the EC, was
given two very different sets of evaluation criteria, and it is
difficult to imagine
how such an evaluation process could reconcile the conflicting
descriptions of the
evaluation criteria and apply these evaluation criteria with any
degree of
uniformity or accuracy. Indeed, proceeding with an internally
inconsistent source
selection plan appears to the court to be an irrational
enterprise, not a rational one.
It is also disturbing to note that the EC relied heavily on the
descriptions of subfactors
of criteria found in Set B, when Set B differed so significantly
from the
descriptions of evaluation criteria sub-factors advertised in
the LRFP. See CACI
Field Servs., Inc. v. United States, 13 Cl. Ct. 718, 728 (1987)
(noting that if the
government “utilized the evaluation criteria listed in the SSP
rather than the RFP,
and if the SSP evaluation factors were significantly different
from the RFP
evaluation factors (in terms of content and/or relative
importance), then [the
protestor] would be entitled to relief assuming that it was
prejudiced by such
actions.”), aff’d, 854 F.2d 464 (Fed. Cir. 1988). The court now
turns to another
flaw in the rating procedures set forth in the SSP.
B. Differing Weighting Schemes
As stated in the background section of this
opinion, the weighting scheme in
the LRFP informed offerors that Technical Capability and
Services Offered were
approximately equal in weight, Past Performance had
approximately half the
weight of Technical Capability or Services Offered, and
Management Approach
had less weight than Past Performance. AR Tab 5 at 3. The SSP
has three
weighting pronouncements, all of them distinguishable from the
weighting scheme
announced in the LRFP. First, in what appears to be the
weighting scheme to be
applied to Evaluation Criteria Set A,9 the SSP states that:
Technical and Production Capability and
Services and Items Offered are approximately equal in
importance. Technical and Production Capability and
Services and
Items Offered are each approximately twice as important
as Past Performance. Past Performance is approximately
twice as important as Management Approach.
AR Tab 4 at 7. Aside from a slight
difference in describing the weight of the
Management Approach rating, this pronouncement is roughly
consistent with the
LRFP.
Next, however, within the text of the
Services Offered evaluation criteria,
Set A announces that the sub-factors announced in Services
Offered (version Set
A) are “listed in order of relative importance.” Id. at 8. The
same sub-factors in
the LRFP are not similarly designated, AR Tab 5 at 2, and many
offerors with
experience in FAR-governed procurements would assume that the
listed Services
Offered sub-factors were equal in importance, see, e.g., Bean
Stuyvesant, L.L.C. v.
United States, 48 Fed. Cl. 303, 321 (2000) (“In the absence of a
statement of the
relative importance of the factors or subfactors, [the FAR
instructs that] each factor
or subfactor must be weighed equally.” (citing Isratex, Inc. v.
United States, 25 Cl.
Ct. 223, 229 (1992))). Finally, Set B contains an assignment of
weight to the four
evaluation factors, so that the percent weighting is as follows:
Technical
Capability (40%); Services Offered (30%); Past Performance
(20%), and
Management Approach (10%). AR Tab 4 at 11-13. The court notes
that most
mathematicians would dispute that 40% is approximately equal to
30%, and finds
that Set B’s weighting scheme varies significantly from the
weighting scheme set
forth in the LRFP.
Having thus discovered that the SSP
contains weighting schemes that differ
from the scheme presented in the LRFP, the court cannot
immediately determine
the gravity of this error. The court found no evidence of
mathematical scoring in
the administrative record, and defendant’s counsel agreed that
there is no evidence
before the court that “numerical scoring” was utilized by the
Army in this
procurement. Oral Arg. Tr. at 53. Nor has the court found any
evidence that the
EC or the SSA adhered to, or even applied, the weighting scheme
presented to offerors in the LRFP. To the extent that the two
competing proposals for this CE
newspaper contract each had a substantial chance of award, as
discussed below, the
court finds the absence of any evidence of the application of
the weighting scheme
set forth in the LRFP troubling.
(sections deleted)
VII. Evaluation Errors Committed by the
Evaluation Committee
In addition to the creation of a flawed SSP, the failure to get
the SSP
approved before offers were received on August 12, 2010, and the
failure to
document any weighting of the evaluation criteria ratings, this
court has found
significant errors in every area of the consensus evaluation
produced by the EC.
The court turns first to Technical Capability, one of the two
most important
evaluation criteria.
(sections deleted)
IX. Significantly Flawed Award Decision
and Prejudice to the Protestor
The court has closely examined this
procurement and found (1) procedural
errors in establishing the SSP; (2) a confusing and internally
inconsistent SSP; (3)
ratings that were based on evaluation criteria different from
those stated in the
LRFP; (4) ratings that were irrational or were in violation of
the governing
regulation; and (5) a failure to apply the weighting scheme for
evaluation criteria
set forth in the LRFP. These errors are significant, and the
court finds that the
decision to award this contract to TVP was “‘arbitrary,
capricious, an abuse of
discretion, or otherwise not in accordance with law.’” Banknote,
365 F.3d at 1350-
51 (quoting Advanced Data Concepts, 216 F.3d at 1057-58). The
SSA’s award
decision, although rational, in the main, as to the different
ratings assigned to
Huntsville and TVP in Management Approach and Services Offered,
was arbitrary
and capricious in the remainder of its assessment of proposals.
These errors are
too significant to disregard as harmless or de minimis errors.
(sections deleted)
XI. Nature of the Permanent Injunction
When the injunctive relief factors are
weighed together, the court concludes
that a permanent injunction must issue. The award of Contract
No. W9124P-10-RA001
must be set aside. The court has considered a variety of
solutions to the
hardships faced by Huntsville, TVP and the Army in the wake of a
permanent
injunction. In the court’s view, the injunction must be timed to
take effect on May
23, 2011. This timing should allow for an orderly transition and
a fair opportunity
to compete for the remaining six months of the first year of
contract performance
(and the follow-on option years) improperly awarded to TVP, if
the Army chooses
to award a new contract for the publication of the Redstone
Rocket.
The court does not set any restrictions on the Army’s options
for the
re-solicitation of proposals, or for the re-evaluation of
proposals received under
Letter Request for Proposals W9124P-10-R-A005. The court notes,
however, that
a related protest would be reviewed under the standards
referenced in this opinion.
The Clerk’s Office will assign a related protest to the
undersigned. (The Huntsville
Times Co. Inc. v. U. S. and Tennessee Valley Printing Co., Inc.,
No. 10-812C, March 31, 2011) (pdf)
C. The Relevance of Source Selection Plans
to Court Review
As the ratings used in the technical evaluation of each
offeror’s proposal, and their
definitions, were provided in the SSEP rather than the
Solicitation,21 a consideration of the
relevance of a source selection plan to our Court’s review is in
order. As the government notes,
see Def.’s Br. at 33-34, our Court has acknowledged the
long-standing rule of the GAO that
source selection “plans generally do not give outside parties
any rights.” ManTech Telecomms.
& Info. Sys. Corp. v. United States, 49 Fed. Cl. 57, 67 (2001).
The rule traces back to a decision
in which the GAO analogized such plans to directives and other
internal agency regulations, and
found that failure to comply with such an “internal instruction”
was no “basis for questioning the
validity of [an] award since an internal agency guideline does
not have the force and effect of
law.” Robert E. Derecktor of R.I., Inc., 84-1 CPD ¶ 40, 1984 WL
43785, at *4 (Comp. Gen. Feb.
2, 1984).
Thus, unlike the treatment of a statute, the prejudicial
violation of a source selection plan
is not, per se, a ground for a protest. Since the agency is free
to change or waive its internal
policies, under the GAO approach “agencies are permitted to
deviate from their stated evaluation
plans so long as the agency’s evaluation is reasonable.” Textron
Marine Sys., 91-2 CPD ¶ 162,
1991 U.S. Comp. Gen. LEXIS 967, at *9 n.3 (Comp. Gen. Aug. 19,
1991). On the other hand,
adherence to a source selection plan may result in an
unreasonable procurement decision and
thus a protest that is sustained -- for instance, when an
evaluation methodology blurs all technical
distinctions and effectively makes a lesser important price
factor decisive. See Trijicon, Inc., 71
Comp. Gen. 41, 91-2 CPD ¶ 375, 1991 U.S. Comp. Gen. LEXIS 1245,
at *10-12 (1991); see also
The MIL Corp., 2005 CPD ¶ 29, 2004 U.S. Comp. Gen. LEXIS 283, at
*9-11 (Comp. Gen. Dec.
30, 2004) (finding evaluation unreasonable when an unfavorable
rating was given to an offeror
for a past performance subfactor concerning which the offeror
had no relevant information). To
the GAO, what is relevant is what an agency actually did in its
evaluation, not what it may have
intended to do -- unless the intention was expressed in the
Solicitation itself. See Frank E. Basil, Inc., 69 Comp. Gen.
472, 90-1 CPD ¶ 492, 1990 U.S. Comp. Gen. LEXIS 518, at *5-6
(1990)
(holding that “agencies do not have the discretion to announce
in a solicitation that one
evaluation plan will be used and then follow another in the
actual evaluation,” unless offerors are
informed and given the “opportunity to structure their proposals
with the new evaluation scheme
in mind”).
This Court has developed a refinement of
the GAO approach, which recognizes that the
act of choosing an evaluation methodology is itself a
discretionary decision in the evaluation
process, and which takes stock of the natural and logical
consequences of this act. See, e.g., Fort
Carson, 71 Fed. Cl. at 592-93; Beta Analytics, 67 Fed. Cl. at
399, 407. The FAR does not
require that agencies use any particular rating method, see 48
C.F.R. § 15.305(a), or disclose
rating methods in the solicitation, except for the “approach for
evaluating past performance.” 48
C.F.R. §§ 15.304(d), 15.305(a)(2)(ii). No law or regulation
appears to prevent an agency’s
changing from one undisclosed rating method to another, and in
that respect a source selection
plan is similar to general guidelines or internal policies which
are adopted by an agency. But
source selection plans differ from these other internal policies
in a significant respect -- as,
presumably, when an agency adopts a source selection plan for a
particular procurement, its
officials are employing their expertise to select a rating
methodology they believe will best meet
the agency’s needs that are the subject of the specific
solicitation.
In a negotiated procurement under the FAR,
the SSA, acting on behalf of the agency
head, is ultimately responsible for the evaluation and the best
value decision. 48 C.F.R.
§§ 15.303(b), 15.308. He must assemble an “evaluation team”
which is “tailored for the
particular acquisition” and possesses the “expertise to ensure a
comprehensive evaluation of
offers.” 48 C.F.R. § 15.303(b)(1). He must ensure that the
agency is consistent in the
information requested of offerors, and that “proposals are
evaluated based solely on the factors
and subfactors contained in the solicitation.” 48 C.F.R. §
15.303(b)(3)-(4). And although “the
SSA may use reports and analyses prepared by others, the source
selection decision shall
represent the SSA’s independent judgment.” 48 C.F.R. § 15.308.
On this last point, nothing
prevents the SSA from basing his judgment upon the evaluations
and ratings of others, and
indeed the provision cited expressly allows the SSA’s decision
to be based on “business
judgments and tradeoffs made or relied on by the SSA.” Id.
(emphasis added).
Since the SSA is responsible for the source
selection decision, and will want to efficiently
make use of the talents and expertise of the evaluation team,
the contents of a source selection
plan guiding his subordinates would naturally be his concern.
For Department of Defense
procurements, including those of the military departments, the
Defense Federal Acquisition
Regulation Supplement (“DFARS”) requires that for “high-dollar
value” and certain other
acquisitions, the SSA “shall approve a source selection plan
before the solicitation is issued.” 48
C.F.R. § 215.303(b)(2).22 A mandatory procedure, see 48 C.F.R.§
202.101 (definition for “Procedures, Guidance, and Information”
(“PGI”)), requires that this source selection plan
include “[a] description of the evaluation process, including
specific procedures and techniques
to be used in evaluating proposals.” PGI § 215.303(b)(2)(C)(3).
Thus, at least where procedures
similar to the Defense Department’s are followed, the source
selection plan is drawn up before
any proposals are seen by evaluators, and is expressly approved
by the SSA.23 The evaluation
procedures and techniques embodied in these plans may vary, of
course, given the particular
needs being addressed in a procurement, and could employ
detailed and objective rules, on the
one hand, or more subjective and deferential standards, on the
other. But the important point for
our purposes is that once the SSA approves a particular source
selection plan, he should expect,
unless informed to the contrary, that it has been followed.
Thus, mandatory instructions that
evaluators “must” and “shall” rate proposals a certain way are
fixed among the foundational
assumptions of the SSA.
Because the SSA relies on the evaluators
working for him to follow source selection plan
mandates, departures from the plan could undermine the
rationality of the ultimate source
selection decision. He might believe, for instance, that a
certain rating was produced by one
formula, when in fact another formula was used by evaluators,
creating a disconnect between his
decision and the underlying facts. Now, it could be the case
that the formula used was better
adapted to the needs of the agency than the original one
selected -- but it is also possible that the
change in formula was an unintended error or, perhaps worse,
reflected a (conscious or
unconscious) reaction to the actual features and qualities
presented by a particular proposal.
Thus, the failure of evaluators to follow the specific
procedures and techniques mandated by a
source selection plan, to the prejudice of a protester, could be
evidence of an erroneous or biased
evaluation. See Fort Carson, 71 Fed. Cl. at 592-93. Any such
suspicions could be dispelled by
records showing that the departure from the plan was consciously
chosen prior to the viewing of
proposals, or articulating a rational reason for the departure.
Although an agency may have wide
discretion in selecting the procedures and techniques to be used
in an evaluation, provided these
have some possible relation to the relevant factors and
subfactors, when its evaluators choose to
depart from these after being exposed to offerors’ proposals,
the integrity of the process would
demand that some reason be documented. And to ensure that there
is no disconnect between the
SSA’s decision and its underlying assumptions, the choice to
depart must have been made known
to him so that it may receive at least implied approval.
There are, of course, many good reasons why
a source selection plan may be changed
midstream, and documentation of these is but a minor burden. The
record in this case provides
one example of this, as the SSEP was changed to eliminate a
consideration from the ratings
definitions for the SBPP factor because this consideration was
not contained in the Factor
elements described in the Solicitation. AR at 10447-48. This
change was memorialized in a
contemporaneous memorandum from the CO, id., and resulted in a
revised SSEP, AR at 10449-
98, that was expressly approved by the SSA. See AR at 10450. In
other circumstances, the
evaluators may conclude that the plan’s definitions are too
rigid or otherwise not well-suited for
the factors they are judging. But if no reason is given for
departing from a source selection plan
(or the departure is not highlighted to allow the SSA to
articulate a reason), a departure could be
due to error and the resulting ratings may be different than the
evaluators intended. Or, in
extreme (and, the Court hopes, rare) cases, the departure could
have been intended to benefit a
particular offeror.24 Particularly when the SSA bases his
ultimate decision not on the proposals
themselves but on briefings in which the ratings are presented
as the inputs for his calculus, an
unjustified departure from a source selection plan may rob this
ultimate decision of its rational
basis.
Source selection plans could be relevant to
court review for other reasons. The regularity
presumed by an SSA may also be presumed by the court. Thus, if
the source selection plan
required a certain event to take place, such as a consensus
meeting, the presumption of regularity
will support that event’s occurrence, absent other record
evidence. Cf. Beta Analytics, 67 Fed.
Cl. at 397 (finding no presumption because there was no plan
requirement) (citing Tecom, Inc. v.
United States, 66 Fed. Cl. 736, 769-70 (2005)). Under this
presumption, “predicate acts that
were required of public officials could be presumed upon proof
of their natural results.” Tecom,
66 Fed. Cl. at 769. The presumption can apply to substantive
matters, such as the inference that
a particular finding was made to support a rating, see Beta
Analytics, 67 Fed. Cl. at 400, or to
procedural matters -- for instance when a plan articulates the
circumstances under which fewer
than all proposals would get a second review. Cf. id. at 407
(finding no plan provision justifying>
a reevaluation for just one offeror). In the latter case, a
decision that at first glance might appear
to treat offerors unequally would instead be seen as the fair
application of a preexisting rule
(under which not every offeror qualified for the particular
treatment).
A source selection plan may also be the
source of a restriction on evaluators, foreclosing
certain actions. Thus, an action that might initially appear to
be fair procedurally, such as a rescoring
of all proposals, could be the ground for a successful protest
when such reevaluations are
specifically forbidden by the source selection plan. But in the
absence of such a restriction --
even if the reevaluation resulted in a change in ordering or
ranking of offerors -- the action would not be objectionable, so
long as the offerors were given equal treatment. See Fort
Carson,
71 Fed. Cl. at 599.
And, of course, when a source selection
plan is the source of the ratings definitions that
are followed in the course of evaluating offerors, it figures
prominently in court review of a
procurement decision. This review “entails identifying the
judgments made by the relevant
officials and verifying that the relevant information was
considered, the relevant factors were
employed, and a satisfactory explanation was articulated.” Id.
at 592 (citing Overton Park, 401
U.S. at 416 and Motor Vehicle Mfrs. Ass’n, 463 U.S. at 43). A
procurement decision’s
explanation necessarily includes the ratings assigned to
proposals. When a rating, given its
definition, cannot be squared with an actual proposal, the
decision to assign that rating is
arbitrary. But since courts are not to second guess
discretionary determinations that are technical
and subjective in nature, see E.W. Bliss, 77 F.3d at 449, the
less objective the rating criteria
happen to be, the harder it is for a protester to establish that
a rating decision was arbitrary. See
Beta Analytics, 67 Fed. Cl. at 399.
With these various ways in which a source
selection plan may be relevant to court review,
it is little wonder that the source selection plan is identified
in our rules as among the “relevant
core documents” of the administrative record which may be
produced early to expedite a case.
RCFC App. C, ¶ 22(b). (USfalcon,
Inc., v. U. S., No. 09-602c, May 21, 2010) (pdf) |
