Jump to content

What are the lessons learned from the JEDI fiasco?


Recommended Posts

DOD has cancelled the Joint Enterprise Defense Infrastructure (JEDI) acquisition. The RFP was issued on July 26, 2018, almost three years ago. The acquisition was strangled in its crib by bid protest litigation. DOD is starting over, and the IT industry is already lining up to file new bid protests.

According to Bloomberg Law, "Key House lawmakers vowed to consider changes to the Pentagon’s arcane technology buying process after senior Defense Department officials canceled its flagship cloud computing program."

What lessons learned should be learned by Congress, the Executive Branch, and DOD?

Link to comment
Share on other sites

I don't know any of JEDI's inside workings, but I would guess that the program manager and especially the contracting officer were not trained, equipped, and empowered to act and make decisions -- especially the contracting officer, since procurement culture seems to want contracting officers to be clerks who mindlessly implement whatever review committees or attorneys or anyone else says instead of professionals who deliver results.  Probably, every single decision had to be reviewed and changed to make everyone happy.  In a discussion with the contracting officer, the chief of the contracting office, the chief of the policy office, one or two procurement analysts, an attorney, the small business specialist, and so forth, the contracting officer's opinion is probably the most irrelevant.  Just guessing...

Link to comment
Share on other sites

If an acquisition is highly competitive and numerous firms can show that they could (have a high probability to) successfully fulfill the requirements for a competitive price,  how about a lottery to select the winner?

The government could include a stipend pool to split among those technically acceptable unsuccessful offers to help offset the cost to compete.

Link to comment
Share on other sites

31 minutes ago, joel hoffman said:

If an acquisition is highly competitive and numerous firms can show that they could (have a high probability to) successfully fulfill the requirements for a competitive price,  how about a lottery to select the winner?

I like that.

Link to comment
Share on other sites

While a lottery might be a reasonable way to choose a grounds maintenance contractor, given a choice of methods I would not select a critically important business partner by lottery. I think there is more to an important business relationship than mere competence, especially when the relationship will bear on military effectiveness and national security.

I don't think proposal-writing contests are a good method, either.

Link to comment
Share on other sites

If multiple firms are qualified with competitive pricing, nothing wrong with maintaining competition during the performance period.  Technology will rapidly change over the next few years.  Having contractors deliver using an “open architecture” approach needs stressed for the good of the country where one company can easily pick up where another left off.

Link to comment
Share on other sites

10 hours ago, formerfed said:

If multiple firms are qualified with competitive pricing, nothing wrong with maintaining competition during the performance period.

Wouldn't that approach be transactional, rather than relational?

If so, would such an approach build trust and a solid foundation for a long-term strategic service relationship between customer and contractor? Wasn't that the kind of relationship that DOD was hoping to build under JEDI—long-term and strategic??

 

Link to comment
Share on other sites

Vern,

Yes, it would be more of a transactional approach.  I think relational business arrangements are important for the government. But in the case of JEDI, it may not be best.  Technology evolves so rapidly in this field.  Who would have envisioned a system that AWS has a few years ago or Microsoft suddenly being a major player?  I don’t know much what JEDI is except a network with a cloud and use of machine learning and AI.  But I’m guessing what we see today will be radically different in a few years.

My question is do we want our national defense to be wed to a single company?  A multiple award approach gives some alternatives.  

Link to comment
Share on other sites

2 hours ago, formerfed said:

My question is do we want our national defense to be wed to a single company?  A multiple award approach gives some alternatives.  

@formerfedHere is how DOD senior management testified about the decision to go single-award with JEDI, as quoted from the GAO's Oracle decision:

Quote

 

The Department [of Defense] has adopted cloud technology in a variety of different ways, but they are very disparate, and in very particular fashions. We have not yet been able to adopt a commercial cloud solution at enterprise scale. And this particular RFP is about trying to do that.

*3 It's also important to understand that doing so is pretty technically complex. The Department today operates a wide variety of different systems that we'll need to integrate. We have a bunch of particular requirements about the way that we deploy and operate systems that we need to account for. And as I mentioned earlier, we have a broad technical base that will have to be brought up to speed on how this technology works.

Doing that for a single solution provided to the Department by either a vendor or a team of vendors is a big lift already. Trying to do that for multiple solutions, with the Department operating as the integrator, would be exceedingly complex. And I don't think we would be successful.... [A]lso worth highlighting is that, we and the Department ... need to get our minds around how to make sure that our data and applications are secure in the cloud environment provided to us, as part of the JEDI Cloud contract. You know, we have got a really complex process today for how we think about managing risk.... [W]e've got to focus on the infrastructure and the platform that we're using. We have to focus on the software, the way it's been configured, the way it's deployed....

And so part of this effort is to work with the winner of the JEDI Cloud contract, so that we can help the Department better understand the risk [it is] accepting, better manage that risk, but also do so in a more timely manner, so that our war fighters get access to applications and services much faster.

And again, trying to do that ... with [one] vendor is a thing, I think, the Department knows how to do. It's going to take a considerable amount of our technical experts. Trying to do that with multiple vendors simultaneously, I just don't think we have the technical expertise to do that well.

 

What do you think?

Link to comment
Share on other sites

@Vern Edwards

I don’t know.  The quoted statements make sense on the surface.  But it poses serious problems in relaying on a single source, there won’t be competition for a long time (this is a ten year contract that’s taken three years to get this far so a single award might really translate into 15 years or more), and what will be the incentive for a contractor to stay with newer technology?

If this contract was awarded five years ago, the likely sources would be IBM, Google, and maybe Adobe.  Now the contenders are AWS and Microsoft.  Sales force is another company that came out of nowhere.

Considering the size and importance of JEDI, there has to be other approaches like contractor for an integrator, dividing up the work into discreet pieces, requiring partnering, and probably others.

Link to comment
Share on other sites

23 hours ago, Vern Edwards said:

Here is how DOD senior management testified about the decision to go single-award with JEDI, as quoted from the GAO's Oracle decision:

"We have not yet been able to adopt a commercial cloud solution at enterprise scale."

...

"And again, trying to do that ... with [one] vendor is a thing, I think, the Department knows how to do. It's going to take a considerable amount of our technical experts. Trying to do that with multiple vendors simultaneously, I just don't think we have the technical expertise to do that well."

Vern,

I want to take a few things off the table, to narrow discussion and answer your original question.  I looked through the Oracle decision at the above quoted part and I think I can infer that these quotes are not about problems with the procurement process, they are about technical security risks of allowing multiple companies into the test and production environments.  See footnote 7 actually quoting from the CO's D&F:

Quote

A single cloud environment does not mean that all data and applications are hosted in a single physical environment where everything is vulnerable to a single attack. Rather, a single cloud environment is subdivided into many virtual private enclaves, like a honeycomb, where applications and data are logically isolated from other users. . . . While security of data within a single cloud is largely standard and automatic, managing security and data accessibility between clouds creates seams that increase security risk for multiple reasons. Crossing clouds requires complex manual configuration that is prone to human error and introduces security vulnerabilities. . . . Systems in different clouds, even when designed to work together, require complex integration. . . . Connections that are not correctly configured and managed at both endpoints introduce new attack vectors. . . . I find that multiple awards increase security risks. AR, Tab 24, Contracting Officer’s Singe-Award MFR, at 458.

The GAO denied Oracle's protest at this part, thus validating the reasonableness of the written findings.  So again I infer their concerns were with resulting performance of the product, not with the procurement process.

I also saw that the decision states only FFP TOs would be awarded.  This helps us rule out the possibility that DOD sought to avoid the need for complex source selection (e.g. Cost Realism Analysis) and/or TO Fair Opportunity processes in choosing to pursue a single-award IDIQ instead of multiple-award.  Sounds like it was truly a D&F that met the intent of the statute.

This leaves a few things on the table for discussion to answer your original question of what improvements Congress, the Executive Branch, and DOD can get from contracting folks.  I submit this, for starters: It is not as difficult as you think to get a single-award IDIQ or a single-award GSA FSS BPA acquisition strategy approved.  It's about as difficult to write up the needed documentation as writing a JOFOC, which happens all the time.  We in our contracting offices can publicize this possibility to our clientele if we can get early buy-in from our respective Policy approvers (whom we can show the Oracle decision as supporting evidence).

Link to comment
Share on other sites

14 minutes ago, WifWaf said:

It is not as difficult as you think to get a single-award IDIQ or a single-award GSA FSS BPA acquisition strategy approved.

It can't be too difficult. There are a lot of single-award IDIQs. But I would still like to get rid of the multiple-award preference.

Link to comment
Share on other sites

22 hours ago, Vern Edwards said:

But I would still like to get rid of the multiple-award preference.

Across the board or case by case?  Continuing the analogy to JOFOCs, CICA is designed so that by default the government begins its procurements from a competitive stance.  In low complexity acquisitions this a good place for Congress to keep the Executive Branch perched.  In a unique acquisition like JEDI it wouldn’t make sense during contract administration for reasons identified in the CO’s D&F.  Congress’s intended benefits weren’t there, or were overcome by security concerns.

Is there a specific type of contractor that could be excepted from having a multiple award preference?  Like how 8(a) below a threshold don’t need a JOFOC (completing the analogy).  Alternatively, is there a specific agency who’s regulation should include an exception to/class deviation from the preference?

Link to comment
Share on other sites

@WifWafI think multiple awards should be an option, not a preference. I would eliminate the policy stated in FAR 16.504(c). I would prohibit protests against a decision to make a single award. Multiple awards are expensive, and I don't think there is substantial evidence that multiple awards and "fair opportunity" competitions have resulted either in lower costs or better quality for the taxpayer.

If anything, I would require justification for the making of multiple awards. To prevent abuses of single award IDIQ contracts I would require tighter "scope" descriptions, an absolute prohibition against post-award scope expansions, and a strict limit on the use of extension options.

But the multiple award preference is a business welfare program. It would be hard to get rid of it.

Link to comment
Share on other sites

@WifWafIn addition to the comments in my last post, I would require that a single award IDIQ contract minimum be based on the winning contractor's proposal preparation costs. I would require competing offerors to state their estimated proposal preparation costs in their proposals, and require that the winning offeror's contract minimum cover its costs. Estimated proposal preparation costs would be an evaluation factor.

That would alert agencies to the proposal preparation costs that they are forcing contractors to incur and might motivate them to think more carefully when writing proposal preparation instructions. It might also motivate offerors not to spend too much on elaborate proposals.

It might put an end to essay-writing contests.

Link to comment
Share on other sites

On 7/15/2021 at 12:25 PM, Vern Edwards said:

But the multiple award preference is a business welfare program. It would be hard to get rid of it.

I agree it would be hard to completely rescind.  But I bet DoD could get an exception from it in the DFARS for their newly defined "Nontraditional Defense Contractors".  Maybe Congress can strike while the iron is still hot on that topic, especially if they want to increase the defense industry pool more rapidly with the best of Silicon Valley.  I would comment on the Proposed Rule (5 years from now of course), to make sure it says something as simple as the below:

"216.504   Indefinite-quantity contracts.

"(c) Multiple award preference—(1) Planning the acquisition. [(i)  The preference for multiple awards does not apply to Nontraditional Defense Contractors.]

"(ii)(D)(1) The senior procurement executive has..."

 

Reference DFARS 202.101:

Quote

Nontraditional defense contractor means an entity that is not currently performing and has not performed any contract or subcontract for DoD that is subject to full coverage under the cost accounting standards prescribed pursuant to 41 U.S.C. 1502 and the regulations implementing such section, for at least the 1-year period preceding the solicitation of sources by DoD for the procurement (10 U.S.C. 2302(9)).

 

Link to comment
Share on other sites

@Vern EdwardsI can provide that opponent perspective, as my DOD office was frightened by how much the 2016 NDAA broadened commerciality without preparation of the workforce or exception of certain industries.  Nontraditional Defense Contractor services' treatment as commercial items was introduced in that NDAA.  It was like we were just supposed to learn how to shift from cost analysis to market-based pricing on a trial-by-error basis.  I even had someone high up in the commercial cadre tell us DOD, "Might see a lot of waste the next 5 years or so," while it gets up to speed on real market research and value analysis pricing.  Sure, we were just used to getting CCoPD by default, but by my reading of "Nontraditional Defense Contractor" I worried this definition was going to be easy to abuse.  Whereas I thought these changes were supposed to fix the government's IT acquisition woes, truly on its face the definition that I provided above has nothing to do with IT at all.  It could apply to every small business in every industry, couldn't it?  Because 9903.201-1 CAS applicability exempts contracts and subcontracts with small businesses from all CAS requirements.

So, an abuse could already play out at DOD in which all subcontracts between any prime and any small business, acquiring [insert any noncommercial service or supply], are claimed by the prime to be excepted from certified cost or pricing data submission requirements by their proposal response to solicitation provision DFARS 252.215-7008(b)(1)(ii)(E), which currently just instructs prime to promise the sub is Nontraditional:

Quote

(b) Exceptions from certified cost or pricing data. (1) In lieu of submitting certified cost or pricing data, the Offeror may submit a written request for exception by submitting the information described in paragraphs (b)(1)(i) and (ii) of this provision. The Contracting Officer may require additional supporting information, but only to the extent necessary to determine whether an exception should be granted and whether the price is fair and reasonable.

...

(ii) Commercial item exception. For a commercial item exception, the Offeror shall submit, at a minimum, information that is adequate for evaluating the reasonableness of the price for this acquisition, including prices at which the same item or similar items have been sold in the commercial market. Such information shall include—

...

(E) For items provided by nontraditional defense contractors, a statement that the entity is not currently performing and has not performed, for at least the 1-year period preceding the solicitation of sources by DoD for the procurement or transaction, any contract or subcontract for DoD that is subject to full coverage under the cost accounting standards prescribed pursuant to 41 U.S.C. 1502 and the regulations implementing such section.

Before award, the prime and the CO then debate and delay over getting a Commercial Item Determination instead, or getting adequate pricing data (e.g. the right pricing on truly comparable terms and conditions), until finally the prime's slow-roll tactic wins, and the CO has to apply DFARS 212.102(a)(iii) to treat it commercial using originally proposed pricing with maybe a minor tweak - and no establishment of cost history to benefit the DOD office's future cost estimating.

All that to say, now under my idea of excepting Nontraditional Defense Contractors from the FAR 16.504(c) Multiple-Award IDIQ Preference, we could add to the potential for overpricing by removing the possibility of any competition challenging the requirement for 15 years after award (as the IDIQ ordering period and final TO POP play out).  Is this now a bad idea?  Or were these fears unfounded?

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...