Jump to content
  • entries
    1,831
  • comments
    55
  • views
    73,524

GAO Finds NASA Effectively Evaluated Risks of the Unknown


Koprince Law LLC

487 views

NASA is going back to the moon and is looking for private companies to help get it there. In 2018, NASA awarded nine IDIQ Commercial Lunar Payload Service contracts for commercial payload delivery services between the Earth and the lunar surface. This is a sea-change for NASA as “no [United States] commercial company has ever attempted to launch, transit, and land” on the moon. Prior to award, NASA asked for task order proposals to include a description of risks and mitigation efforts. You might be asking–how can NASA effectively evaluate risk for something that has not been done before? A protester asked the same question, but GAO agreed with NASA’s risk analysis on the project and found the protester’s questions to be mere disagreement with the evaluation.

The risk analysis at issue was discussed in Deep Space Systems, Inc., B-417714 (Comp. Gen. Sept. 26, 2019). The Task Order at issue was for “all activities necessary to safely integrate, transport, and operate NASA payloads using contractor provided assets, including launch vehicles, lunar landers, and associated resources.” Challenging a proposed launch vehicle and lunar lander solution is a highly technical effort. In this post, we’ll highlight a subset of the legal issues and provide links for some of the more technical aspects, if you are interested in more information.

Back to the case at hand. NASA made a best value award to Intuitive Machines, LLC following a technical review of likelihood of successful payload delivery. In other words, NASA viewed Intuitive Machines as one of the less risky payload delivery proposals.

Each offeror was “required to define the key elements associated with their mission and to describe top risks and the related mitigation plans necessary to meet the proposed landing date.” NASA would then assign each proposal a confidence rating based “on offerors’ assessments of their own risk, as well as the steps the offerors would take to mitigate such risk.” Interestingly, NASA admitted that “all proposed approaches received . . . are unproven at this point in time.” Meaning, NASA was supposed to measure risks of unproven technologies and procedures.

This leads to the question of how NASA would evaluate risks of something that had never been done before? Turns out that NASA assessed risk based on the individual components of the payload delivery system and then aggregated these components into a cumulative score. GAO’s take on this? Deep Space Systems provided nothing to overturn NASA’s assigned risk scores, so the agency’s evaluation stands.

The first challenged risk evaluation was of Intuitive Machines’ “plan to use a liquid oxygen and liquid methane (LOX/methane) propulsion approach.” NASA’s evaluation stated that using LOX/methane would “advance the state of the art” but was “unproven” for a full mission.

In response, “NASA asserts that LOX/methane propulsion is a mature technology suitable for present use on space missions.” It is not only considered viable for an Artemis moon lander, but a similar LOX/methane propulsion system was used on NASA’s Morpheus lander. In fact, many from the Morpheus team now work for Intuitive Machines. Of note, the Morpheus lander only went through earth-based testing and was never tested in space; while Artemis is still a work in progress.

NASA’s only caveat was that “using LOX Methane for a full mission is unproven[.]” In other words, the LOX/methane approach has been proven in theory and in testing, just not in a full orbital, lunar, or other, mission. NASA then stated that Intuitive Machines’ risk score reflects this fact.

The second challenged risk evaluation, which is an underlying component of the first, is that Intuitive Machines allegedly did not “adequately explain how the firm would manage all aspects of thermal and cryogenic performance” which may result in “excessive fuel burn off” and increase risk of the mission.

In response to this second challenged evaluation, “NASA disputes the severity of the risk posed by boil-off and [Intuitive Machines’] need to perform cryo-fluid management.” NASA countered that “maintaining sufficiently cold temepratures for LOX and methane storage to prevent boil-off is relatively straightforward” and just required proper insulation of the liquids. In other words, cryo-fluid management was a “manageable risk[.]” While NASA “noted that [Intuitive Machines’] proposal could have benefited from a more thorough discussion of how it would perform” liquid insulation over the mission, “the ease with which this risk could be mitigated did not warrant a lower confidence rating.”

Deep Space Systems’ retorts to NASA’s stances were summarily dismissed as mere disagreement by GAO. GAO repeated its common refrain that “disagreement with [NASA]’s judgment, without more, does not render the evaluation reasonable.” A protest must establish that the evaluation was unreasonable and that it was outside the evaluation factors found in the solicitation.

In support of its LOX/methane challenge, Deep Space Systems compared Intuitive Machines’ approach to most commercial space vehicles which “rely on a single fuel tank . . . that does not require an active ignition process[.]” This was not enough to overcome NASA’s evidence from its Morpheus testing and Artemis planning. In support of its fuel burn off challenge, Deep Space Systems’ argument that the “liquids are difficult to store and to transport” only showcased risks of which NASA was already aware and said were easily mitigable.

In both the LOX/methane and fuel burn off aspects, GAO found that NASA reasonably explained its ratings approaches. Without sound evidence to show NASA’s years of LOX/methane testing and already recognized concerns were unreasonable, GAO found that Deep Space Systems’ position was just a “disagreement with NASA’s judgment . . . which is insufficient for [GAO] to sustain the protest.”

Notably, GAO did not opine on NASA’s ability to holistically evaluate proposals for a project that is “unproven at this point in time.” NASA evaluated the risk of each component of Intuitive Machines’ proposal and GAO found this evaluation approach reasonable. Even though this type of project has never been performed, Deep Space Systems was unable to show NASA’s evaluation methodology was unreasonable, and GAO dismissed the protest arguments as mere disagreement with NASA’s evaluations.

Disagreeing with an agency, with nothing more, will almost always end up with GAO siding with the agency. Let us know if you need help bolstering your arguments, whether the contract is to send supplies to the moon or just across town.


View the full article

1 Comment


Recommended Comments

By all means, if the protester (or anyone else) knows a better way to evaluate risk for a lunar mission, they should share their thoughts with NASA -- it's their duty as patriotic citizens. 

Link to comment
Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...