Treatment & Immobilization of Hanford Radioactive Tank Waste
|
Report to
Congress Treatment & Immobilization of Hanford Radioactive Tank Waste |
APPENDIX B
Expert and Non-Proponent Reviews
B.1 Introduction
Throughout the authorization-to-proceed decision process, the U.S. Department of Energy (DOE) had many different reviews of their work performed by expert reviewers chartered by the DOE Richland Operations Office (DOE-RL) and non-proponent reviewers chartered by DOE Headquarters. These reviews were designed to help assure DOE that it had a program that was well positioned for successful implementation. In addition to the experts brought in to assist in developing specific portions of the process, more than 100 expert reviewers from the following were asked to evaluate the work done for the authorization to proceed:
These reviewers had expertise in a variety of key areas relating to the Tank Waste Remediation System (TWRS) program, such as:
The reviewers were tasked to evaluate the overall decision process as well as specific portions of the contract decision-making process. Extensive expert reviews were conducted for each of the key steps in the process and for the process as a whole, plus the final decision. These expert reviews are shown in Figure B-1.
As a result of the issuance of a report by the National Research Council during the course of this work, Assessing the Need for Independent Project Reviews in the Department of Energy (Duscha 1998), DOE Headquarters also chartered non-proponent reviews of the authorization-to-proceed process, involving outside experts and DOE personnel with no vested interest in Hanford or the outcome of this decision. These non-proponent reviews are shown in Figure B-2. Each of the expert reviews and non-proponent reviews developed an evaluation plan and criteria to evaluate the work, produced a report that identified key concerns or findings, and provided recommendations to address the problems identified.
B.2 Process
In the early stages of this activity, an expert review was conducted specifically for the authorization-to-proceed decision logic, as discussed in Appendix A and shown in Figure A-1, to determine if there were any gaps present, if the process was logical, and whether the ultimate decision would be defensible. Based on the results of this early review, expert reviews were then performed on many of the key steps in the process:
A non-proponent review was performed to evaluate the entire expert review process and all of the key steps listed above. A non-proponent review of the DOE Office of Radiological, Nuclear, and Process Safety for TWRS Privatization (Regulatory Unit) was performed by the Office of Environment, Safety and Health at DOE Headquarters. Finally, a non-proponent review team was asked to evaluate the final decision and to determine if DOE had followed the decision logic it had prepared and whether the decision had been adequately documented in this Report to Congress.
B.3 Review Teams and Results
Each of the review teams varied in its number of personnel, the backgrounds of its personnel, the type of review it performed, and at what time in the process its evaluations were performed. The scope of the readiness-to-proceed reviews is summarized in greater detail in Appendix A.1.
Figure B-2. Non-Proponent Reviews
Overall Authorization-to-Proceed Decision Process. Five individuals from academia, the U.S. Department of Defense, and private industry participated in this expert team early in the review process. Its findings indicated that the decision process "is logically sound and the staff has made an effort to ensure that all technical, cost, and risk concerns are covered in the best-value judgment." Recommendations were provided to strengthen the process, and these recommendations were accepted and incorporated.
Hanford M&I Contractor Readiness-to-Proceed. As part of the process of developing its readiness-to-proceed plan, the Hanford M&I Contractor brought in a team of 19 outside subject matter experts to evaluate its plan. The team concluded, "there is reasonable assurance that the M&I contractor will be able to deliver feed to the privatization contractor." A total of 30 people, split equally between DOE and national laboratory personnel, then performed an evaluation of the Hanford M&I Contractor plan. The results of their review confirmed that the Hanford M&I Contractor was ready to proceed provided certain risks were addressed in the privatization contracts. These risks included adequacy of funding to meet the existing schedule and adequacy of float in the schedule for receipt of the final waste forms. It appears these risks will be reduced substantially by changes proposed by BNFL in the original schedule for construction and operations. The Hanford M&I Contractor plans will be refined during the design phase to assure DOE that the Hanford M&I Contractor is prepared to deliver feed and to store and dispose of the final waste products according to the new schedule proposed by BNFL.
The non-proponent review team determined that the conclusions of the external review team were supportable and adequately documented. The non-proponent review team recommended that DOE expand the self-assessment to fully evaluate any additional interfaces as a result of the contract negotiations. This recommendation was accepted and will be completed through the use of Interface Control Documents and Integrated Product/Process Teams.
DOE Readiness-to-Proceed. The expert review team assembled to conduct this assessment consisted of a former DOE Ohio Field Office Manager and a former U.S. Department of Defense senior manager with expertise in fixed-price contracting. The team concluded that the management assessment performed by DOE-RL was conducted in a structured manner and was credible, and it agreed with the results, observations, and path forward. However, the team expressed concern that additional staffing was needed on the project team to ensure a successful execution of the privatization contract. They also suggested additional training in fixed-price contracting for all staff directly or indirectly involved with the project. As described in Section 6.4, actions are underway to address these concerns.
The non-proponent review supported the conclusions and recommendations of the expert review of the readiness-to-proceed review, particularly those related to staffing limitations. The non-proponent review team determined the expert review "was comprehensive and no significant gaps were identified in the overall methodology." Recommendations provided by the non-proponent review team included developing a time-dependent internal staffing plan, augmenting the DOE staff, and refining the interface review process after Part B is started. These recommendations were accepted, matrixed staff was brought in from other offices within DOE-RL, and the staffing plan is currently being developed. The non-proponent review team also recommended that the Headquarters elements perform a readiness-to-proceed evaluation to ensure that there is a recognition and understanding of the unique interface requirements of this fixed-price contract. This Headquarters readiness-to-proceed evaluation is presently under development. Additional information on the management approach is provided in Section 6, "Management of the TWRS Phase I Project."
DOE Regulatory Unit Readiness-to-Proceed. The Office of Environment, Safety, and Health performed a non-proponent review of the Regulatory Unit. Based on evaluation criteria established by the review team, the team concluded that the Regulatory Unit met the criteria and is ready to regulate in Part B provided identified weaknesses are corrected. These weaknesses are categorized as those requiring completion of corrective actions prior to regulating in the design phase and those that could be addressed during the design phase.
The areas to be addressed prior to regulating in the design phase include the following:
The issues to be addressed early in the design phase include the following:
The Acting Assistant Secretary for Environmental Management issued the EH report, "Independent Assessment of the Tank Waste Remediation System Privatization Regulatory Unit – Readiness for Phase B" on May 1, 1998. Corrective actions are under way, and several of the weaknesses already have been partially or completely resolved.
Environmental Impact Statement-Record of Decision (62 FR 8693) Review: Because of the diverse nature of the three environmental documents that were reviewed for this report, this five-member team had a wide range of backgrounds in waste management, scientific, and technical fields. The objective of this review was to ensure compliance with the National Environmental and Policy Act regulation. The results of its reviews indicated there was no need for a Supplemental Environmental Impact Statement or other National Environmental Policy Act documentation prior to proceeding with Phase I, Part B.
Evaluation Reports of Privatized Contractor Deliverables. The expert review of the privatized contractor deliverables chartered by DOE-RL was comprised of a multi-disciplined team of 13 outside reviewers from academia, private industry, DOE high-level waste sites, and other federal agencies. The reviewers were organized into specific teams, based on individual expertise, to conduct the evaluations for all except the health and safety deliverables, which were reviewed by the Regulatory Unit. The input from these teams was used as the basis for DOE acceptance for payment of deliverables and assessments of technical viability of the contractors. The evaluation concluded that the BNFL team demonstrated that its proposed technical and regulatory approach were viable. The review also indicated that the LMAES technology approach, facility concept, and regulatory strategy had significant risks for both the contractor and DOE, and were determined not to be viable. More detailed results of this review are discussed in Section 4 and Appendix A of this report.
The non-proponent review of this evaluation agreed with the viability determinations and concluded that "omissions, uncertainties, and lack of integration within the LMAES deliverables were systemic and undermined confidence in their successful implementation." It further recommended several specific steps to minimize technical risk to DOE in proceeding with the BNFL proposal. These steps included:
DOE-RL plans to have a symposium during the design phase to evaluate the technical issues relating to technetium separation, and the other issues were addressed during the negotiation process.
Price Reasonableness and Cost Savings. The objectives of the Price Reasonableness and Cost Savings Evaluation were to determine whether the contractor’s price could be determined to be fair and reasonable to the government and whether the private contractor’s approach saved money in comparison to a traditional Management and Operations (M&O) contractor approach. The reviewers evaluated (1) the methods used to determine the costs, (2) the methods used to determine whether the contractor’s price was fair and reasonable, (3) the methods used to determine whether a more equitable risk allocation was possible between DOE and the contractors, and (4) the evaluation of the contractor’s prices on the TWRS life cycle costs. The expert review team that evaluated the financial model consisted of four members from private industry and five from academia.
The non-proponent reviews were chartered by two independent offices at DOE Headquarters. Specifically, the Government Fair Cost Estimate and M&O Cost Estimate were evaluated by two separate teams chartered by the Office of Field Management. These reviews, as well as the review of the financial model, were then evaluated by another non-proponent team chartered by DOE Headquarters. The results of this final review stated each of the areas was reviewed adequately and the teams performing the reviews were qualified. The non-proponent team identified several areas that had not been addressed in the review, including reconciling the M&O cost estimate prepared by the Office of Field Management team with the DOE-RL estimate to harmonize the two cost estimates to support the fixed-unit price determination.
Best-Value Determination: The expert reviewer for this determination noted "the hierarchical structure of the alternatives, their specifications, and the sequence of the decisions are well formulated." The main recommendation was to restructure the list of criteria.
The non-proponent review team that looked at this determination noted that some portions of this evaluation had been subjected to an expert review and some portions had not. However, the non-proponent review team also noted this was a result in part of the fact that negotiations were ongoing at the time of the non-proponent review. The non-proponent review team recommended that the final non-proponent review team evaluating the final ATP decision evaluate the portions that had not been reviewed as part of its analysis, and this was completed by that group. Another recommendation was to perform another Best Programmatic Value Determination at the end of the two-year design phase. The Best Programmatic Value Determination will be continued as part of the optimization steps described in the overall recommendations. (See also Section 5.2.2.)
Contract Negotiation. Three expert reviewers evaluated the DOE’s negotiation position and approach and provided private sector perspectives to the DOE negotiating team, particularly on legal and financial issues. Because of the sensitive nature of this information, the non-proponent evaluation was limited to key DOE officials at Heaquarters, particularly the Office of Procurement and Assistance Management, the Contract Reform and Privatization Project Office, and the Office of General Counsel.
Non-Proponent Review of Overall Authorization-to-Proceed Process. In addition to the specific recommendations provided in the sections listed above, the non-proponent review team identified several overall recommendations. The primary recommendation was to include optimization studies to (1) evaluate alternative financing concepts to ensure that the Phase I, Part B contract has the flexibility to accommodate changes that would benefit the DOE, (2) evaluate the cost and time savings as a result of changes to technical requirements, and (3) complete a programmatic re-evaluation to assure that no better programmatic solution exists. In addition, the team recommended that an "open item tracking list" system should be put in place to assure that all commitments to responses and commitments to actions are completed. The review team concluded that "authorization to proceed to Phase IB should be given" provided the commitments to all expert review recommendations were resolved and the optimization studies were included in the contract.
The optimization studies are either underway or are now included in the contract requirements for completion early in the design phase. The open tracking system has been developed and is close to completion.
Final Authorization-to-Proceed Decision. The role of this five-member team was to determine whether this Report to Congress "clearly and adequately presents the DOE decision and the rationale used to arrive at that decision" and whether "the deal presented represents the best deal for the government compared to other options" evaluated. The team was an expert review team chartered by DOE Headquarters. The results indicated "the fixed price approach should be a better deal for the government both from the standpoint of lower cost and improved performance (higher likelihood of success). However, the magnitude of the financial savings will not be quantified until Phase IB-1 [design phase]." The team supported and endorsed the 24-month period to move from a 5% to a 30% design in the resulting plant. It was stated this period was of great benefit to the government because it tremendously reduced the risk and uncertainties of this project.
B.4 Conclusion
DOE had many different reviews of its work performed by experts and non-proponent reviewers throughout the process of developing the authorization-to-proceed decision in order to assure itself it had a well-positioned program ready for successful implementation. The findings and recommendations strengthened the process. Although not all of the findings will be resolved prior to the design phase, the review teams supported DOE’s plan to proceed with this phase. Outstanding issues are being resolved as part of the design phase.
B.5 References
62 FR 8693, February 26, 1997, "Record of Decision for the Tank Waste Remediation System," Federal Register.
Duscha, L. A., 1998, Assessing the Need for Independent Project Reviews in the Department of Energy, National Research Council, Washington, D.C.
National Environmental Policy Act of 1969, Public Law 91-190, 42 United States Code 4321 et seq.
Proceed to next section