Capabilities for uncertainty quantification (UQ) with respect to nuclear data have been developed at PSI in the recent years and applied to the UAM benchmark. The guiding principle for the PSI UQ development has been to implement nonintrusive “black box” UQ techniques in stateoftheart, productionquality codes used already for routine analyses. Two complimentary UQ techniques have been developed thus far: (i) direct perturbation (DP) and (ii) stochastic sampling (SS). The DP technique is, first and foremost, a robust and versatile sensitivity coefficient calculation, applicable to all types of input and output. Using standard uncertainty propagation, the sensitivity coefficients are folded with variance/covariance matrices (VCMs) leading to a local firstorder UQ method. The complementary SS technique samples uncertain inputs according to their joint probability distributions and provides a global, allorder UQ method. This paper describes both DP and SS implemented in the lattice physics code CASMO5MX (a special PSImodified version of CASMO5M) and a preliminary SS technique implemented in MCNPX, routinely used in criticality safety and fluence analyses. Results are presented for the UAM benchmark exercises I1 (cell) and I2 (assembly).
The OECD/NEA benchmark for uncertainty analysis in modeling (UAM) was launched a few years ago to promote the development, assessment, and integration of comprehensive uncertainty quantification (UQ) methods in bestestimate multiphysics coupled simulations of LWRs during normal as well as transient conditions [
In order to rigorously establish the accuracy (or bias) of the socalled bestestimate codes, the precision (or uncertainty) must be quantified. (The measure of accuracy is bias: low accuracy implies a large bias and high accuracy implies a small bias. The measure of precision is uncertainty: low precision implies large uncertainty and high precision implies small uncertainty.) This includes propagation of input uncertainty (all inputs are really distributions) to output uncertainty, which is the basic task of UQ. The most straightforward benefit of UQ is the new information about the distribution of outputs which can be used to qualify designs and/or provide confidence in results. However, with UQ a much more rigorous validation procedure is also available and the value of this should not be underestimated. With UQ, one can compare
Consider input,
The cornerstone of local, firstorder UQ methods is the capability to calculate sensitivity coefficients:
Nonintrusive SA can then be implemented simply as a numerical differentiation of
Although DP can be straightforwardly extended to simultaneously estimate
Using the calculated sensitivity coefficients in UQ simply requires the classic firstorder uncertainty propagation formula [
Samplingbased UQ, or stochastic sampling (SS), has been historically used for nonlinear systems with few correlated parameters [
In the case of the distribution of nuclear data, one generally assumes that the input
Decompose VCM
Make
The random samples are then given as
The term “Choleskylike” is used because a true Cholesky factorization requires a (square) symmetric positive definite (SPD) matrix whereas a general VCM can be symmetric positive
Given
In neutronics UQ, the variance (or standard deviation) is used most often as the measure of uncertainty. With UQ methods based on the uncertainty propagation formula (e.g., DP), the variance of outputs is simply the diagonal of the output VCM. With SS, it is convenient to use the sample variance from sample statistics:
Although both direct perturbation (DP) and stochastic sampling (SS) schemes are “nonintrusive” by nature, in order to develop UQ techniques for the CASMO5 M lattice physics code, some source modifications were necessary as CASMO5M’s nuclear data library is stored in a proprietary binary format and “perturbed libraries” could not be easily created.
For the relatively newer developments concerning UQ with MCNPX, ACE format libraries may be created directly and thus no source code modifications of MCNPX are required. The following sections will first describe the CASMO5MX code, then the DP and SS techniques as designed for use with CASMO5MX, and finally the SS technique development for MCNPX.
The capability to perturb the nuclear data library of the lattice physics code is the first step in order to perform any “nonintrusive” UQ with respect to nuclear data. Because of the aforementioned proprietary nature of CASMO5M’s 586group ENDF/BVII.R0based nuclear data library, source code modifications were the easiest way to gain access to this library to perform perturbations. For this purpose, a special module called “PERTXS” and a corresponding crosssection (XS) “perturbation file” was developed. The perturbation file can simply be thought of as a new (optional) input file that demands nuclear data perturbations to apply to the nominal library at runtime. This PSImodified version of CASMO5 M will hereon be referred to as CASMO5MX and the DP technique has been described in [
Currently CASMO5MX allows
elastic scattering
inelastic scattering
fission
capture
average neutrons per fission
average fission spectrum
In addition, external utilities have been created to perturb any parameter contained in the standard input file, facilitating sensitivity/uncertainty analysis with respect to such parameters as clad thickness, fuel enrichment, and so forth. With nuclear data perturbations, it is important to understand that perturbations are made
A very convenient feature of CASMO5MX is that perturbations may be supplied in any group structure, for example, the 19group “coarse” structure used by default in CASMO5 M for UO2 assembly method of characteristics (MOC) transport calculations, an arbitrary twogroup structure, or the full 586group library structure. However, using coarse groups for perturbations keeps data files smaller and in most cases, it has been found that using a very fine group structure (e.g., 586 library groups) does not significantly alter the final output uncertainty estimates. (A small study of this will be provided later.) Additionally, because the underlying VCM data is in the SCALE6 44group structure, it does not make too much sense to go beyond this. Inside CASMO5MX, the following perturbation formulas are used to map perturbations from the input group structure to the 586group library structure:
Aligned and nonaligned coarse perturbation group structures.
The ability to supply perturbations in any group structure effectively gives the user the ability to generate sensitivity profiles at different resolutions for different reactions. For example, to simply evaluate the orderofmagnitude effect for a particular reaction, twogroup perturbations could be supplied. If the sensitivity is high, perturbations in a finer structure could be made to generate a refined sensitivity profile. The limiting resolution is simply that of the underlying 586group library.
The uncertainty in groupwise nuclear data is typically expressed only in terms of variance/covariance matrices (VCMs), which implies an underlying Gaussian (normal) distribution of the data. At the singlenuclide, singlereaction level with
With correlations that cannot be neglected and huge datasets (e.g., 300 nuclides with 44 energy groups and 6 reactions is about 80,000 “inputs”), nuclear data uncertainty propagation is difficult and unique. Because this data is only recently being fully utilized, there are few choices for robust and reasonable VCM evaluations. The SCALE6 VCM [
Because the SCALE6 VCM library is provided in a 44group structure,
make perturbations in the 44group structure, relying on (
convert the 44group VCMs to a different group structure, ideally to a coarse group structure
The second option has been investigated and the code ANGELO which performs the conversion has been provided for the purposes of this benchmark [
Many lattice physics codes, including CASMO5 M, store a single “combined scattering matrix” for each nuclide, lumping elastic and inelastic scattering with the
However, it became apparent that the combined treatment tends to underestimate the uncertainty due, in particular, to inelastic scattering in U238 [
The way that resonance selfshielding is performed in CASMO5M makes it difficult to perturb nuclear data
The main difficulties applying the DP technique to calculate sensitivity coefficients, namely, fixedprecision and eliminating secondorder and higher effects, have been overcome using an adaptive technique [
a scoping calculation is used to assess the magnitude of the response change;
then extra calculations are made which satisfy precision requirements;
finally a polynomial fit (linear or parabolic) is constructed from the pool of available calculations and used to estimate the sensitivity coefficient.
Numerous schemes have been designed within this general framework, for example, using one or two scoping calculations and one or two extra calculations, for a range of two to four calculations per input parameter. Clearly with nuclear data one cannot hope to perform DP on all 80,000 parameters. However, CASMO5MX/DP serves numerous purposes:
provide sensitivity profiles for codetocode comparisons (e.g., with SCALE6 TSUNAMI),
provide reference local, firstorder uncertainty results to assess other CASMO5MX methodologies, such as SS,
provide sensitivity coefficients for nonnuclear data parameters, for example, fuel enrichment.
Figure
Summary of DP modes.
DP mode  Available results  Fit used to estimate 

2point simple 

Linear (not robust) 
3point adaptive 

Linear using 
4point adaptive 

Parabolic using 
Flowchart of CASMO5MX/DP direct perturbation methodology.
Although Figure
The CASMO5MX stochastic sampling (SS) methodology from [
DP varies a single input parameter at a time
DP is first a sensitivity analysis technique and with UQ possible through local and firstorder uncertainty propagation, whereas SS is first a UQ technique (global and allorder) with approximation due to a finite sample size.
Due to the adaptive nature, the robust DP presented requires serial execution of up to 4 cases (although sensitivities of different inputs may be investigated simultaneously) whereas SS is inherently parallel.
Flowchart of CASMO5MX/SS stochastic sampling methodology.
The basic sequence in SS (refer to Figure
Each input is sampled
CASMO5MX is run
The distribution of the
Note that, in Figure
In parallel to CASMO5MX/SS, activities to implement SS in the Monte Carlo code MCNPX have led to the development of MCNPX plus nuclear data uncertainty with stochastic sampling, MCNPX/NUSS, which functions very similarly to CASMO5MX/SS, except that due to the open nature of the MCNPX ACE library format, it is possible to create perturbed nuclear data libraries and source code modification are not necessary, as shown in Figure
Flowchart of nuclear data UQ with MCNPX/NUSS.
Because the currently used VCM library is based on the SCALE 44group structure, data perturbations
An overview of the UAM Phase I cases analyzed in this paper is provided in Table
Overview of UAM Phase 1 cases analyzed with CASMO5MX (C5) and MCNPX/NUSS (MC).
Exercise  Model  Fuel  Cond.  State point parameters  Code  

Bor. (ppm) 



CR  C5  MC  
I1  PB2 BWR cell  UO2  HZP  0  552.83  0.754  0  No  X  X 
HFP  0  900  0.461  40  No  X  —  
TMI1 PWR cell  UO2  HZP  0  551  0.748  0  No  X  X  
HFP  0  900  0.766  0  No  X  ——  
GenIII cell  MOX  HFP  0  900  0.701  0  No  X  —  
 
I2  TMI1 PWR lattice  UO2  HFP  0  900  0.748  0  Yes/No  X  — 
This section presents CASMO5MX results for both exercises I1 and I2. All uncertainty results are in terms of relative standard deviation in percent. For both CASMO5MX/DP and SS, perturbations are made in the 19group CASMO5M group structure, unless otherwise noted. The number of samples used was
The uncertainty summary of exercise I1 cases is given in Table
Uncertainty summary for exercise I1 PB2 cases.
Parameter  PB2 HZP  PB2 HFP  

C5MX/DP  C5MX/SS  C5MX/DP  C5MX/SS  
Eigenvalue  1.3454 ± 0.55%  0.54%  1.2290 ± 0.66%  0.66% 
U235 abs.  60.5 b ± 0.99%  1.01%  40.7 b ± 1.22%  1.23% 
U235 fis.  49.7 b ± 1.01%  1.02%  32.8 b ± 1.23%  1.23% 
U238 abs.  0.915 b ± 1.08%  1.09%  0.852 b ± 1.07%  1.10% 
U238 fis.  0.0939 b ± 3.70%  3.76%  0.0882 b ± 4.51%  4.55% 
Uncertainty summary for exercise I1 TMI1 and GenIII MOX cases.
Parameter  TMI1 HZP  TMI1 HFP  GenIII MOX 

Eigenvalue  1.4293 ± 0.50%  1.4099 ± 0.51%  1.1076 ± 0.95% 
U235 abs.  43.6 b ± 1.05%  42.4 b ± 1.06%  15.2 b ± 1.37% 
U235 fis.  35.3 b ± 1.05%  34.3 b ± 1.07%  11.0 b ± 1.19% 
U238 abs.  0.911 b ± 1.10%  0.934 b ± 1.11%  0.893 b ± 1.14% 
U238 fis.  0.101 b ± 3.59%  0.101 b ± 3.62%  0.118 b ± 3.69% 
Pu239 abs.  27.3 b ± 1.23%  
Pu239 fis.  17.6 b ± 1.30%  
Pu240 abs.  21.7 b ± 1.33%  
Pu240 fis.  0.639 b ± 2.10%  
Pu241 abs.  31.6 b ± 1.28%  
Pu241 fis.  23.7 b ± 1.30%  
Pu242 abs.  11.9 b ± 5.03%  
Pu242 fis.  0.492 b ± 4.85%  
Am241 abs.  32.8 b ± 4.41%  
Am241 fis.  0.753 b ± 2.62% 
To assess the effect of the perturbation group structure, two additional group structures were investigated as shown in Table
Effect of changing the perturbation group structure for the PB2 HZP cell case.
Parameter  Perturbation group structure  

19group  31group  44group  
Eigenvalue  0.55%  0.55%  0.54% 
U235 abs.  0.99%  0.99%  0.92% 
U235 fis.  1.01%  1.02%  0.94% 
U238 abs.  1.08%  1.07%  1.03% 
U238 fis.  3.70%  3.67%  3.73% 
The most influential parameters are easily defined by sorting from greatest to least variance fraction, and the cumulative value can be used to limit the important parameters, for example, the set representing 99% of the total variance, as shown in Figure
Breakdown of eigenvalue uncertainty as function of perturbation group structure for the PB2 HZP cell case.
Breakdown of 1group U238 absorption cross section uncertainty (a) and U235 fission cross section uncertainty (b) in terms of variance fractions for the PB2 HZP cell case.
SCALE6 nuclear data uncertainty for U235 fission spectrum,
The lattice physics cases in exercise I2 are concerned with propagating both nuclear data uncertainty and the socalled “technological parameter” uncertainty to the twogroup nodal data used in conventional core simulators based on twogroup nodal diffusion. The output parameters of interest here are mainly the homogenized macroscopic cross sections for fast and thermal absorption (
Uncertainty summary for the exercise I2 TMI1 HFP case, assuming only nuclear data uncertainty.
Parameter  Unrodded  Rodded  

DP  SS  DP  SS  
Eigenvalue  1.3997 ± 0.50%  0.50%  1.0284 ± 0.53%  0.53% 

0.01 ± 0.87%  0.91%  0.0133 ± 0.94%  0.99% 

0.108 ± 0.21%  0.22%  0.136 ± 0.18%  0.18% 

0.00861 ± 0.50%  0.51%  0.00851 ± 0.49%  0.49% 

0.186 ± 0.44%  0.45%  0.190 ± 0.44%  0.45% 

0.0158 ± 1.03%  1.07%  0.0137 ± 1.18%  1.20% 

1.43 ± 0.83%  0.86%  1.39 ± 0.88%  0.89% 

0.372 ± 0.01%  0.02%  0.376 ± 0.02%  0.02% 
Unr. peak loc.  1.09 ± 0.03%  0.04%  0.802 ± 0.06%  0.11% 
Gd pin loc.  0.405 ± 0.56%  0.51%  0.506 ± 0.63%  0.50% 
Rod. peak loc.  0.951 ± 0.04%  0.05%  1.26 ± 0.14%  0.12% 
ADF^{1 }  0.975 ± 0.04%  0.04%  1.020 ± 0.05%  0.05% 
ADF^{2}  1.070 ± 0.03%  0.03%  1.470 ± 0.06%  0.06% 
PWR assembly locations (southeast quarter shown).
At the time of this publication, the probability distributions of the technological parameters were not generally agreed upon, and so only a sensitivity analysis has been performed using CASMO5MX/DP which can easily compute sensitivity coefficients of any input file parameter. As in the benchmark specification, five technological parameters were considered: fuel density (
Sensitivity summary with respect to technological parameters for exercise I2 TMI1 HFP (values > 0.5 shown in bold).
Parameter  Unrodded  Rodded  












eigenvalue  0.13  −0.05  0.00  −0.03  −0.29  0.26  0.11  −0.01  −0.05  −0.10 

0.27 

0.00  −0.01 

0.16 

0.00  −0.01 




0.00  −0.02 



0.00  −0.02 




0.00  −0.03 



0.00  −0.03 




0.00  −0.02 



0.00  −0.03 


−0.17  −0.16  −0.03  −0.19 

−0.20  −0.18  −0.03  −0.22 


0.02 

0.01  −0.03  −0.35  0.02  −0.42  0.01  −0.04  −0.36 

−0.02  −0.15  0.02  0.10 

−0.02  −0.17  0.02  0.10 

Unr. Peak Loc.  0.00  0.02  0.00  0.01  0.13  0.00  0.00  0.00  0.00  0.26 
Gd Pin Loc.  0.33  0.23  0.01  0.07 

0.25  0.10  0.01  0.05 

Rod. Peak Loc.  0.07  0.07  0.00  0.00  0.15  0.00  0.00  0.00  0.00  0.00 
Results obtained with MCNPX/NUSS for eigenvalue uncertainty in the PB2 and TMI1 cell models at HZP are shown in Table
Uncertainty summary using MCNPX/NUSS for HZP cases.
Parameter  PB2 HZP  TMI1 HZP 

Eigenvalue  1.3443 ± 0.54%  1.4305 ± 0.49% 
Although the number of samples was fairly small at 80, a study of the running average eigenvalue and uncertainty (onesigma error bars) in Figure
Cumulative moving average of eigenvalue with MCNPX/NUSS versus number of samples for exercise I1 PB2 HZP (a) and TMI1 HZP cases (b).
In this section, various results from the previous section will be further discussed, namely,
BWR uncertainties predicted by both the CASMO5MX/DP and SS methodologies,
BWR versus PWR uncertainties,
UO2 versus MOX uncertainties,
CASMO5MX versus MCNPX/NUSS results.
Consistent trends are observed with both methodologies for the exercise I1 PB2 (BWR) case, with slightly higher uncertainties observed at HFP, both in eigenvalue (denoted “Kinf”) and 1group cross sections, especially U238 fission. This is due to spectrum hardening in the HFP case, with nearly 40% void, which acts to increase uncertainty because data in the fast range is generally more uncertain. For the 1group cross sections, a faster spectrum also increases the impact of U238 inelastic scattering, which contributes greatly to the overall uncertainty [
Comparison of CASMO5MX/DP and/SS methods.
At HZP conditions, almost identical uncertainties are observed for the exercise I1 PB2 (BWR) and TMI1 (PWR) cases (see Figure
Comparison of PWR and BWR uncertainties with CASMO5MX/SS.
For MOX fuel from the exercise I1 GenIII MOX case, nearly double the uncertainty (0.95%) in eigenvalue is observed compared to UOX fuel (0.51%). See the graphical summary in Figure
Comparison of MOX and UO2 fuel uncertainty.
The MCNPX results showed a total uncertainty in eigenvalue of 0.54% using MCNPX/NUSS which was very consistent with both the CASMO5MX/SS and CASMO5MX/DP results
Comparison of top 5 contributors for MCNPX/NUSS versus CASMO5MX/DP for exercise I1 PB2 HZP.
Nuclide/reaction  MCNPX/NUSS  CASMO5MX/DP 

U238/102  0.32%  0.37% 
U235/452  0.30%  0.27% 
U235/102  0.17%  0.20% 
U238/4  0.12%  0.12% 
U235/18  0.10%  0.12% 
 
Total  0.54% (0.50% in top 5)  0.54% (0.53% in top 5) 
The UAM benchmark has provided the opportunity to develop stateoftheart methodologies for uncertainty quantification (UQ) and the framework for international collaboration and comparison. At PSI, within the STARS project, the first development was CASMO5MX, a modification of the production CASMO5 M code to perturb nuclear data libraries through an auxiliary input file with the capabilities to provide perturbations in any group structure and perturb individually the inelastic
Results for the UAM benchmark exercises were presented, including the LWR cell cases from exercise I1 and the PWR assembly case from exercise I2. For the cell cases, uncertainty in the eigenvalue and 1group collapsed microscopic cross sections (in terms of relative standard deviation) was found to be about 0.5% and 1%, respectively. For the GenIII MOX case, the eigenvalue uncertainty was nearly double (1%) and Pu242 and Am241 1group cross sections uncertainties’ reached 5%. In
Finally, sensitivity coefficients were calculated for technological parameters for the exercise I2 TMI1 PWR assembly and it was found that the radius of the fuel pellet is the most sensitive parameter, having sensitivity coefficients of absolute value from 1 to 2 for many outputs. For example, the sensitivity coefficient of the removal cross section (
Future work in the area of neutronics UQ at PSI includes enhancement of the MCNPX/NUSS continuousenergy Monte Carlo strategy, implementing the capability to perturb fission product yields and decay constants, and extension of the SS methodology from the lattice code CASMO5 M to the core simulator SIMULATE3.