Computational and Mathematical Methods in Medicine
Volume 2017 (2017), Article ID 7025281, 12 pages
https://doi.org/10.1155/2017/7025281

Research Article

Sensitivity Analysis of the Integral Quality Monitoring System® Using Monte Carlo Simulation

Oluwaseyi M. Oderinde and F. C. P. du Plessis

Department of Medical Physics, University of the Free State, P.O. Box 339, Bloemfontein 9300, South Africa

Correspondence should be addressed to F. C. P. du Plessis; duplessisfcp@ufs.ac.za

Received 17 May 2017; Revised 6 July 2017; Accepted 20 July 2017; Published 27 August 2017

Academic Editor: Michele Migliore

Copyright © 2017 Oluwaseyi M. Oderinde and F. C. P. du Plessis. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The Integral Quality Monitoring (IQM) System is a real-time beam output verifying system that validates the integrity and accuracy of patient treatment plan (TP) data during radiation treatment. The purpose of this study was to evaluate the sensitivity of the IQM to errors in segment using EGSnrc/BEAMnrc Monte Carlo (MC) codes. Sensitivity analysis (SA) techniques were applied to study the significance of small alterations of field sizes (segments) on the IQM signal response. One hundred and eighty multileaf segments were analyzed with methods that include scatter plots (SP), brute force, variance-based (VAR), and standard regression coefficient SA. The segments were altered randomly within ±1, ±2, and ±3 mm leaf steps for 10 MV photon beams. SP analysis gradient and VAR maximum index are 1.045 and 0.556 for the smallest segment while the largest segment has the value of 0.018 and 0.504, respectively. The brute force and standard regression displayed maximum sensitivity indices around the unaltered segments. These tests conclusively indicated that the IQM was more sensitive to alterations of small segments compared to larger segments. This is important since small segment variation will cause a higher dose output variation that should be picked up during online beam monitoring.

1. Introduction

The goal of radiotherapy is to improve the quality of treatment: minimizing the normal tissue exposure and maximizing the therapeutic ratio. In the quest for an optimum treatment, Paliwal introduced the concept of an online beam delivery check for noncomputerized Linacs [1]. It consisted of a transmission chamber to detect possible errors in treatment delivery and to serve as a pretreatment quality assurance (QA) tool. This concept adds an additional record-and-verify system to the Linac head. Since then, vendors have developed and suggested some dose monitors for offline/online dose verification of external photon beam radiotherapy. The suggested online monitors are the electronic portal imaging device (EPID), DAVID™, and the Dolphin® system [2–4].

The Integral Quality Monitoring (IQM) System is a prototype online dose verification system that was released by iRT Systems, Germany. The IQM system is a double wedge-shaped ionization chamber that is attached below the Linac treatment head. It is a dose measuring system that validates the TP data in real time [5]. The IQM is capable of monitoring a 40 × 40 cm² field defined at the isocenter. The double wedge shapes of the IQM device are defined by the outer polarizing electrodes which are kept at a potential of 500 volts. The inner electrode plate is grounded to zero volts and is designated as a collecting electrode (Figure 1). The electrodes are made of 1.5 mm aluminum. The output signal is a function of the photon beam fluence that irradiates the double-wedged chamber.

Figure 1: IQM double wedge ionization chamber model (permission granted) [27].

The major interest of utilizing the prototype IQM device is its ability to function as a beam delivery check system during real-time treatment [6, 7].

Sensitivity analysis is the process of undertaking a systematic review of models; it studies the significance of each of the model inputs on the model output [8, 9]. It identifies and determines the impact of inputs on its outputs [10–15]. This builds a theory that relates the input variables with the output variables. There are several sensitivity analysis techniques stated in various articles such as scatter plots, one-at-a-time, partial derivatives, brute force, partial correlation coefficient, standard regression coefficient, variance-based SA, Sobol sensitivity indices, fast first-order index, Spearman rank correlation coefficient, Fourier amplitude sensitivity test, and Morris one-at-a-time screening [10, 16–23]. The sensitivity ranking of the input for several techniques may vary slightly, but the focus is based on the consistent parameter that influences the output [13, 19, 24]. The choice of method of sensitivity analysis is guided by the problem constraints. The constraint can be correlated inputs, nonlinearity, multiple outputs, given data, or random variables (e.g., simple random sampling, Monte Carlo, Latin Hypercube, Morris method, and quasi-random sequence). Random sampling, which is the focus of this research, requires the application of scatter plots, brute force, variance-based, and standard regression coefficient sensitivity analysis techniques which are efficiently independent of one another [14, 16, 18, 21, 25, 26]. These four sensitivity analysis techniques were used for an in-depth study of the consistent input parameters that influenced the output value.

The aim of this study was to investigate the sensitivity of the double wedge-shaped ionization chamber (IQM) to errors in segment using the BEAMnrc/EGSnrc Monte Carlo (MC) simulation. The input value in this case is the accelerator beam segment and the output is the corresponding dose scored in the double wedge ionization chamber of the IQM system.

2. Materials and Methods

2.1. Simulation Setup

BEAMnrc MC was used to simulate an accurate source model of an Elekta Synergy linear accelerator equipped with an Agility 160-Leaf multileaf collimator (MLC) alongside the IQM. The IQM model was located 4.5 cm below the lowest diaphragm of the Linac model (Figure 2). The gradient of the IQM model is positioned perpendicularly to the MLC movement. During each simulation, the spatial integral dose was scored in the air region of the wedge-shaped ionization chamber of the IQM model.

Figure 2: Setup of the IQM geometry located 4.5 cm below the -diaphragm for 10 × 10 cm² field.

To start the sensitivity study, regular fields of 3 × 3, 5 × 5, and 7 × 7 cm² were simulated at 10 MV photon beams and moved along the gradient of the IQM model. Figure 3 shows how a single segment was moved along the slope of the IQM. Dose responses were recorded along the gradient at positions of interest. The MLC defined field size remains unchanged along the IQM gradient, but the spot of the radiation beams on the IQM model was changed at every interval.

Figure 3: Movement of segment along the gradient of the IQM model.

The sensitivity of the IQM was also studied by simulating six segments (regular and irregular) which were randomly altered within ±1, ±2, and ±3 mm of their original leaf positions to simulate leaf positional errors. The leaf positions were defined at the isocenter (100 cm SSD) and the beam energy studied was 10 MV. The regular and irregular segments (shown in Table 1) were chosen to include a wide range of MLC shaped aperture conditions. Each of the open leaves of the segments was randomly altered. The ±1 mm alteration means that any randomly generated value between −1 mm and +1 mm can be chosen to simulate a shift from the original leaf position. Note that there is a probability of having a zero value from the random generator, which means that the particular leaf will not be altered. For a segment, each positional error (±1, ±2, and ±3 mm) was altered 10 times to have a total of 30 altered models per segment. This procedure was repeated for each of the segments used in this study (Table 1). MC simulations were done for 180 altered and 6 unaltered segments, and the corresponding IQM signal responses were collected from each simulation for analysis.

Table 1: List of segments altered randomly within ±1, ±2, and ±3 mm.

Segments
Segment area (cm²)

Segment 1 (3 × 3 cm²)
9.00

Segment 2 (irregular)
25.83

Segment 3 (irregular)
70.82

Segment 4 (irregular)
47.49

Segment 5 (1 × 1 cm²)
1.00

Segment 6 (irregular)
19.99

The number of histories was large enough to reduce uncertainties in the scored IQM dose to less than 1%. The following simulation parameters were set: the global electron cut-off energy (ECUT) was set to 0.7 MeV and the global photon cut-off energy (PCUT) was set to 0.01 MeV for efficiency [28]. The maximum step size (SMAX) was set to default because EXACT was used in the boundary algorithm and PRESTA II was used in the electron-step algorithm. The maximum fraction energy loss/step (ESTEPE) was set to 0.25 (25%) and the maximum first elastic scattering moment per step (Xlmax) was set to 0.5. The skin depth for the boundary crossing algorithm (skindepth_for_bca) was set to 0 cm. Spin effects for electron elastic scattering (spin_effects) were turned on for appropriate backscattering simulation and electron impact ionization (eii_flag) was turned off. Bremsstrahlung angular sampling (IBRDST) was set to the default method and its cross sections were derived from the Bethe-Heitler method. Bound Compton scattering (IBCMP) was turned on and its cross section data were obtained from its default method. Pair angular sampling (IPRDST) was set to the Bethe-Heitler method (default). Photoelectron angular sampling (IPHTER) and Rayleigh scattering (IRAYLR) were turned off, and atomic relaxations (IEDGFL) were turned on. The photon cross-section data bases (photon_xsections) were set to the PEG4 dataset. All EGSnrc simulation parameters were conventionally set for a better reflection of realistic radiation transport [29].

2.2. Scatter Plots Sensitivity Analysis

Scatter plots (SP) can be used for qualitative analysis to determine the most sensitive parameter if more than one input variable is used that can alter the output result. It is achieved by plotting the graph of the input values against the corresponding output values. The linearity of the scatter plots determines the sensitivity of the model. An increase in the gradient of the linear equation indicates that the model is more sensitive to the input parameter under consideration [30, 31]. SP can be utilized to determine the IQM sensitivity by correlating the effect of segment area (SA) input on its output signal ().

2.3. Brute Force Sensitivity Analysis

The brute force sensitivity analysis method is based on noninteractive input variables whereby the input variable is altered to study its effect on the output value. It generates the sensitivity of a model due to perturbation of the input variables [32]. Quantification of the input variables is necessary. As mentioned above, this study quantifies SA as the input parameter. A sensitivity index (SI) was calculated based on the change in () over the corresponding change in SA (ΔSA) [33–35]:

where is the signal for the unaltered segment and is the signal after segment alteration (). Large SI indicates that the change in SA has a significant impact on .

2.4. Variance-Based Sensitivity Analysis

The variance-based sensitivity analysis (VAR) method is a global sensitivity analysis; it is based on the idea that one can deduce the sensitivity of a model through its variance . It is a probability distribution of the output uncertainties (in this case, the output uncertainty is the uncertainty in ). It focuses on interactions between the input variables and the effect of each of the input variables on the output value. It is a measure of the importance of input variables on the outputs [22, 36, 37]. In our application, the sensitivity of the output to the input variable is therefore measured by the amount of variance in caused by alterations in SA:

In (2), represents and is the change in segment size generated at random for a given segment. It should be noted that only SA plays a role in the variation of and thus index = 1 in (2). If some other input variable also played a role, then index = 1, 2.

The sensitivity measure of conditional variance for a single input variable is given by

The sensitivity measure of sensitivity index is given by

where is the unconditional variance.

is the main effect index (first-order sensitivity index or correlation ratio) and it describes the main effect of the given data on the value [17, 31, 38].

In this study, the effects of other input variables on the IQM output value are not present. Therefore, an extension for the total effect index is not applicable since the focus is on only one input variable.

2.5. Standard Regression Coefficient Sensitivity Analysis

This is a regression analysis that tests the significance of an independent variable (input) on its dependent variable (output) by using both the mean and the variance of the independent and dependent variables across the observable model. It is generally used for linear regression [22, 23].

The generalized linear regression model is

The normalized regression model for an input variable is

The standard regression coefficient (SRC) and is the regression coefficient for the th sample of the input (segment area), where

are the standard deviation of output signal () of the IQM and segment area (), respectively. The sensitivity of the model is determined by the SRC value. The higher the SRC value, the more sensitive the variation in the input value on the variation of the output value.

3. Results and Discussion

Figure 4 shows the sensitivity gradient profiles for 3 × 3, 5 × 5, and 7 × 7 cm² fields at 10 MV photon beams. This is to compare the gradient response between different segments. The sensitivity profile increases along the gradient of the IQM model. There is a noticeable plateau region in the sensitivity profile. This is the region of the higher separation distance between the collecting plate and the polarizing plate. It should also be noted that the photon beam of a field that is incident on the IQM model is not the same along the gradient of the IQM. The incident aperture changes along the gradient. The combined effect of nonuniform incident aperture and the separation distance between the plates gives the sensitivity profile. In Figure 4, the gradient of the trend line increases with an increase in field sizes.

Figure 4: Sensitivity factor along the IQM CM gradient for 3 × 3, 5 × 5, and 7 × 7 cm² fields (trend gradients are 0.005, 0.009, and 0.010).

In Figure 4, the largest air volume region has the highest signal response. Around the region of the largest air volume of the IQM chamber, there is a noticeable plateau. This plateau is caused by loss in lateral electron equilibrium (Farrokhkish’s presentation also stated this idea at the World Medical Physics Conference (IUPESM 2015) [27]). Loss of lateral equilibrium could be a result of the photon beam being unable to completely cover the air volume of the IQM chamber at this region [39].

Figures 5–10 depict five segments that were altered randomly within the limits stated above and their sensitivity analysis results.

(a)

(b)

(c)

(d)

(e)

Figure 5: Sensitivity analysis results for the 3 × 3 cm² segment. Panel (a) shows the unaltered segment; panel (b) shows the scatter plot; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

(a)

(b)

(c)

(d)

(e)

Figure 6: Sensitivity analysis results for an irregular segment (SA = 25.83 cm²). Panel (a) shows the unaltered segment; panel (b) shows the scatter plots; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

(a)

(b)

(c)

(d)

(e)

Figure 7: The same data as in Figure 6, but for the irregular segment with an area of 70.82 cm². Panel (a) shows the unaltered segment; panel (b) shows the scatter plots; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

(a)

(b)

(c)

(d)

(e)

Figure 8: The same data as in Figure 4, but for the irregular segment with SA of 47.49 cm². Panel (a) shows the unaltered segment; panel (b) shows the scatter plots; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

(a)

(b)

(c)

(d)

(e)

Figure 9: The same data as in Figure 4, but for the irregular segment with an area of 1.0 cm². Panel (a) shows the unaltered segment; panel (b) shows the scatter plots; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

(a)

(b)

(c)

(d)

(e)

Figure 10: Sensitivity analysis results for an irregular segment (SA = 19.99 cm²). Panel (a) shows the unaltered segment; panel (b) shows the scatter plots; panels (c), (d), and (e) show the brute force, variance-based, and standard regression coefficient results, respectively.

In Figure 5(a), a regular 3 × 3 cm² field is presented. In Figure 5(b), SP data are shown; for each maximum alteration level such as ±1 mm, there are 10 data points which correspond to the 10 trials that were measured as outlined above. Data for the ±2 mm and ±3 mm cases are also plotted in Figure 5(b). There exists a correlation between and SA, indicating a definite response to the IQM signal when the segment area is altered. The SA range depends on the allowed segment alteration limit. For the ±3 mm randomly altered case, SA ranged between 8.38 cm² and 9.36 cm². Originally, SA was 9.00 cm². For the ±2 mm case, this range shrank to between 8.66 and 9.29 cm². For the ±1 mm case, there is an even spread around the unaltered SA. The linear trend line of SP has a gradient of 0.13 cm⁻². In Figure 5(c), the brute force analysis tool shows the rate of change of with respect to SA. Thirty SI values are shown indicating 30 trials. High SI values are noted around the unaltered SA. In Figure 5(d), SI values were analyzed for 30 trials using the VAR method. SI increased with increasing SA values. In Figure 5(e), the 30 normalized SRC values are depicted. High SRC values are seen around the unaltered SA.

The irregular segment in Figure 6(a) was altered within ±1, ±2, and ±3 mm. SP (Figure 6(b)) have a trend-line gradient of 0.048 cm⁻² for the SA alterations that span the range of 25.10–26.46 cm². In Figures 6(c) and 6(e), the brute force analysis and SRC values are displayed. The two methods show that the most sensitive altered segment is around the unaltered segment (SA = 25.80 cm²). In Figure 6(d), the VAR indices correspond to SA values for the 30 trials.

Figure 7(a) shows an irregular segment (SA = 70.86 cm²), the largest SA considered in this study. In Figure 7(b), IQM shows a sensitivity to SA indicated with a gradient of 0.018 cm⁻². In Figure 7(c), the brute force sensitivity indices show that higher sensitivity indices are noted around the unaltered segment. One of the alterations within ±1 mm has the highest sensitivity index of 0.52. In Figure 7(d), the variance of the IQM signal output determines the sensitivity of the IQM model by correlating the VAR sensitivity indices with 30 trials. An increase in SA causes an increase in VAR sensitivity index. In Figure 7(e), SRC were plotted against SA. The highest SRC value was observed in one of the alterations within ±2 mm. This is the most sensitive altered segment out of the 30 trials.

In Figure 8(a), the model of an irregular segment of 47.49 cm² is presented. In Figure 8(b), SP values for 30 trials are shown. An increase in SA causes a gradual increase in . The values are narrowly scattered around the trend line with a gradient of 0.030 cm⁻². In Figure 8(c), the brute force sensitivity indices are displayed for the trials considered. It is observed that the most sensitive indices are for small alterations of the original SA. In Figure 8(d), the VAR SI correlate with SA; an increase in SA causes an increase in SI. In Figure 8(e), the normalized SRC values were plotted against the 30 SAs considered. The maximum SRC value is observed around the region of the original SA.

In Figure 9(a), a regular segment of SA = 1 × 1 cm² is shown. This is the smallest SA considered in this study. In Figure 9(b), the data points align with the linear trend line with a gradient of 1.045 cm⁻². In Figures 9(c) and 9(e), the brute force analyses and SRC values show high sensitivities for all the trials considered with the highest SI around the original segment area. In Figure 7(d), the VAR SI shows a linear progression with an increase in segment areas of the 30 trials.

Figure 10(a) is an irregular segment that was altered within ±1, ±2, and ±3 mm. SP (Figure 10(b)) have a trend-line gradient of 0.06 cm⁻² for the SA alterations that span the range of 19.21–20.79 cm². In Figures 10(c) and 10(e), the brute force analysis and SRC values are displayed. The two methods show that the most sensitive altered segment is around the unaltered segment (SA = 19.99 cm²). In Figure 10(d), the VAR indices correspond to SA values for the 30 trials.

The highest degree of linearity in SP is found in the smallest SA of 1 × 1 cm² (Figure 9(b)). For this SA, the SP values were almost on the trend line and it has the smallest gradient of 0.018 cm⁻² (Figure 7(b)).

In Figure 11, the gradient of scatter plots (SP) and variance-based (VAR) sensitivity analysis in Figures 5–10 were plotted. A power function was fitted to the SP trend-line gradient data as a function of original SA (). It shows that an increase in SA causes a decrease in the SP trend-line gradient that is at first strongly dependent on SA but becomes less sensitive for SA ≥ 26 cm². This indicated that the IQM detects small shifts in the smallest segment with the largest differential signal which is to be expected since the allowed random shifts between ±1, ±2, and ±3 mm of the MLC leaves would make up a larger percentage change in SA compared to the largest segment; larger segments will cause smaller differential signals. SP data are a quick means of investigating the SI of data but do not compare the rate of change in IQM signals to the rate of change in SA like the brute force sensitivity analysis.

Figure 11: Analysis for scatter plots and variance-based sensitivity analysis for Segments 1–6.

In Figure 11, the SP and VAR sensitivity analysis methods show power function relationships with respect to SA although the VAR is virtually constant over SA.

VAR indices for the five segments considered decrease weakly with an increase in SA and can be considered for practical purposes to be constant with an average value of 0.525. The fitted power function

has the derivative of

From (9), it is seen that the rate of change in VAR with change in SA is approximately inversely proportional to SA for the segment sizes studied. It has a maximum at the smallest segment and decreases as SA increases starting off at an initial value of 0.013.

Brute force sensitivity analysis for the segments in Figures 5(c), 6(c), 7(c), 8(c), and 10(c) shows that the highest SI is within the region of the unaltered (original) segments, while in the small field (Figure 9(c)) it is high for all trials considered. This again indicates that the IQM signal is sensitive to alterations of small segments and less sensitive to alterations of large ones. The brute force technique means that if a segment is altered, the magnitude of alteration (perturbation) which is the difference between the unaltered and altered SA must be evaluated based on the difference in the magnitude of . If the minimal difference in SA gives a significant difference in , then the model is very sensitive to such input parameter. This technique, unlike scatter plots, considers the rate of change of the altered parameters. If the SP is highly linear (Figure 9(b)), the brute force SI (Figure 9(b)) will display a noticeable deviation from zero across all altered segments.

SRC values for the five segments displayed maximum (normalized) values around the region of the unaltered segments. For the smallest SA (Figure 9(e)), higher SRC values were calculated across all 30 altered SA trials. SP data give a reasonable indication of this outcome. For a perfectly linear graph, the rate of change of the output value per the input variable will be uniform, which makes the SRC values seem uniform across the altered segments around original SA = 1 × 1 cm². This means that the IQM is most sensitive to alterations of small SA.

In total, 6 original segments were each altered 30 times randomly within ±1, ±2, and ±3 mm positional errors. For each segment, a Monte Carlo simulation was done to determine the IQM signal. Sensitivity analysis results indicated that the IQM is sensitive to detect these alterations in SA. The sensitivity is more pronounced in small SA.

4. Conclusion

The sensitivity of the IQM in this study shows its potential to detect small alterations in SA. The SP and VAR relation to SA is approximately constant at large SA but displays power function relationships at smaller SA values. All sensitivity analysis methods employed in this study indicated that the IQM signal () will indicate small segments alterations even for the larger segments.

Conflicts of Interest

Both authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research project was funded by the South Africa Medical Research Council (MRC) with funds from the National Treasury under its Economic Competitive and Support Package. This research and the publication thereof is the result of funding provided by the SAMRC under the High Energy Advanced Radiation (HARD) Sponsorship Program SAMRC-RFA-UFSP-01-2013/HARD.

References

B. R. Paliwal, M. Zaini, T. McNutt, E. J. Fairbanks, and R. Kitchen, “A consistency monitor for radiation therapy treatments,” Medical Physics, vol. 23, no. 10, pp. 1805–1808, 1996.

M. G. Herman, J. M. Balter, D. A. Jaffray et al., “Clinical use of electronic portal imaging: Report of AAPM Radiation Therapy Committee Task Group 58,” Medical Physics, vol. 28, no. 5, pp. 712–737, 2001.

B. Poppe, C. Thieke, D. Beyer et al., “DAVID - A translucent multi-wire transmission ionization chamber for in vivo verification of IMRT and conformal irradiation techniques,” Physics in Medicine and Biology, vol. 51, no. 5, pp. 1237–1248, 2006.

I. Al Amri, R. Ravichandran, S. S. Sivakumar et al., “Radiotherapy pre-treatment dose validation: a second verification of monitor units (MU) with a commercial software,” Journal of Medical Physics, vol. 37, no. 4, pp. 235–239, 2012.

iRT Systems, “Integral Quality Monitor Improving patient safety,” iRT Systems White Paper Presentation, 2014.

M. K. Islam, B. D. Norrlinger, J. R. Smale et al., “An integral quality monitoring system for real-time verification of intensity modulated radiation therapy,” Medical Physics, vol. 36, no. 12, pp. 5420–5428, 2009.

J. Chang, R. K. Heaton, R. Mahon et al., “A method for online verification of adapted fields using an independent dose monitor,” Medical Physics, vol. 40, no. 7, article 072104, 2013.

A. Saltelli, M. Ratto, T. Andres et al., Global Sensitivity Analysis: The Primer, John Wiley & Sons, New York, NY, USA, 2008.

I. M. Sobol, “Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates,” Mathematics and Computers in Simulation, vol. 55, no. 1-3, pp. 271–280, 2001.

R. Confalonieri, G. Bellocchi, S. Bregaglio, M. Donatelli, and M. Acutis, “Comparison of sensitivity analysis techniques: a case study with the rice model WARM,” Ecological Modelling, vol. 221, no. 16, pp. 1897–1906, 2010.

C. Tiede, Integration of Optimzation Algorithms with Sensitivity Analysis with Application to Volcanic Regions, Daramstadt University of Technology, 2005.

A. Saltelli, P. Annoni, I. Azzini, F. Campolongo, M. Ratto, and S. Tarantola, “Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index,” Computer Physics Communications, vol. 181, no. 2, pp. 259–270, 2010.

C. Bucher, “Computational Analysis of Randomness in Structural Mechanics,” in Structures and Infrastructures Book Series, vol. 3, CRC Press, 2009.

T. Homma and A. Saltelli, “Importance measures in global sensitivity analysis of nonlinear models,” Reliability Engineering and System Safety, vol. 52, no. 1, pp. 1–17, 1996.

T. Ishigami and T. Homma, “An importance quantification technique in uncertainty analysis for computer models,” in Proceedings of the First International Symposium on Uncertainty Modeling and Analysis, pp. 398–403, IEEE Computer Society Press, 1990.

A. Saltelli and P. Annoni, “How to avoid a perfunctory sensitivity analysis,” Environmental Modelling and Software, vol. 25, no. 12, pp. 1508–1517, 2010.

J. Zhao and C. Tiede, “Using a variance-based sensitivity analysis for analyzing the relation between measurements and unknown parameters of a physical model,” Nonlinear Processes in Geophysics, vol. 18, no. 3, pp. 269–276, 2011.

W. Becker, J. Rowson, J. E. Oakley, A. Yoxall, G. Manson, and K. Worden, “Bayesian sensitivity analysis of a model of the aortic valve,” Journal of Biomechanics, vol. 44, no. 8, pp. 1499–1506, 2011.

D. M. Hamby, “A review of techniques for parameter sensitivity analysis of environmental models,” Environmental Monitoring and Assessment, vol. 32, no. 2, pp. 135–154, 1994.

V. Schwieger, “Variance-based sensitivity analysis for model evaluation in engineering survey,” in Proceedings of the INGEO and FIG Regional central and eastern european conference on engineering survey, pp. 11–13, Bratislava, Slovakia, 2004.

D. Liu and Z. Zou, “Sensitivity analysis of parameters in water quality models and water environment management,” Journal of Environmental Protection, vol. 3, no. 8, pp. 863–870, 2012.

Y. Gan, Q. Duan, W. Gong et al., “A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model,” Environmental Modelling & Software, vol. 51, pp. 269–285, 2014.

B. Iooss and P. Lemaître, “A review on global sensitivity analysis methods,” in Uncertainty Management in Simulation-Optimization of Complex Systems, pp. 101–122, Springer, 2015.

J. C. Helton, J. D. Johnson, C. J. Sallaberry, and C. B. Storlie, “Survey of sampling-based methods for uncertainty and sensitivity analysis,” Reliability Engineering System Safety, vol. 91, no. 10-11, pp. 1175–1209, 2006.

A. Saltelli and R. Bolado, “An alternative way to compute Fourier amplitude sensitivity test (FAST),” Computational Statistics and Data Analysis, vol. 26, no. 4, pp. 445–460, 1998.

A. Saltelli and S. Tarantola, “On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal,” Journal of the American Statistical Association, vol. 97, no. 459, pp. 702–709, 2002.

M. Farrokhkish, “Response Characteristics of a Large-Area Ion Chamber with Various Radiotherapy Beams,” in Proceedings of the World Congress on Medical Physics and Bioengineering, 2015.

I. J. Chetty, B. Curran, J. E. Cygler et al., “Report of the AAPM Task Group No. 105: Issues associated with clinical implementation of Monte Carlo-based photon and electron external beam treatment planning,” Medical Physics, vol. 34, no. 12, pp. 4818–4853, 2007.

D. W. O. Rogers, B. Walters, and I. Kawrakow, “BEAMnrc users manual,” Source, vol. 509, pp. 1–260, 2011.

T. Most, “Variance-based sensitivity analysis in the presence of correlatio input variables,” 2011, Lecture note, https://www.dynardo.de/en/library.

A. Saltelli, S. Tarantola, F. Campolongo, and M. Ratto, Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models, John Wiley & Sons, 2004.

X. Zhou and H. Lin, Local Sensitivity Analysis, Springer, 2008.

D. Hwang, D. W. Byun, and M. T. Odman, “An automatic differentiation technique for sensitivity analysis of numerical advection schemes in air quality models,” Atmospheric Environment, vol. 31, no. 6, pp. 879–888, 1997.

A. Hakami, M. T. Odman, and A. G. Russell, “High-order, direct sensitivity analysis of multidimensional air quality models,” Environmental Science and Technology, vol. 37, no. 11, pp. 2442–2452, 2003.

D. S. Cohan, A. Hakami, Y. Hu, and A. G. Russell, “Nonlinear response of ozone to emissions: Source apportionment and sensitivity analysis,” Environmental Science and Technology, vol. 39, no. 17, pp. 6739–6748, 2005.

K. Chan, A. Saltelli, and S. Tarantola, “Sensitivity analysis of model output: Variance-based methods make the difference,” in Proceedings of the 1997 Winter Simulation Conference, pp. 261–268, December 1997.

F. Kamp, S. Brüningk, G. Cabal, A. Mairani, K. Parodi, and J. J. Wilkens, “Variance-based sensitivity analysis of biological uncertainties in carbon ion therapy,” Physica Medica, vol. 30, no. 5, pp. 583–587, 2014.

E. Plischke, “How to compute variance-based sensitivity indicators with your spreadsheet software,” Environmental Modelling and Software, vol. 35, pp. 188–191, 2012.

X. A. Li, M. Soubra, J. Szanto, and L. H. Gerig, “Lateral electron equilibrium and electron contamination in measurements of head-scatter factors using miniphantoms and brass caps,” Medical Physics, vol. 22, no. 7, pp. 1167–1170, 1995.