Sensitivity Analysis

Resources

Sudret, Bruno. 2008. “Global Sensitivity Analysis Using Polynomial Chaos Expansions.” Reliability Engineering & System Safety 93 (7): 964–79. https://doi.org/10.1016/j.ress.2007.04.002.
ASEN 6412 Class Notes - slides_08.pdf (which is quite similar to the above)

Main Idea

What uncertain inputs have the largest impact on some output?

Sobol' Indices

Let $X \sim U ([0, 1])^{d}$ , $f : R^{d} \to R$ (deterministically), and $Y = f (X)$ . Under some conditions (?), we can decompose $f$ as

f (X) = f_{0} + \sum_{i = 1}^{d} \underset{univariate}{\underset{⏟}{f_{i} (X_{i})}} + \sum_{i = 1}^{d} \sum_{j = i + 1}^{d} \underset{bivariate}{\underset{⏟}{f_{i j} (X_{i}, X_{j})}} + \dots + f_{12 \dots d} (X_{1}, X_{2}, \dots, X_{d}) .

Note, the bivariate terms are Upper Triangular. The higher order terms are not shown, except the final term, which takes $d$ inputs, similar to the original function. We can use Multi-index Notation to write this expansion as

f (X) = f_{0} + \sum_{{i_{1}, \dots, i_{s}} \subset {1, \dots, d}} f_{i_{1} \dots i_{s}} (X_{i_{1}}, \dots, X_{i_{s}}) .

This decomposition is not yet unique. By adding a restriction on the orthogonality of the terms, the decomposition is then unique. For ${i_{1}, \dots, i_{s}} \neq {j_{1}, \dots, j_{t}}$ ,

\int_{[0, 1]^{d}} f_{i_{1} \dots i_{s}} (X_{1}, \dots, X_{s}) \cdot f_{j_{1} \dots j_{t}} (X_{1}, \dots, X_{t}) d X = 0.

We can compute these functions via

\begin{aligned} f_{0} & = \underset{E (Y)}{\underset{⏟}{\int_{[0, 1]^{d}} f (X) d X}}, \\ f_{i} (X_{i}) & = \underset{E (Y | X_{i})}{\underset{⏟}{\int_{[0, 1]^{d - 1}} f (X) d X_{\sim {i}}}} - f_{0}, \\ f_{i j} (X_{i}, X_{j}) & = \underset{E (Y | X_{i}, X_{j})}{\underset{⏟}{\int_{[0, 1]^{d - 2}} f (X) d X_{\sim {i, j}}}} - f_{0} - f_{i} - f_{j}, \\ \dots \end{aligned}

Now, assume that $f$ is Square Integrable, meaning

\int_{[0, 1]^{d}} f^{2} (X) d X < \infty .

Plugging in the expansion, noting the aforementioned orthogonality constraint, and decoupling, we can represent the variance of $Y$ ( $E (Y^{2}) - E (Y)^{2}$ ) as

\int_{[0, 1]^{d}} f^{2} (X) d X - f_{0}^{2} = \sum_{{i_{1}, \dots, i_{s}} \subset {1, \dots, d}} \underset{D_{i_{1} \dots i_{s}}}{\underset{⏟}{\int_{[0, 1]^{s}} f_{i_{1} \dots i_{s}}^{2} (X_{i_{1}}, \dots, X_{i_{s}}) d X_{i_{1}} \dots d X_{i_{s}}}} = D .

The Sobol' Indices are then

S_{{i_{1}, \dots, i_{s}}} = \frac{D_{{i_{1}, \dots, i_{s}}}}{D},

and sum to $1$ . The first-order indices are $S_{i}$ , while the total indices $S_{i}^{T}$ include the interactions with higher-order terms. $\sum S_{i} \leq 1$ , but $\sum S_{i}^{T} \geq 1$ , as the total indices include the interaction terms for multiple $i$ (e.g. $S_{12}$ contributes to both $S_{1}^{T}$ and $S_{2}^{T}$ ).

Sobol' Indices from PCE

We can compute the Sobol' Indices from a Polynomial Chaos Expansion (PCE) instead. The PCE, with coefficients $c_{1 \dots d}$ and basis functions $ψ_{1 \dots d}$ is given as

f (X) = c_{0} + \sum_{{i_{1}, \dots i_{s}} \subset {1, \dots, d}} c_{i_{1} \dots i_{s}} ψ_{i_{1} \dots i_{s}} (X_{1}, \dots, X_{s}) .

This is quite similar to the Sobol' expansion. We can relate the PCE coefficients to the Sobol' indices, accounting for the basis functions as well. This gives

D_{i_{1} \dots i_{s}} = c_{i_{1} \dots i_{s}} \int_{[0, 1]^{s}} ψ_{i_{1} \dots i_{s}}^{2} (X_{i_{1}}, \dots, X_{i_{s}}) d X_{i_{1}} \dots d X_{i_{s}} .

Often, $E (ψ_{i_{1} \dots i_{s}}^{2}) = 1$ , meaning $D_{i_{1} \dots i_{s}} = c_{i_{1} \dots i_{s}}$ . Thus, we can rely on non-intrusive PCE techniques (such as Sparse Regression) to avoid computing all the integrals above.

Resources

Related

Main Idea

Sobol' Indices

Sobol' Indices from PCE