User:Egm6936.f10/Generalized polynomial chaos

=Generalized polynomial chaos=

The content of this section is mainly from Xiu 2010. For more details, one can refer to Xiu 2010, Chapter 5, p.57.

Some fundamentals of generalized polynomial chaos (gPC) will be presented in this section. Although expansion based on piecewise polynomials is possible, we will only focus on the globally smooth orthogonal polynomials expansion.

Single random variable case
Let $$\displaystyle Z$$ be a random variable with a distribution function $$\displaystyle F_Z(z)=P(Z \leq z)$$ and finite moments
 * {| style="width:100%" border="0"

$$\displaystyle \mathbb E[\left | Z \right |^{2m}] = \int \left | z \right |^{2m}dF_Z(z) < \infty, \;\;\; m\in \mathcal N $$ (ab1) where $$\displaystyle \mathcal N = \{0, 1, 2,..., N\}$$ and for a finite nonnegative integer N is an index set. The generalized polynomial chaos basis functions are the orthogonal polynomial functions satisfying
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle \mathbb E[\Phi_m(Z)\Phi_n(Z)] = \gamma_n\delta_{mn}, \;\;\; m,n \in \mathcal N $$ (ab2) where
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle \gamma_n = \mathbb E[\Phi_n^2(Z)],\;\;\; n \in \mathcal N $$ (ab3) are the normalization factors. If $$\displaystyle Z$$ is continuous, then its probability density function (PDF) exists such that $$\displaystyle dF_Z(z) = \rho(z)dz$$ and the orthogonality can be written as
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle \mathbb E[\Phi_m(Z)\Phi_n(Z)]=\int\Phi_m(Z)\Phi_n(Z)\rho(z)dz = \gamma_n\delta_{mn},\;\;\; m,n \in \mathcal N $$ (ab4) Similarity, when $$\displaystyle Z$$ is discrete, the orthogonality can be written as
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle \mathbb E[\Phi_m(Z)\Phi_n(Z)]=\sum_{i}\Phi_m(Z_i)\Phi_n(Z_i)\rho_i = \gamma_n\delta_{mn},\;\;\; m,n \in \mathcal N $$ (ab5) Above establish the correspondence between the distribution of the random variable $$\displaystyle Z$$ and the type of orthogonal polynomials of its gPC basis.
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }

Strong approximation
Let $$\displaystyle Z$$ be a function of a random variable $$\displaystyle Z$$ whose probability distribution is $$\displaystyle F_Z(z) = P(Z\leq z)$$ and support is $$\displaystyle I_Z$$. A generalized polynomial chaos approximation in a strong sense is $$\displaystyle f_N(Z) \in \mathbb P_N(Z)$$, where $$\displaystyle \mathbb P_N(Z)$$ is the space of polynomials of $$\displaystyle Z$$ of degree up to $$\displaystyle N\geq 0$$, such that $$\displaystyle \left \| f(Z)-f_N(Z) \right \| \rightarrow 0$$ as $$\displaystyle N \rightarrow \infty$$, in a proper norm defined on $$\displaystyle I_Z$$. One obvious strong approximation is the orthogonal projection. Let
 * {| style="width:100%" border="0"

$$\displaystyle L_{dF_Z}^2(I_Z) = \{f : I_Z \rightarrow \mathbb R | \mathbb E[f^2] < \infty \} $$ (ab6) be the space of all mean-square integrable functions with norm $$\displaystyle \left \| f \right \|_{L_{dF_Z}^2} = (\mathbb E[f^2])^{1/2}$$. Then, for any function $$\displaystyle f \in L_{dF_Z}^2(I_Z) $$, we define its Nth-degree gPC orthogonal projection as
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle P_N f = \sum_{k=0}^{N}\hat{f_k}\Phi_k(Z), \hat{f_k} =\frac{1}{\gamma_k} \mathbb E[f(Z)\Phi_k(Z)] $$ (ab7) The existence and convergence of the projection follow directly from the classical approximation theory; i.e.,
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle $$ (ab8) which is also referred to as mean-square convergence. Though the requirement for convergence ($$\displaystyle L^2$$-integrable) is rather mild, the rate of the convergence will depend on the smoothness of the function $$\displaystyle f$$ in terms of $$\displaystyle Z$$. The smoother $$\displaystyle f$$ is, the faster the convergence.
 * style="width:95%" |
 * style="width:95%" |
 * f - P_N f || _{L_{dF_Z}^2} \rightarrow 0,\;\;\; N \rightarrow \infty
 * 
 * }

When gPC expansion $$\displaystyle f_N(Z)$$ of a function $$\displaystyle f(Z)$$ converges to $$\displaystyle f(Z)$$ in a strong norm, such as the mean-square norm of (ab8), it implies that $$\displaystyle f_N(Z)$$ converges to $$\displaystyle f(Z)$$ in probability, i.e., $$\displaystyle f_N \overset{P}{\rightarrow} f$$, which further implies the convergence in the distribution, i.e., $$\displaystyle f_N \overset{d}{\rightarrow} f$$, as $$\displaystyle N \rightarrow \infty$$.

In this strong approximation, it is necessary to know the explicit form of $$\displaystyle f$$ in terms of $$\displaystyle Z$$. In practice, however, sometimes only the probability distribution of $$\displaystyle f$$ is known. Hence, a gPC expansion in terms of $$\displaystyle Z$$ that converges strongly cannot be achieved because of the lack of information about the dependence of $$\displaystyle f$$ on $$\displaystyle Z$$.

Weak approximation
Let $$\displaystyle Y$$ be a random variables with distribution function $$\displaystyle F_Y(y)=P(Y \leq y)$$ and let $$\displaystyle Z$$ be a (standard) random variable in a set of gPC basis functions. A weak gPC approximation is $$\displaystyle Y_N \in \mathbb P_N(Z)$$, where $$\displaystyle \mathbb P_N(Z)$$ is the linear space of polynomials of $$\displaystyle Z$$ of degree up to $$\displaystyle N \geq 0$$, such that $$\displaystyle Y_N$$ converges to $$\displaystyle Y$$ in a weak sense, e.g., in probability.

Obviously, a strong gPC approximation implies a weak approximation, not vice versa. Example: Let $$\displaystyle Y \doteqdot \mathcal N(\mu, \sigma^2)$$ be a random variable with normal distribution. Naturally we choose $$\displaystyle Z \in \mathcal N(0,1)$$, a standard Gaussian random variables, and the corresponding Hermite polynomials as the gPC basis. Then a first order gPC Hermite expansion
 * {| style="width:100%" border="0"

$$\displaystyle Y_1(Z) = \mu H_0 + \sigma H_1(Z) = \mu + \sigma Z $$ (ab9) will have precisely the distribution $$\displaystyle \mathcal N(\mu, \sigma^2)$$. Therefore, $$\displaystyle Y_1(Z)$$ can approximate the distribution of $$\displaystyle Y$$ exactly. However, if all that is known is the distribution of $$\displaystyle Y$$, then one cannot reproduce pathwise realization of $$\displaystyle Y$$ via $$\displaystyle Y_1(Z)$$. In fact, $$\displaystyle \tilde{Y_1}(Z) = \mu H_0 - \sigma H_1(Z)$$ has the same $$\displaystyle \mathcal N(\mu, \sigma^2)$$ distribution but entirely different pathwise realization from $$\displaystyle Y_1$$. When $$\displaystyle Y$$ is an arbitrary random variable with only its cdf known, a direct gPC projection in the form of (ab7) is not possible. At this moment, we still seek an Nth degree gPC expansion in the form of
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle Y_N = \sum_{k=0}^{N} a_k\Phi_k(Z), $$ (ab10) with
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle a_k = \frac{1}{\gamma_k}\mathbb E[Y\Phi_k(Z)],\;\;\; 0 \leq k \leq N $$ (ab11)
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }

where $$\displaystyle \gamma_k = \mathbb E[\Phi_k^2] $$ are the normalization factor, then the expextation in the coefficient evaluation is not properly defined and cannot be carried out, as the dependence between $$\displaystyle Y$$ and $$\displaystyle Z$$ is unknown. A strategy to circumvent the difficulty by using the distribution function $$\displaystyle F_Y(y)$$ was proposed. This results in a weak gPC approximation. By definition, $$\displaystyle F_Y : I_Y \rightarrow [0,1]$$, where $$\displaystyle I_Y$$ is the support of $$\displaystyle Y$$, $$\displaystyle F_Z(z) = P(Z\leq z) : I_Z \rightarrow [0,1]$$. Since $$\displaystyle F_Y$$ and $$\displaystyle F_Z$$map $$\displaystyle Y$$ and $$\displaystyle Z$$, respectively, to a uniform distribution in $$\displaystyle [0,1]$$, we rewrite the expectation in (ab11) in terms of a uniformly distributed random variable in $$\displaystyle [0,1]$$. Let $$\displaystyle U = F_Y(y) = F_Z(z) \doteqdot \mathcal U(0,1)$$; then $$\displaystyle Y = F_Y^{-1}(U)$$ and $$\displaystyle Z = F_Z^{-1}(U)$$. Now (ab11) can be rewritten as
 * {| style="width:100%" border="0"

$$\displaystyle a_k = \frac{1}{\gamma_k}\mathbb E_U[F_Y^{-1}(U)\Phi_k(F_Z^{-1}(U))] = \frac{1}{\gamma_k}\int_{0}^{1}F_Y^{-1}(u)\Phi_k(F_Z^{-1}(u))du $$ (ab12) This is properly defined finite integral in $$\displaystyle [0,1]$$ and can be evaluated via traditional method (e.g., Gauss quadrature). $$\displaystyle \mathbb E_U$$ represents that the expectation is over the random variable $$\displaystyle U$$. Alternatively, one can choose to transform the expectation in (ab11) into the expectation in terms of $$\displaystyle Z$$ by utilizing the fact that $$\displaystyle Y = F_Y^{-1}(F_Z(Z))$$. Then the expectation in (ab11) can be rewritten as
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }
 * {| style="width:100%" border="0"

$$\displaystyle a_k = \frac{1}{\gamma_k}\mathbb E_Z[F_Y^{-1}(F_Z(Z))\Phi_k(Z)] = \frac{1}{\gamma_k}\int_{I_Z}F_Y^{-1}(F_Z(z))\Phi_k(z)dF_Z(z) $$ (ab13) (ab12) and (ab13) are mathematically equivalent. $$\displaystyle $$
 * style="width:95%" |
 * style="width:95%" |
 * 
 * }

 I got a question on the difference between strong approximation and weak approximation used in Xiu's book. Case 1. The distinction between strong and weak lies in which function space is utilized for the function of random variable.

If the moment of the random variable is finite, that is "mean-square integrable" as from the book, and we can have corresponding gPC basis functions for approximation of this random variable, then this approximation is not only convergence in probability, but also in distribution. This is strong approximation. But if we don't have the finite moment restriction, say only require the mean is finite, we still can find corresponding gPC functions to approximate the random variable, then this approximation is weak. It only converges in probability, but not in distribution.

Case 2. The difference of strong and weak depends on the convergence rate.

If the gPC expansion converges both in probability and distribution, it is strong approximation. If it only converges in probability, it is weak approximation. But Xiu has pointed out in the book that both strong approximation and weak approximation are converges in probability and distribution.

Case 3. The difference lies in which function we used in the evaluation of the coefficient factors for the random variable.

If we use the explicit function of random variable, which actually unattainable in practice, the approximation is strong. If we use the cdf of random variable, the approximation is weak. Both approximations are converges in probability and distribution. Hylon.Chen 03:11, 15 February 2012 (UTC)