Duplicate

Lecture 4 | Probability

Tags
Date
Gaussian Random Variable
f(x)=12πσexp⁡(−12σ2(x−μ)2)f(x) = \frac{1}{\sqrt{2\pi}\sigma}\exp(-\frac{1}{2\sigma^2}(x-\mu)^2)
•
Mean: E(X)=∫xf(x)dx=μ{\mathbb{E}}(X) = \int xf(x)dx = \mu
•
Variance: var(X)=E(X−E(X))2=σ2{\bm{\text{var}}}(X) = {\mathbb E}(X- {\mathbb E}(X))^2 = \sigma^2
•
Notation: X∼N(μ,σ2)X \sim {\mathcal N}(\mu, \sigma^2)
•
Central Limit Therem
Multivariate Gaussian
Conditional Density of Multivariate Gaussian
•
두 gaussian distribution 의 consitional distribution 도 gaussian 임
◦
두 gaussian p(x),p(y)p(x),p(y) 에 대해 p(x∣y)p(x|y) 또한 gaussian
◦
E(x∣y)=E(x)+ΣxyΣyy−1(y−E(y)){\mathbb E}(x|y) = {\mathbb E}(x) + \Sigma_{xy}\Sigma_{yy}^{-1}(y-{\mathbb E}(y))
◦
Σx∣y=Σxx−ΣxyΣyy−1Σyx\Sigma_{x|y} = \Sigma_{xx} - \Sigma_{xy}\Sigma_{yy}^{-1}\Sigma_{yx}
•
Proof 는 중요하진 않지만, 각 라인을 따라갈 순 있어야 함

Matrix Inversion Lemma

•
A11,A22A_{11}, A_{22} 가 invertible 하고, A=[A11A12A21A22]A = \begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix} 일 때,
[A11A12A21A22]−1A[I−A11−1A120I]−1=[A1100A22−A21A11−1A12]−1\begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}^{-1} A \begin{bmatrix} I & -A_{11}^{-1}A_{12} \\ 0 & I \end{bmatrix}^{-1} = \begin{bmatrix} A_{11} & 0 \\ 0 & A_{22}-A_{21}A_{11}^{-1}A_{12} \end{bmatrix}^{-1}

Random Processes

•

Linear Regression

•
f(x)=xTwf(x) = {\bf x}^T{\bf w}
•
y(x)=f(x)+ϵy(x) = f(x) + \epsilon, where ϵ∼N(0,σn2)\epsilon \sim {\mathcal N}(0, \sigma_n^2)
•
Weight-Space View: Bayesian Formulation
•
Bayesian Formulation
p(w∣y,X)=P(Y∣X,w)P(w)P(y∣X)p({\bf w} | y, X) = \frac{P(Y|X,{\bf w})P({\bf w}) }{P(y|X)}
Weight-Space View: Posterior Distribution
Weight-Space View: Kernel Trick (1)
•
ϕ:RD→RN\phi:{\mathbb R}^D \to {\mathbb R}^N: high dimenstional feature space 로의 mapping function
Function-Space View
Kriging
•