\documentclass[11pt,onecolumn]{article}
\usepackage{amssymb, amsmath, amsthm,graphicx, paralist,algpseudocode,algorithm,cancel,url,color}
\usepackage{sectsty}
\usepackage{fancyvrb}
\usepackage{mathrsfs}
\usepackage{multirow}
\usepackage{hhline}
\usepackage{booktabs}
\usepackage[table]{xcolor}
\usepackage{tikz}
\newcommand{\bvec}[1]{\mathbf{#1}}
\newcommand{\R}{\mathbb{R}}
\DeclareMathOperator{\Tr}{Tr}
\sectionfont{\Large\sc}
\subsectionfont{\sc}
\usepackage[margin=1 in]{geometry}
\begin{document}
\noindent
\textsc{\Large CS 3220: Homework 4}\\
Instructor: Anil Damle\\
Due: October 28, 2020\\
\subsection*{Policies}
You may discuss the homework problems freely with other students, but please refrain from looking at their code or writeups (or sharing your own). Ultimately, you must implement your own code and write up your own solution to be turned in. Your solution, including plots and requested output from your code should be submitted via the CMS as a pdf file. Additionally, please submit any code written for the assignment via the CMS as well.\\
\hrule
\vspace{5mm}
\subsection*{Question 1:}
We talked about random vectors in class and how to bound their norms. Now, lets say we have a fixed vector $v\in\R^{n}$ and we would like to understand the angle between a random vector and $v.$ Specifically, let
\[
Z = \begin{bmatrix} Z_1\\ \vdots\\Z_n\end{bmatrix}
\]
be a random vector with entires that are independent and identically distributed random variables $Z_i\sim \mathcal{N}(0,1).$ Furthermore, let $\|v\|_2 = 1$ (since we are concerned with angles we may as well normalize $v$).
\begin{enumerate}[(a)]
\item Show that $\mathbb{E}[Z^Tv] = 0.$
\item Show that $P\{\lvert Z^Tv / \sqrt{n} \rvert > t\} \leq \frac{1}{nt^2}.$
\end{enumerate}
\subsection*{Question 2:}
We discussed multivariate normal random variables in class, for this problem we will do a few experiments with them to better understand their structure/behavior.
\begin{enumerate}[(a)]
\item First, for $n = 2$ generate a symmetric positive definite matrix $\Sigma\in\R^{2\times 2}.$ You may do this by forming a lower triangular matrix $L\in\R^{2\times 2}$ with non-zero diagonal and then letting $\Sigma = LL^T.$ While your specific choice of $\Sigma$ is up to you, it should not be diagonal. Also, pick a non-zero mean vector $\mu.$
\item Using built in routines to generate length 2 vectors with $\mathcal{N}(0,1)$ entries, generate 100 instances of random vectors distributed as $\mathcal{N}(\mu,\Sigma).$ Provide a scatter plot of your generated samples.
\item Now, compute the SVD of $\Sigma,$ let the left singular vectors be $u_1$ and $u_2$ and the singular values be $\sigma_1$ and $\sigma_2.$ Provide another scatter plot of your samples along with $\sigma_1 u_1$ and $\sigma_2 u_2$ shifted by the mean $\mu.$ What do you observe? How does the geometry of your samples related to the SVD?
\end{enumerate}
\end{document}