\documentclass[english,course]{Notes}

\title{Lecture \#: Topic}
\subject{Theory of Machine Learning}
\author{Instructor: Aditya Bhaskara \qquad Scribe: Your Name}
\date{01}{01}{2017}
\place{{\bf CS 5966/6966: Theory of Machine Learning}}

\newcommand{\Tr}{\text{Tr}}
\newcommand{\iprod}[1]{\langle #1 \rangle}
\newcommand{\R}{\mathbb{R}}
\renewcommand{\norm}[1]{\lVert #1 \rVert}
\newcommand{\su}[1]{^{(#1)}}
\newcommand{\bu}{\mathbf{u}}
\renewcommand{\deg}{\text{deg}}

\begin{document}
\vspace{-10pt}
\begin{abstract}
This is a short, two-line summary of the day's lecture. Should provide a rough set of topics covered or questions discussed.
\end{abstract}

%%% Notes start in detail
\section{Introduction}
This is where you can start writing the notes. Here is how you add an equation.

\begin{equation}
\label{eq:transition1}
p\su{t}(v) = \sum_{u \in \text{nbr}(v)} p\su{t-1}(u) \cdot \frac{1}{\deg(u)}.
\end{equation}

Adding a side comment is rather easy: simply write it in a TODO environment. Here is an example.
\TODO{This text appears to the side of the notes. It is very helpful to recap definitions that may have been forgotten, or add tangential comments.} 

\section{Another section}
Let define $p\su{t}$ as a vector in $\R^n$ whose $i$th entry is $p\su{t}(i)$.
We can use Eq.~\eqref{eq:transition1} to write down $p\su{t}$ in terms of $p\su{t-1}$.  The {\em transition matrix} $M$ will be defined so that we have
\[ p\su{t} = M p\su{t-1}. \]

\eqref{eq:transition1} tells us what the entries of $M$ should be. Considering the $v$th row of the output, we get
\[ p\su{t}(v) = \sum_{u} M_{uv} p\su{t-1}(u). \]
Comparing with~\eqref{eq:transition1}, we must set $M_{uv} = 0$ if there is no edge $uv$, and $M_{uv} = \frac{1}{\deg(u)}$ when $uv$ is an edge.  If all the degrees are equal to $d$, this is precisely the adjacency matrix scaled by a factor $d$. Otherwise, a bit of thought shows that $M$ is the adjacency matrix with its $i$th column scaled by $1/\deg(i)$. 

Let us write $A$ to denote the adjacency matrix, and $D$ to denote the diagonal matrix of degrees\TODO{I.e., $D(i,i) = \deg(i)$ and $D(i,j)=0$ for $i \ne j$.} Then the discussion above implies that
\[ M = A D^{-1}. \]
We can thus write $p\su{t}$ in a simple way:
\[ p\su{t} = M p\su{t-1} = M^2 p\su{t-2} = \dots = M^t p\su{0}.\]
What if we repeat this process forever? Does the distribution $p\su{t}$ coverge to some vector? Does it cycle between a small number of vectors? If it does converge, at what rate?

We now develop machinery to answer questions of this kind. The key observation is that the vector $p\su{t}$ evolves exactly like the {\em power method} we saw in the last lecture. This suggests the connection to eigenvalues of $M$.  Further, if the process does converge, it must be to some vector $p$ that satisifes $p = Mp$, or equivalently, $p$ must be an eigenvector of $M$, with eigenvalue $1$.

\section{Eigenvalues of $M$}
The first thing to notice about $M$ is that it is not symmetric. Thus it is not immediately clear that it even has real eigenvalues. Luckily, our matrix has the following special form:\TODO{Note that $B(i,j) = 1/\sqrt{\deg(i)\deg(j)} = B(j, i)$. Also, $D^{1/2}$ and $D^{-1/2}$ are the diagonal matrices whose $(i,i)$th entries are $\sqrt{\deg(i)}$ and $1/\sqrt{\deg(i)}$ respectively.}
\[ M = P B P^{-1}, \qquad \text{where $B = D^{-1/2} AD^{-1/2}$ and $P = D^{1/2}$.}\]
The matrix $B$ is simply the normalized adjacency matrix we encountered earlier, which is a symmetric matrix. The equation above implies that $M$ and $B$ are {\em similar}, i.e., they have the same eigenvalues. In fact, if $v$ is an eigenvector of $B$ satisfying $B v = \lambda v$, then $P	 v$ is an eigenvector $M$,\TODO{The notion of similar matrices is a standard one in linear algebra.}
\[ M (Pv) = P B P^{-1} Pv = \lambda Pv. \]
Now, the eigenvalues of $B$ are something we understand! As we saw earlier, they all lie between $-1$ and $1$, and the multiplicity of $1$ is precisely the number of connected components in $G$. We also saw that the smallest eigenvalue is $-1$ if and only if $G$ is bipartite. 

Suppose $G$ is connected. This means that $B$ has a unique eigenvector $\bu$ of eigenvalue $1$ (up to scaling), and we also know its entries -- the $i$th entry is $\sqrt{\deg(i)}$. Thus the corresponding eigenvector of $M$ is $D^{1/2} \bu$, whose $i$th entry is $\deg(i)$.  Let us scale it so that the entries sum to $1$, and denote this vector by $\pi$.  The $i$th entry of $\pi$ is thus 
\[ \pi(i) = \frac{\deg(i)}{\sum_{j} \deg(j)} = \frac{\deg(i)}{2m}, \] 
where $m$ is the number of edges in $G$.



\section{Stationary distribution}
The stationary distribution for a random walk is a distribution that remains unchanged upon taking a step of the walk. I.e., a vector $\bu$ with nonnegative entries that sum up to $1$, and which satisfies $M\bu = \bu$.  Observe that this is exactly what we saw with $\pi$ above. Thus $\pi$ is one stationary distribution for the random walk $M$.

Is it the only stationary distribution? If $G$ is connected, the uniqueness of the top eigenvector implies that this is indeed the case. Furthermore, the key property of the stationary distribution is that for {\em any} start distribution $p\su{0}$, taking sufficiently many steps of the random walk converges to the stationary distribution, as long as the graph is not bipartite.

To see this, suppose we start with $p\su{0}$ and repeatedly apply $M$.  Thus we have $p\su{t} = M^t p\su{0}$. From the analysis of the power iteration we saw last class, we have that as long as $\iprod{p\su{0}, \pi} >0$ and the gap between the largest and the second largest (in magnitude) eigenvalues is $>0$, the power method converges. The first condition is true because for any distribution $p\su{0}$, we have $\iprod{p\su{0}, \pi} \ge \min_i \pi(i) >0$ (because degrees are all $>0$).  The second condition follows because in any graph that is not bipartite, we have a non-zero gap $1- \max_{i \ne n} |\lambda_i(M)|$.

\section{Convergence to stationarity}
We can also quantify the convergence analysis above.  

\begin{definition}
	The time $T$ for which $\norm{p\su{T} - \pi}_1 < 1/4$ is called the {\em mixing time} for the random walk $M$.
\end{definition}

Suppose $G$ is connected.  Then we know that $1 - |\lambda_{n-1}| \ge 1/n^3$, as a consequence of one of the problems on HW2.\TODO{The exercise was for $d$-regular graphs, and had a bound $1/n^2$ for the Laplacian.  When we consider the normalized Laplacian, we get a bound $1/n^3$, since the max degree is $\le n$.}  Further, if we have a graph that is not bipartite, we can show that $1- |\lambda_1| \ge 1/n^3$.  

Finally, since $\min_i \pi(i) \ge 1/m > 1/n^2$, we have that for $t = O(\log n/n^3)$, we will have $\norm{\pi - p\su{t}}_2 < 1/n$.  A mild technicality here is that in the last class, we measured closeness in the $\ell_2$ norm, while in the definition of the mixing time, we care about $\ell_1$ norm.  To go from one bound to the other, we use the inequality that for any $n$ dimensional vector $x$, \[ \norm{x}_2 \le \norm{x}_1 \le \sqrt{n} \norm{x}_2. \]

Thus the above reasoning shows that the mixing time in any connected, $n$-vertex non-bipartite graph is $O(n^3 \log n)$.

In the next class, we will see how to get rid of the non-bipartite assumption, by looking at a simple modification of the random walk.




\end{document}
