CS5968/6968 -- Techniques in Algorithms and Approximation

Course Outline

The goal of the course is to introduce some of the ideas and techniques essential for research in algorithms, with specific emphasis on approximation algorithms. The course is aimed at early graduate students and advanced undergrads with basic background in algorithm design and mathematical proofs. A rough set of topics we will cover:

Convex relaxations for problems (LP and SDP), rounding, duality
Spectral methods, random walks in graphs
Convex optimization, gradient descent
Multiplicative weight update method (and how it leads to fast, distributed algorithms)
Hardness of approximation, Beyond worst case analysis

We will also discuss other aspects we care about in algorithm design, like robustness to noise (in the data) and parallelizability. In short, the goal is to give a taste of the various beautiful techniques in the area, and bring students to a level where they can easily follow and contribute to research in algorithms.

Prerequisites

There are no formal prerequisites, but students are expected to have a good grasp of the basic concepts in probability, linear algebra, and graph theory. They should also be comfortable with mathematical proofs and basic algorithm design.

Logistics

Time : 2:00pm-3:20pm, Tuesday and Thursday

Location : WEB L122

Exams

Mid-term: we will have the course mid-term during regular class on Tuesday, March 8th, 2016. Here are some practice problems. Here is the midterm with solutions.

Final: the course final will be on Tuesday, May 3, 2016, from 2-3:30PM. Here are some practice problems.

Instructor and Office Hours

Name: Aditya Bhaskara, MEB 3120
Contact: For any questions about the course, please email me at bhaskara AT cs DOT utah DOT edu
Office hours: Wednesdays 11AM - Noon

Class Schedule, Notes

There is no single textbook for the course, so the lecture slides and notes will be posted here, and will be the main references.

Week 1: Introduction, course overview, and review of basic probability. Introduction to linear programming relaxations and approximation algorithms. (Notes from Lecture 1, Lecture 2)
Week 2: Recap of integrality gaps. Concentration bounds and the probabilistic method. Introduction to graph partitioning and the sparsest cut problem. (Notes from Lecture 3, Lecture 4)
Week 3: The spectrum of a graph, Cheeger's inequality. (Notes from Lecture 5, Lecture 6)
Week 4: More on eigenvalues and their computation; Intro to random walks in graphs. (Notes from Lecture 7, Lecture 8)
Week 5: Random walks, mixing time, connection to expansion; proving expansion via "canonical paths". (Notes from Lecture 9, Lecture 10)
Week 6: Convex optimization, definitions, gradient descent, separation oracles and the ellipsoid algorithm (Notes from Lecture 11, Lecture 12)
Week 7: Semidefinite programming, formulating problems as SDPs, the Goemans-Williamson algorithm for MaxCUT (Notes from Lecture 13, Lecture 14)
Week 8: Farkas lemma and duality in linear programs. (Section 8 of Goemans' notes for Farkas' lemma, and these notes for LP duality)
Week 9: Midterm exam
Week 10: Von Neumann's Min-max theorem, the multiplicative weight update method. (See Section 3 of Xi Chen's notes, and the survey of Arora, Hazan and Kale)
Week 11: How to find linear classifiers and play two-player games via MWU (Sections 3.1 and 3.2 of the survey above); routing with low congestion (see these notes for details)
Week 12: Hardness of approximation: basics of complexity, gap "inducing" and gap preserving reductions; gap amplification; statement of the PCP theorem (See Luca Trevisan's survey, and this outline)
Week 13: Why the PCP theorem implies hardness of Max-3SAT, and a survey of known inapproximability results; Beyond worst case analysis -- different models. (For the first lecture, see Sections 3.1 and 6 of Luca Trevisan's survey, and for the second, see these notes)
Week 14: Stochastic block model (two blocks); Smoothed analysis of the condition number, introduction to bicriteria approximations.
Week 15: Review of the course, Prep for the final (Slides from the review lecture).

Homeworks

Homework 1 (Due January 28, 11:59PM)
Homework 2 (Due February 4, 11:59PM)
Homework 3 (Due February 26, 11:59PM)
(Some of the problems will be non-graded, meant mostly for understanding the material.)

References/Additional Material

Apart from the lecture notes, the following references (books, links to other course notes, etc) could be useful for interested students. If you find references I may have missed, please let me know.

Books and Surveys:

Randomized Algorithms, an excellent textbook by Rajeev Motwani and Prabhakar Raghavan.
The Design of Approximation Algorithms, by David Williamson and David Shmoys (freely available for personal use).
An easily readable survey on concentration inequalities, by Fan Chung and Linyuan Lu.
Introductory lecture notes on probability by David Pollard can be found here.
A beautiful set of lecture notes on expander graphs, by Shlomo Hoory, Nati Linial and Avi Wigderson.

Writing assignment:

Here are some interesting results in algorithms (biased towards topics we covered in class). Some are more technical than the others. The goal is to pick up one of the papers, and write a two-page summary on it. The goal is to learn how to summarize the essential ideas from a research paper. Make sure you mention (a) the main motivation, (b) the result, (c) key ideas in the proof, and (d) a rough description of the technical details.

Approximate Graph Coloring by Semidefinite Programming, by Karger, Motwani and Sudan. (Link)
Polynomial time approximation schemes for Euclidean traveling salesman and other geometric problems, by Sanjeev Arora. (Link)
Local Graph Partitioning using PageRank Vectors, by Anderson, Chung and Lang. (Link)
A Constructive Proof of the Lovasz Local Lemma, by Robin Moser. (Link)
Max Cut and the Smallest Eigenvalue, by Luca Trevisan. (Link)
Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web, by Karger, Lehman, Leighton, Levine, Lewin and Panigrahy. (Link)
How Bad is Selfish Routing?, by Roughgarden and Tardos. (Link)
(For the more ambitious) Smoothed Analysis: Why The Simplex Algorithm Usually Takes Polynomial Time, by Spielman and Teng. (Link)

Grading

The homeworks will account for 50% of the grade. The midterm and final will be 20% each. The remaining 10% will be based on a reading assignment, where the goal is to read a research paper and write a clear, two-page summary (or alternately give a 15 minute presentation).