CS294 Lecture 16: Zig-Zag Graph Product

In which we give an explicit construction of expander graphs of polylogarithmic degree, state the properties of the zig-zag product of graphs, and provide an explicit construction of a family of constant-degree expanders using the zig-zag product and the polylogarithmic-degree construction.

A family of expanders is a family of graphs {G_n = (V_n,E_n)}, {|V_n|=n}, such that each graph is {d_n}-regular, and the edge-expansion of each graph is at least {h}, for an absolute constant {h} independent of {n}. Ideally, we would like to have such a construction for each {n}, although it is usually enough for most applications that, for some constant {c} and every {k}, there is an {n} for which the construction applies in the interval {\{ k, k+1, \ldots, ck \}}, or even the interval {\{ k, \ldots, ck^c\}}. We would also like the degree {d_n} to be slowly growing in {n} and, ideally, to be bounded above by an explicit constant. Today we will see a simple construction in which {d_n = O(\log^2 n)} and a more complicated one in which {d_n = O(1)}.

Continue reading

CS294 Lecture 15: Abelian Cayley graphs

In which we show how to find the eigenvalues and eigenvectors of Cayley graphs of Abelian groups, we find tight examples for various results that we proved in earlier lectures, and, along the way, we develop the general theory of harmonic analysis which includes the Fourier transform of periodic functions of a real variable, the discrete Fourier transform of periodic functions of an integer variable, and the Walsh transform of Boolean functions.

Earlier, we prove the Cheeger inequalities

\displaystyle  \frac{\lambda_2}{2} \leq \phi(G) \leq \sqrt{2 \lambda_2}

and the fact that Fiedler’s algorithm, when given an eigenvector of {\lambda_2}, finds a cut {(S,V-S)} such that {\phi(S,V-S) \leq 2\sqrt{\phi(G)}}. We will show that all such results are tight, up to constants, by proving that

  • The dimension-{d} hypercube {H_d} has {\lambda_2 = 1- \frac 2d} and {h(H_d) = \frac 1d}, giving an infinite family of graphs for which {\frac{\lambda_2}{2} = \phi(G)}, showing that the first Cheeger inequality is exactly tight.
  • The {n}-cycle {C_n} has {\lambda_2 = O(n^{-2})}, and {\phi(C_n) = \frac 2n}, giving an infinite family of graphs for which {\phi(G) = \Omega(\sqrt{\lambda_2})}, showing that the second Cheeger inequality is tight up to a constant.
  • There is an eigenvector of the 2nd eigenvalue of the hypercube {H_d}, such that Fiedler’s algorithm, given such a vector, outputs a cut {(S,V-S)} of expansion {\phi(S,V-S) = \Omega(1/\sqrt{d})}, showing that the analysis of the Fiedler’s algorithm is tight up to a constant.

In this lecture we will develop some theoretical machinery to find the eigenvalues and eigenvectors of Cayley graphs of finite Abelian groups, a class of graphs that includes the cycle and the hypercube, among several other interesting examples. This theory will also be useful later, as a starting point to talk about constructions of expanders.

For readers familiar with the Fourier analysis of Boolean functions, or the discrete Fourier analysis of functions {f: {\mathbb Z}/N{\mathbb Z} \rightarrow {\mathbb C}}, or the standard Fourier analysis of periodic real functions, this theory will give a more general, and hopefully interesting, way to look at what they already know.

Continue reading

CS 294 Lecture 14: ARV Analysis, Part 3

In which we complete the analysis of the ARV rounding algorithm

We are finally going to complete the analysis of the Arora-Rao-Vazirani rounding algorithm, which rounds a Semidefinite Programming solution of a relaxation of sparsest cut into an actual cut, with an approximation ratio {O(\sqrt {\log |V|})}.

In previous lectures, we reduced the analysis of the algorithm to the following claim.

Continue reading

CS294 Lecture 13: ARV Analysis, cont’d

In which we continue the analysis of the ARV rounding algorithm

We are continuing the analysis of the Arora-Rao-Vazirani rounding algorithm, which rounds a Semidefinite Programming solution of a relaxation of sparsest cut into an actual cut, with an approximation ratio {O(\sqrt {\log |V|})}.

In previous lectures, we reduced the analysis of the algorithm to the following claim.

Continue reading

CS294 Lecture 12: ARV Analysis

In which we begin the analysis of the ARV rounding algorithm

We want to prove

Lemma 1 (ARV Main Lemma) Let {d} be a negative-type metric over a set {V} such that the points are contained in a unit ball and have constant average distance, that is,

  • there is a vertex {z} such that {d(v,z)\leq 1} for every {v\in V}
  • {\sum_{u,v\in V} d(u,v) \geq c\cdot |V|^2}

Then there are sets {S,T \subseteq V} such that

  • {|S|, |T| \geq \Omega(|V|)};
  • for every {u\in S} and every {v\in T}, {d(u,v) \geq 1/{O(\sqrt {\log |V|})}}

where the multiplicative factors hidden in the {O(\cdot)} and {\Omega(\cdot)} notations depend only on {c}.

In this lecture, we will show how to reduce the ARV Main Lemma to a statement of the following form: if {\{ {\bf x}_v \}_{v\in V}} is a set of vectors such that the metric {d(\cdot, \cdot)} in the ARV Main Lemma can be written as {d(u,v) = || {\bf x}_u - {\bf x}_v ||^2}, and {{\bf g}} is a random Gaussian vectors, and if {\ell} is such that with {\Omega(1)} probability, there are {\Omega(n)} disjoint pairs {u,v} such that {d(u,v) < \ell} and {| \langle g, {\bf x}_u\rangle - \langle g, {\bf x}_v \rangle | \geq \Omega(1)}, then {\ell \geq \Omega(1/\sqrt{\log n})}. We will then prove such a statement in the next lecture.

Continue reading

CS294 Lecture 10: Bourgain’s Theorem

In which we prove Bourgain’s theorem.

Today we prove the following theorem.

Theorem 1 (Bourgain) Let {d: V\times V \rightarrow {\mathbb R}} be a semimetric defined over a finite set {V}. Then there exists a mapping {F: V \rightarrow {\mathbb R}^m} such that, for every two elements {u,v \in R},

\displaystyle  || F(u) - F(v)||_1 \leq d(u,v) \leq ||F(u)-F(v)||_1 \cdot c\cdot \log |V|

where {c} is an absolute constant. Given {d}, the mapping {F} can be found with high probability in randomized polynomial time in {|V|}.

Together with the results that we proved in the last lecture, this implies that an optimal solution to the Leighton-Rao relaxation can be rounded to an {O(\log n)}-approximate solution to the sparsest cut problem. This was the best known approximation algorithm for sparsest cut for 15 years, until the Arora-Rao-Vazirani algorithm, which will be our next topic.

Continue reading

CS294 Lecture 9: The Sparsest Cut Problem

In which we introduce the sparsest cut problem and the Leighton-Rao relaxation.

1. The Uniform Sparsest Cut problem, Edge Expansion and {\lambda_2}

Let {G=(V,E)} be an undirected graph with {n:= | V |} vertices.

We define the uniform sparsity of a cut {(S,V-S)} as

\displaystyle  {\sf usc}_G(S) := \frac {E(S,V-S)}{ |S| \cdot |V-S| }

(we will omit the subscript when clear from the context) and the uniform sparsest cut of a graph is

\displaystyle  {\sf usc}(G):= \min_{S} {\sf usc}_G(S)

In {d}-regular graphs, approximating the uniform sparsest cut is equivalent (up to a factor of 2 in the approximation) to approximating the edge expansion, because, for every cut {(S,V-S)}, we have

\displaystyle  \phi(S,V-S) = \frac {E(S,V-S)}{d \cdot \min \{ |S|, |V-S| \} }

and, noting that, for every, {S},

\displaystyle  \frac 1n |S| \cdot |V-S| \leq \min \{ |S|, |V-S| \} \leq \frac 2n |S| \cdot |V-S|

we have, for every {S},

\displaystyle  \phi (S,V-S) \leq \frac nd \cdot {\sf usc}(S) \leq 2 \phi(S,V-S)

and so

\displaystyle  \phi(G) \leq \frac nd \cdot {\sf usc}(G) \leq 2 \phi(G)

It will be instructive to see that, in {d}-regular graphs, {\lambda_2} is a relaxation of {\frac nd {\sf usc}(G)}, a fact that gives an alternative proof of the easy direction {\lambda_2 \leq 2 \phi(G)} of Cheeger’s inequalities.

Continue reading

I choose to remember Scalia by what he loved best

His own words:

  • On where black kids should go to college

    “There are those who contend that it does not benefit African-Americans to get them into the University of Texas, where they do not do well, as opposed to having them go to a less-advanced school, a less — a slower-track school, where they do well”

    Scalia during oral arguments concerning Fisher vs. University of Texas

  • Gay sex should be illegal, or else we may have less homophobia

    “Today’s opinion is the product of a Court, which is the product of a law-profession culture, that has largely signed on to the so-called homosexual agenda, by which I mean the agenda promoted by some homosexual activists directed at eliminating the moral opprobrium that has traditionally attached to homosexual conduct. ”

    Scalia’s dissent in Lawrence vs Texas

  • Native Americans, during their religious rituals, cannot be excluded from obeying the law because of their religious belief

    “Conscientious scruples have not, in the course of the long struggle for religious toleration, relieved the individual from obedience to a general law not aimed at the promotion or restriction of religious beliefs. The mere possession of religious convictions which contradict the relevant concerns of a political society does not relieve the citizen from the discharge of political responsibilities. (…) To permit this would be to make the professed doctrines of religious belief superior to the law of the land, and in effect to permit every citizen to become a law unto himself.”

    Scalia’s majority opinion in Employment Division v. Smith

  • To counteract the consequences of the above ruling, Congress passed the Religious Freedom Restoration Act, to protect the ability of religious minorities to carry on their rituals. Subsequently, the Supreme Court found that corporations have religious beliefs, and that not providing comprehensive health coverage is a ritual

    “By requiring the Hahns and Greens and their companies to arrange for such coverage, the HHS mandate demands that they engage in conduct that seriously violates their religious beliefs. (…) The contraceptive mandate, as applied to closely held corporations, violates RFRA”

    Alito’s majority opinion, joined by Scalia, in Burwell v. Hobby Lobby, inc.

  • On the majority opinion in Obergefell v. Hodges

    “If, even as the price to be paid for a fifth vote, I ever joined an opinion for the Court that began: ‘The Constitution promises liberty to all within its reach, a liberty that includes certain specific rights that allow persons, within a lawful realm, to define and express their identity,’ I would hide my head in a bag. The Supreme Court of the United States has descended from the disciplined legal reasoning of John Marshall and Joseph Story to the mystical aphorisms of the fortune cookie.”

    Scalia’s dissent in Obergefell v. Hodges