5. Painting Pictures with Neural Networks Using VAEs

 As you saw in Chapter 4, Teaching Networks to Generate Digits, deep neural networks are a powerful tool for creating generative models for complex data such as images, allowing us to develop a network that can generate images from the MNIST hand-drawn digit database. In that example, the data is relatively simple; images can only come from a limited set of categories (the digits 09 through 9) and are low-resolution grayscale data.

What about more complex data, such as color images drawn from the real world? One example of such "real world" data is the Canadian Institute for Advanced Research 10 class dataset, denoted as CIFAR-10. It is a subset of 60,000 examples from a larger set of 80 million images, divided into ten classes - airplanes, cars, birds, cats, deep, dogs, frogs, horses, ships, and trucks. While still an extremely limited set in terms of the diversity of images we would encounter in the real world, these classes have some characteristics that make them more complex then MNIST. For example, the MNIST digits can vary in width, curvature, and a few other properties; the CIFAR-10 classes have a much wider potential range of variation for animal or vehicle photos, meaning we may require more complex models in order to capture this variation.

In this chapter, we will discuss a class of generative models known as Variational Autoencoders(VAEs), which are designed to make the generation of these complex, real-world images more tractable and tunable. Then do this by using a number of clever simplifications to make it possible to sample over the complex probability distribution represented by real-world images in a way that is scalable.

We will explore the following topics to reveal how VAEs works:

- How neural networks create low-dimensional representations of data, and some desirable properties of those representations

- How variational methods allow us to sample from complex data using these representations

- How using the reparameterization trick allows us to stabilize the variance of a neural network based on variational sampling- a VAE

- How we can use Inverse Autoregressive Flow(IAF) to tune the output of a VAE

- How to implement VAE/IAE in TensorFlow