loveandhate

12M: Transfer Learning  Transfer learning: Transfer learning is the reuse of a pretrained model on a new problem In transfer learning, a machine exploits the knowledge gained from a previous task to improve generalization about the other Transfer learning has the benefit of decreasing the training time for a neural network model and resulting in lower generalization error Sometimes it takes days or even weeks to train a deep neural network from scratch on a complex task. Transfer learning decreases the task substantially How it works: In transfer learning, the early and middle layers are used and we only retrain the latter layers When to use transfer learning: There isn’t enough labeled training data There already exists a network that is pretrained on a similar task When task 1 and task 2 have the same input Implementing transfer learning: Two main approaches: Weight initialization Training a model to reuse it Using a pretrained model (select source model, reuse model, tune model)...

  4M: Regularized autoencoder: Undercomplete autoencoders can fail to learn anything useful if the encoder and decoder are given too much capacity Unlike those, Regularized autoencoders use a loss function that encourages the model to have other properties besides the ability to copy its input to its output Sparsity of the representation Robustness to noise or to missing inputs Smallness of the derivative of the representation Types of regularized autoencoder: Sparse autoencoder Denoising autoencoder Compare and contrast autoencoder vs CNN Autoencoders:  Learn a compact representation of the dat for reconstruction Typically consists of fully connected layers It is an unsupervised learning technique Applicable to general purpose tasks CNN: Extract hierarchical features from images for classification or other tasks Consists of convolutional, pooling and fully connected layers It is a supervised learning technique Specialized for image related tasks While both autoencoders and CN...