Completed DL homeworks I did on the course at Skoltech

1. Did the matrix differentiation task (github doesn't render LaTeX code from .ipynb)

2. Learned the concept of the Module class, forward and backpropagation passes and the Sequential container class for the chains of modules.

3. Implemented myself in numpy and/or learned:

• The most popular layers: Fully-connected (Linear), SoftMax, LogSoftMax, Batch-normalization and Dropout.
• Activation functions: ReLU, Leaky ReLU, ELU, SoftPlus
• Criterions: MeanSquaredError criterion, Negative LogLikelihood criterion (both numerically stable and unstable)
• Optimizers: SGD optimizer with momentum, Adam optimizer
• Convolution Neural Netork related layers: Convolutional (like Conv2d in PyTorch), Max Pooling (like MaxPool2d in PyTorch), Flatten (reshapes the input to the 1-dimension output)

4. Trained:

• Digit classification Linear layer - based network on the MNIST dataset. Compared different activation functions and optimizers. Showed the effect of the Batch normalization and Dropout layers.
• CNN network for the same task.

In this Homework I was training PyTorch skills.

Implemented:

• Conway's Game of Life in PyTorch
• Character recognicion problem on the notMNIST dataset. It consists of 10 letters and 14000 train samples. The solution was made in the low-level PyTorch functionalities.

Implemented:

• Image multiclass classification problem on the Tiny ImageNet dataset (200 classes, 100K training images, 10K validation images). In comparison to the previous part, I was using all PyTorch functionalities. I used ResNet neywork architecture to solve this problem.

Implemented, using UNet architecture:

• Image segmentation problem from the scratch on the dataset with colon-cancer cells images.
• Image denoising, where I increased the quality of an image
• Image inpainting, here the blind spot of the image was filled in accordance with the rest of the image.

Used Autoencoder and Variational Autoencoder to reconstruct face images

NLP practise

Did the Part of Speech (POS) tagging task ith help of Transformer's Encoder

Was trying to reconstruct the voiced speech from the text with help of DureIAN architecture