NLP

Step by step NLP examples using PyTorch and Lightning

1. Character Generation with LSTMCell

• Character RNN that's trained with a covid_19 faq file
• After training, it generates text based on a prime input.
• Generated text and loss values are recorded to and observed

2. Character Generation with LSTM

• Same as above but this time it uses LSTM block

3. Language translation with LSTM

• Seq2Seq training and inference using LSTM layer

4. Language translation with GRU

• Same as above but utilizes GRU instead of LSTM and more input used for GRU and decoder outputs

5. Language translation with Bahdanau attention

• Same as above but utilizes Bahdanau attention (not self attention) using bidirectional GRU

6. Language translation with padding optimization

• Same as above but utilizes pack_padded_sequence, pad_packed_sequence, and source masking in attention to optimize paddings

7. Language translation with transformers

• Basic implementation of transformer architecture for translation problem

References

• seq2seq examples by Ben Trevett https://github.com/bentrevett/pytorch-seq2seq
• Deep Learning Course by Prof.Sebastian Raschka https://www.youtube.com/watch?v=1nqCZqDYPp0&list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51
• Deep Learning Tutorials by UvA https://uvadlc-notebooks.readthedocs.io/en/latest/index.html
• TSAI-DeepNLP-END2.0 by extensive-nlp https://github.com/extensive-nlp/TSAI-DeepNLP-END2.0/