This repo contains all the basic techniques that are to be used in any NLP task.

It includes:

1. Data Pre-processing:

   • Removing unwanted charcacters using Regex or similar technique
     • Removing HTML tags, Unicode and other symbols, numbers, & Links
   • Tokenization
   • Changing case of all words
   • Text Normalization
     • Stemming
     • Lemmatization
   • Stop word removal

2. Embeddings and Representations

   • 1. Bag of Words (BOW)
   • 2. Term Frequency-Inverse Document Frequency (TF-IDF)
   • 3. Word2Vec
   • 4. GloVe

3. Modelling Techniques

   • Classical ML Algorithms like Naive Bayes
   • State of the art models like LSTM, BERT, GPT