• Information Retrieval/Extraction/Filtering
• Machine Translation
• Document/Topic Classification/Summarization
• Question Answering
• Text Mining
• Sentiment Analysis
• Speech Recognition
• Machine Writing/Content Genetation

This is to compute the probability of a sentence or sequence of words.

N-gram is a popular statistical language model.
After building a model, we usually use cross-entropy and perplexity to evaluate the model.
Lower perplexities correspond to higher likelihoods, so lower scores are better on this
metric.

A major concern in language modeling is to avoid the situation p(w) = 0, which could arise as a result of a single unseen n-gram, the solution is using smoothing methods, some smoothing methods includes:

• Add-One(Laplace) smoothing
• Good-Turing smoothing
• Kneser-Ney smoothing
• Witten-Bell smoothing

A sentence/document is represented by the counts of distinct terms that occur within it. Additional information, such as word order, POS tag, semantics and syntax etc, are all discarded.

This is an important math theory/algorithm used in NLP tasks.

• Bayesian Network
• Markov Network
• Condition Random Fields
• Hidden Markov Models
• Estimation Maximization
• Max Entropy

• Latent Dirichlet Allocation (LDA): Based on probabilistic graphical models
• LSA: Uses Singular Value Decomposition (SVD) on the Document-Term Matrix. Based on Linear Algebra
• NMF: Non-Negative Matrix Factorization – Based on Linear Algebra

These are some tasks that may not be the solution to any particular NLP problem but are done as pre-requisites to simplify a lot of different problems in NLP. These are pretty much like reading comprehension we learn in school.

Identify Proper nouns, Common nouns, Verbs, Adjectives, Preposition etc.

Identify name of people, location etc.

Previously, there are some other popular distributed representation of word as vectors, like Tf-Idf.
But they are sparse and long which is not computing efficient. Word2Vec instead is a dense vector representation of words(commonly 100-500 dimensions). and it models the meaning of a word as an embedding.

But how to get the dense vectors? Singular value decomposition(Latent Semantic Analysis) can be used, but a more successful way is through neural network inspired learning strategy.

• CBOW: Predict center/target word based on context words
• Skip-grams: Predict context words based on center/target word.

Other vector based models include: fastText, Doc2Vec, GloVe etc.

Stay tuned…