NLP with Streamlit

In this article, we are going to talk about how we can embed some of the functionalities of NLP(Natural Language Processing) like named entity recognition and sentiment analysis, in a streamlit application.

Streamlit

Streamlit is an amazing technology that turns python scripts into shareable web apps in minutes. Streamlit enables developers to easily create beautiful web apps that can create great experiences for the users.

NLP

Natural Language Processing, or NLP for short, is broadly defined as the automatic manipulation of natural language, like speech and text, by software.

NLP is a branch of computer science—and more specifically, the branch of artificial intelligence or AI— is concerned with giving computers the ability to understand the text and spoken words in much the same way human beings can.

We are going to use hugging fees — Transformers to build this NLP-Streamlit application.

Transformers

 Transformers is a Python module that provides general-purpose architectures for Natural Language Processing. So in order to use Transformers, we’ll need to install it.

pip install transformers

Transformers is tested on Python 3.6+, and PyTorch 1.1.0+ or TensorFlow 2.0+. So, we have to also install either one of or both TensorFlow and PyTorch, to work with transformers.

pip install tensorflow
pip install torch

There are numerous tasks inside the NLP, such as:

• Sentiment Analysis
• Name Entity Recognition
• Text Generation and many more.

Let’s understand these tasks and learn how we can embed these tasks into our streamlit application.

Sentiment Analysis

Sentiment analysis — attempts to extract subjective qualities—attitudes, emotions, sarcasm, confusion, suspicion—from the text.

Required Modules

import streamlit as st
from transformers import pipeline

Implementation

First, let’s create a select-box widget in our application, in order to generate a drop-down menu for various NLP tasks in our streamlit application.

option = st.selectbox(
    "Select an Option",
    ["Sentiment Analysis", ]
)

• Create a text area widget for getting the input text — st.text_area().
• Now for Sentiment analysis, we have to create a classifier, a classifier is created using the pipeline method of the transformer module by the NLP task names — pipeline("sentiment-analysis").
• Pass our input text to that classifier — answer = classifier(text).

Classifiers return a label (“POSITIVE” or “NEGATIVE”) alongside a score, the score represents the accuracy.

• Finally, show up the answer returned by the classifier in our application — st.write(answer).

if option == "Sentiment Analysis":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("sentiment-analysis")
        answer = classifier(text)
        st.write(answer)

Output

Note that first time when you run any of the NLP task, it will take a while to show up the output.

Name Entity Recognition

Named entity recognition, or NEM identifies words or phrases as useful entities. NEM identifies ‘USA’ as a location or ‘BOB’ as a man’s name.

Pipeline tries to identify tokens as belonging to one of 9 classes:

For Named entity recognition functionality first, let’s add the name of the task to our drop-down menu in our application.

option = st.selectbox(
    "Select an Option",
    ["Sentiment Analysis", "Name Entity Recognition"]
)

Now the implementation of the NLP tasks and embedding them is exactly similar to as we have seen in Sentiment analysis. Except that you have to create a separate classifier with the new task name.

classifier = pipeline("ner")

Implementation

import streamlit as st
from transformers import pipeline

option = st.selectbox(
    "Select an Option", ["Sentiment Analysis", "Name Entity Recognition"])

if option == "Sentiment Analysis":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("sentiment-analysis")
        answer = classifier(text)
        st.write(answer)

elif option == "Name Entity Recognition":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("ner")
        answer = classifier(text)
        st.write(answer)

Output

Here, I-ORG and I-LOC stand for Organisation and Location, in the output.

Text Generation

Text Generation (a.k.a open-ended text generation) the goal is to create a related portion of text that is a continuation from the given context.

Classifier

pipeline("text-generation")

Implementation

import streamlit as st
from transformers import pipeline

option = st.selectbox(
    "Select an Option", 
    ["Sentiment Analysis", 
    "Name Entity Recognition", 
    "Text generation"])

if option == "Sentiment Analysis":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("sentiment-analysis")
        answer = classifier(text)
        st.write(answer)

elif option == "Name Entity Recognition":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("ner")
        answer = classifier(text)
        st.write(answer)

elif option == "Text generation":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("text-generation")
        answer = classifier(text)
        st.write(answer)

Notice only the argument passed to the pipeline method is getting changed for different NLP task.

Extractive Question Answering

Extractive Question Answering is the task of extracting an answer from a text given a question.

Classifier

pipeline("question-answering")

Implementation

text = st.text_area(label="Enter Context")
ques = st.text_area(label="Enter Question")
if text and ques: 
    classifier = pipeline("question-answering")
    answer = classifier(question=ques, context=text)
    st.write(answer)

Output

Summarization

Summarization is the task of summarizing a document or an article into a shorter text.

Classifier

pipeline("summarization")

Implementation

• For summarization, we can mention the maximum and the minimum number of words that should be present in the summary, although they both are optional arguments.

text = st.text_area(label="Enter text")
if text: 
   classifier = pipeline("summarization")
   answer = classifier(text, max_length=200, min_length = 10)
   st.write(answer)

Output

Translation

The translation is the task of translating a text from one language to another. 

Classifier

pipeline("translation_en_to_fr")

“en” and “fr” are shortcodes for English and French languages respectively.

Implementation

text = st.text_area(label="Enter text")
if text: 
    classifier = pipeline("translation_en_to_fr")
    answer = classifier(text)
    st.write(answer)

Output

Complete Code

import streamlit as st
from transformers import pipeline

option = st.selectbox(
    "Select an Option", 
    ["Sentiment Analysis", 
    "Name Entity Recognition", 
    "Text generation",
    "Question Answering",
    "Summarization",
    "Translation"])

if option == "Sentiment Analysis":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("sentiment-analysis")
        answer = classifier(text)
        st.write(answer)

elif option == "Name Entity Recognition":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("ner")
        answer = classifier(text)
        st.write(answer)

elif option == "Text generation":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("text-generation")
        answer = classifier(text)
        st.write(answer)

elif option == "Question Answering":
    text = st.text_area(label="Enter Context")
    ques = st.text_area(label="Enter Question")
    if text and ques: 
        classifier = pipeline("question-answering")
        answer = classifier(question=ques, context=text)
        st.write(answer)

elif option == "Summarization":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("summarization")
        answer = classifier(text, max_length=200, min_length = 10)
        st.write(answer)

elif option == "Translation":
    text = st.text_area(label="Enter text")
    if text: 
        classifier = pipeline("translation_en_to_fr")
        answer = classifier(text)
        st.write(answer)

Conclusion

In the article, we have discussed how we can perform the NLP tasks and embed those tasks into our streamlit application. We took various NLP tasks into consideration, the only difference was creating the classifier for the tasks.