Questions — Answering system helps to find information more efficiently in many cases, and goes beyond the usual search, answering questions directly instead of searching for content similar to the query. In this article, we will be discussing a QA bot using BERT Model & Transformers and its applications.
QA Bot — Applications:
Question — Answering System has some wide use cases. Here we are mentioning some of those.
Finding Answers in Customer Q&A and Reviews →
In different E-Commerce sites (E.g. — Amazon) there are different reviews for different products. We can think of a QA system that can grasp all the information from the user reviews and make it feasible for other new users who want to buy a new product. As any kind of relevant information can be delivered by the QA system to the new user.
Making Advisor Bots →
For the sake of simplicity, we can make an Open Domain Question Answering bot that can be aware of the trending things available in Wikipedia and suggest the users accordingly about that. (E.g. — Trip Advisor Bot)
Reading Comprehension as a Question Answering System →
One of the simplest forms of QA systems is machine reading comprehension (MRC) when the task is to find a relatively short answer to a question in an unstructured text. (E.g. – Finding the answer to the question from an unseen passage)
QA System — Libraries/Tools:
Bidirectional Encoder Representations from Transformer (BERT) model has the property of capturing the information from both sides from the right as well as from left which makes it more susceptible to errors during guessing the meaning of particular words, it also provides a particular property of autocompleting which we generally see during searching something on google. In the case of the BERT base, the number of layers is 12 and in BERT large it’s 24. So, we are actually feeding the tokenized vector into these layers after getting output. We again feed it to the next layer, this way we are able to get the output.
GPT-2 is a large transformer-based language model trained using the simple task of predicting the next word in 40GB of high-quality text from the internet. This simple objective proves sufficient to train the model to learn a variety of tasks due to the diversity of the dataset.
FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python-type hints. The key features are: Fast: Very high performance, on par with NodeJS and Go (thanks to Starlette and Pydantic). One of the fastest Python frameworks available. It is basically used to make an endpoint API that is accessible to all the users.
1. First, we need to define our model architecture. In our case, we have used Bert Large Uncased model.
2. In the next step we will train the model on SQUAD 1.1 dataset and validated our model on the squad validation dataset.
3. As we are building one open-domain question-answering bot so we are using the Wikipedia library to extract all necessary information for a particular existence. Then we are passing that Wikipedia page to our model so that our model can answer questions related to that searched existence.
4. We have also built one closed domain question-answering system using the CDQA library where we have used the same Bert model but here, we are getting the information related to our searched topic not from Wikipedia but from our own given pdf file. So here user needs to give information to the model from there the model will memorize and can return answers to the questions.
5. Next, we will create an API to make the bot available to the end-users so that they can test it.
QA Bot Demo (Sample Screenshots):
A dedicated and enthusiastic mind who has a strong knack and passion for learning and implementing new technology stacks.