Chatbots, NLP and Wit.aiBy Pranav Sricharan
December 19, 2018
Chatbots have become quite popular lately and they’ve found their place in almost every place possible. Most of us would’ve come across chatbots in Telegram, Facebook or Slack. They’re also found in customer care sections of many websites. But how do these chatbots work?
In earlier days, chatbots worked by matching the user’s text with a definite set of patterns that they stored in files or a database. One such famous chatbot was A.L.I.C.E., which was written using a language called AIML (Artificial Intelligence Markup Language). AIML used an XML-like syntax and a set of files containing the bot’s knowledge (i.e) patterns and responses. Topic wise knowledge was stored in separate AIML files.
The AIML set of A.L.I.C.E. was made available to the public. This led to the creation of many Alicebot clones. AIML interpreters are available for many common languages. Pandorabots.com is a great place to start writing your own chatbots using AIML. A commonly used AIML chatbot is Natasha from Hike Messenger.
Natural Language Processing
Technology has taken giant leaps forward and has reached the point where it can understand sentences and identify feelings, topics and even sarcasm. The background behind this extensive knowledge about human sentences is Natural Language Processing (NLP). Powerful Machine Learning systems are fed with numerous sentences and the meaning those sentences make. After learning, these systems produce a result such as this:
Why am I talking about all this stuff if it requires powerful machines and almost impossible to do it ourselves?
Actually, it is possible to do it ourselves using APIs such as Wit.ai or dialogue flow. These services do all the hard part themselves and give us just the information that we require to make sense of the sentence. I’ve been trying my luck with Wit.ai, creating a personal assistant called Definitely not JARVIS, and I’ll share my experience so far.
Wit.ai is as simple as it can get. Backed by Facebook, it is highly efficient and produces just what we expect. Wit.ai allows it’s users to create any number of bots (both public and private) from scratch. The user may also build their bot using other user’s bots.
Wit.ai learns by examples. We give our bot a sentence. The bot tries to make some sense out of the sentence. If it can’t understand it, we can highlight specific portions of the text and tell what it is. We can also tell Wit what the sentence means as a whole. Wit understands the sentences based on what it calls entities. When Wit tries to understand a sentence, it tries to identify the entities in the sentence along with a confidence value. This can be useful when performing actions based on the sentence.
On the whole, Wit is fast, learns quickly, very easy to use. You could start building your own bot within minutes. Wit also makes a log of what other users of your app has requested and the response it had generated to be evaluated by us, to help improve your bot’s understanding. There are some issues with teaching Wit how to understand short forms like cya, or using multiple entities on the same part of the text. But it is definitely the best way to get started with building your own bot. I’ll be writing a post on how to build your own chatbot using Wit.ai with much detail soon. Till then try out Wit for yourselves.