One of the projects I want to carry out this year is the development of a documentation-based chatbot to help readers find the information they need, without having to click on several search results (sometimes inaccurate) provided by search engines. To this end, I’ve been reading different articles to learn more about existing technologies and to assess which one fits best to the content I write.

During the “100 days of code” challenge I took up last year, I also learnt about the history of chatbots. It turns out, chatbots are not really that modern (at least, the concept). The first chatbot, ELIZA, was created in 1966 by Weizenbaum. ELIZA impersonated a psychotherapist whose treatment was, let’s say, not very effective 😊. To understand the logic behind it, I tried to create my own psychotherapist bot some months ago (I called it Freud, of course). If you’re interested, you can check it out here: https://github.com/marinahgtech/BotTherapist

Chatbots make interactions with content more human, since the user accesses the content by answering the bot’s questions. The information displayed is determined by the answer provided. Chatbots also leverage the daily work of other stakeholders, like support professionals and consultants, especially when they have to work with complex products. The chatbot helps them find the right content faster, which saves time.

Chatbots usually make use of retrieval models. To put it simply, these models consist of predefined pairs of questions and answers. Some companies have been introducing NLP and deep learning technologies for some years now to offer and almost human interaction between the user and the chatbot (the idea is that, at some point, humans cannot tell if they’re talking to a person or a machine). This is a very ambitious approach, that would be nice to have for us in the near feature. But first, I want to focus on building a basic retrieval model. I’m fascinated about natural language processing anyway, and learning more about it is on my bucket list. I’m starting a Python course as a preparation for an NLP course I want to take this summer.

Turning to the matter in hand: retrieval-based models use a repository of q&a pairs and allow to choose the right answer based on the human’s input. This is all based on heuristics, the function that determines which answer should be retrieved. Machine learning classifiers could be also used to pick up the right answer, but for the sake of simplicity (and feasibility), I’ll start with the simplest model first. This includes short conversations and a closed domain (which, by the way, fits perfectly with the customer needs to find technical information for a certain product).

The retrieval-based model has some limitations, the most important one being the fact that the number of predefined responses is very reduced, but it’s a good place to start for newbies 🙂

I’ll keep you posted on my progress…