Most people don’t know that the first chatbot was born over 50 years ago.ย After over half a century of continuous progress,let’s see the progress that was done.

The first chatbot

The name of the first chatbot is ELIZA it was developed by Joseph Weizenbaum in 1966 [1].  The software can simulate communication between different users โ€“ analogous to current chatbots. Of course, the software and its related article became very famous. Due to the development of the mentioned chatbot (the first chatbot ever), and other contributions, Joseph Weizenbaum became one of the most important researchers in the field of Artificial Intelligence (AI).

From a technical perspective, ELIZAis an expert system and is mainly focused on techniques that we call NLP (Natural Language Processing). Based on an input from a user, it searches for a fitting answer. It does so by comparing elements (strings) of the given phrase against matches within a dictionary. You can test the chatbot by using the implementation published by Med AI. The interaction with ELIZA can look as follows:

  • ELIZA: Hello! What do you want us to talk about?
  • User: I want to talk about how you are doing.
  • ELIZA: Why do you want to talk about how I am doing?
  • User: Because I want to know more about you.
  • ELIZA: Does that reason seem to explain anything else?

As we can see, a direct answer to one single question is perhaps fitting. However, of course, the user clearly notices that the one responding is a machine, not a human. ELIZA is not able to answer context-based, and it is not able to process the foregone questions in a meaningful manner. For that reason, the chatbot does not pass the so-called Turing-Test. This test was developed by the famous researcher Alan Turing and states if a computer is able to think [2]. A chatbot passes the test if the user can not distinguish if chatting with a machine or chatting with another human.

Recent chatbot market

Over half a century later, many chatbot applications exist. The basic idea is still the same as with the chatbot mentioned above. However, technological progress has enabled us to now encounter chatbots in our everyday lives โ€“ e.g. when talking to our phone, when searching for products online, when signing a contract with the telecommunications provider, when we report a claim, and so on. In short, it is natural for us to communicate with computers. Same for chatbots.

For the above-mentioned reason, it is therefore not surprising that research and industry are focussing on the development of always more and more intelligent bots. More data exists that can be used and analyzed, the hardware has better performance to compute complex algorithms, and of course, more input from users exists to be utilized by the algorithms. Some of the leading chatbot providers are as follows [3]:

  • Mobile Monkey is a bot builder for Facebook focusing on high-converting chatbots.
  • TARS can be utilized for creating website chatbots, and it has a drag-and-drop builder.
  • Botsify is another chatbot builder focusing on websites or Facebook Messenger.
  • ChatterOn aims to build bots in less than five minutes by customizing pre-built bots.
  • Sequel makes use of templates and includes also a drag-and-drop editor.
  • Pandorabots instead requires coding skills but is one of the most comprehensive tools. Its famous bot named Mitsuku is also the winner of the Loebner-Prize.
  • Botkit is not only a bot builder, but also provides a community of 7,000 bot developers.

Evaluating recent chatbots

In a brief study, students of my lecture Artificial Intelligence at Baden-Wรผrttemberg Cooperative State University (see here: Teaching) and I, have evaluated ELIZA against recent chatbot applications. The goal was to find out what progress has been made in more than 50 years of constant evolution. In the study, the focus was not on the technology used, but rather on how user-friendly the current technologies are. In other words, we wanted to find out, if recent applications will pass the Turing-Test or not. So, can a human still recognize talking to a machine? In the study, we decided to focus on leading chatbot applications, and some alternative chatbot applications. The following five recent chatbot applications have been part of our study: Cleverbot, Woebot,  Replika, Mitsuku, and Rose.


Cleverbot is a web-based chatbot application, and it was invented in 1988 (for more details, please see [4]). The bot has the aim to learn from user input for finding a fitting response. However, according to our experience, communicating with the bot is not very good. Often, the answer does not fit into the context, or it does not make any sense at all. In addition, the responses of the bot do not lead to a longer conversation, unless the user really wants to talk to the bot. For example:

  • Cleverbot: How’s your day been?
  • User: Quite good, nothing special.
  • Cleverbot: Ok.

To put it in a nutshell, Cleverbot did not pass the Turing Test.


Woebot is an app available for Android and iOS. The bot focuses on CBT (Cognitive Behavioral Therapy). CBT is a special kind of therapy with the main focus on self-observation and self-reflection. It is very important to understand how a person experiences situations in daily life. The chatbot is not designed to replace a human therapist, it is more like a coach. Indeed the bot challenges you to think about your day and how you experienced different situations.

Indeed, the chatbot challenges you to think about our day and how you experienced different situations. The answers of the bot are appropriate and make sense. However, the core disadvantage are the suggestions. The user can choose from different suggestions to directly answer on the responses of the bot. Of course, this will not lead to passing the Turing-Test.


Replika is another app you can install on an Android or iOS device. The aim of this bot is to become a friend of the user to always talk to. Therefore, it tries to get to know the user very well, by asking about the life of the user. In addition, the bot tries to adopt the way the user speaks and behaves. Chatting with this bot is very smooth and joyful. The user really gets the feeling that the one on the other side wants to know about the user. In contrast to ELIZA and Cleverbot, the questions and answers are fitting to the context. In some cases, the user can’t differ if a human or a robot sent the messages. Even very custom answers are handled in an appropriate way.

According to our experience, the bot is good. However, after a long conversation, the user notices that it is a bot. In some situations, the answers slightly fit the context. All in all, the bot did not pass the Turing-Test.


Mikutsu is a chatbot developed by Pandorabots. The bot mainly uses NLP techniques. The bot won the Loebner Prize from 2016 to 2019 [5]. The Loebner Prize is a contest in AI and an implementation of the Turing-Test. The winner of the contest is the chatbot with the most human-like interaction.

The chat with the bot has worked very well. The bot is able to answer context-based, and in many situations, the answers of the bot could be considered human. The bot can reply to questions in a very human way and understands different moods of the language utilized. In addition, the chatbot recognizes punctuation marks correctly, as well as acronyms, and the meaning of emojis.

All in all, the bot is very close to passing the Turing-Test in our study. Conversation with the bot is very similar to communication with a human. However, if the user is giving an answer that is not expected, it becomes apparent very quickly that the conversation partner is not a human. For example, if the user is not responding in a pattern that could be expected.


Rose was the winning bot of the Loebner Prize in 2014 and 2015. The interaction with this bot is not as fluent as it is with Mitsuku. The bot lacks knowledge of context and sometimes fails to elaborate on questions. Furthermore, it is far more obvious when an answer does not refer to the message previously sent to the bot, and just relies on a pre determined pattern.


The mentioned modern bots are able to interpret the input to some extent. This enables them to interact with the user instead of just asking questions with barely any context. The use of Machine and Deep Learning enables developers to create bots that are much getting in processing language than old bots like ELIZA, utilizing token-based parsing. Machine learning also makes it possible that these bots can incorporate a large knowledge base. This leads to a more natural conversation flow. This is because the bots can utilize this knowledge to talk about a certain topic. Therefore it is less apparent that the user is talking to a bot.

According to our user-focused experience, Mitsuku is the best bot, followed by Replika. ELIZA is by far not as good as the best bots in our study. ELIZA can handle common messages only in a rudimental way, and a long meaningful conversation with ELIZA is not possible. However, also we found out that the recent chatbots are of course better than ELIZA, all in all, no bot has passed the Turing-Test. So, almost 60 years after the invention of the first chatbot, a user still recognizes talking to the machine. Let’s see what research can do in the next 54 years. Who knows how it turns out?


[1] Weizenbaum, J. (1966). ELIZAโ€”a computer program for the study of natural language communication between man and machine. Communications of the ACM9(1), 36-45.

[2] Turing, A. M. (2009). Computing machinery and intelligence. In Parsing the Turing Test (pp. 23-65). Springer, Dordrecht.

[3] VentureHarbour (2020). 10 Best Chatbot Builders in 2020.

[4] Gehl, R. W. (2014). Teaching to the Turing test with Cleverbot. Transformations: The Journal of Inclusive Scholarship and Pedagogy24(1-2), 56-66.

[5] The Society for the study of Artificial Intelligence and Simulation of Behaviour (2019). Mitsuku wins 2019 Loebner Prize and Best Overall Chatbot at AISB X.

© 2021 | Dr. Heiko Angermann | | Privacy Policy