Will ChatGPT Save the Chatbot Industry? (Part II)

In part one of this two part series, I reviewed the history of the chatbot, my 2003 patent, and the reasons why the conditions weren’t right for the type of chat experience we’re all now enjoying with ChatGPT. For part two, we get into what has changed and the different ways enterprises can drive modern chatbot experiences with ChatGPT.

The Rise of ChatGPT

From the very first moments that OpenAI’s ChatGPT burst on to the scene, commentators have gushed at it’s magical brilliance. The ability to ask questions and receive a comprehensive response, that can be further refined in an ongoing dialog, appears mercurial. Everyone seems both captivated and frightened, but we’re all wondering how we can use it in our business. However, I think it’s a powerful evolution of the humble chatbot that removes many of the adoption barriers I formerly mentioned in Part I.

1. The interaction is effortlessly natural. You just type or dictate your question.

2. There are no predefined commands to learn. It’s just remarkably intuitive.

3. The hesitancy of the unknown is eliminated because users instinctively expect the conversation to iterate until an answer is found. We don’t feel inhibited, but rather the opposite. We want to use ChatGPT.

4. ChatGPT sparks the imagination of how we could use it to our benefit. Enterprise Buddy in comparison felt quite utilitarian.

5. Large Language Models (LLMs) are rapidly evolving to incorporate rich formatting and media, and we are no longer restricted to text answers.

6. Technological advancement in data storage, processing and programming for AI has progressed so much in 20 years that the market is ripe for widespread adoption beyond the OpenAI offerings.

In fact, ex-Google Matt Web posted this quote on his blog:

“It feels pretty likely that prompting or chatting with AI agents is going to be a major way that we interact with computers into the future, and whereas there’s not a huge spread in the ability between people who are not super good at tapping on icons on their smartphones and people who are, when it comes to working with AI it seems like we’ll have a high dynamic range. Prompting opens the door for non-technical virtuosos in a way that we haven’t seen with modern computers, outside of maybe Excel.”

I whole-heartedly agree!

The Arms Race is On

The AI arms race is definitely in full swing, and we know that if we can’t work out how to leverage this new technology for our benefit, then our competitors will. Consequently, the obvious questions are “How?” and “Why?” I’ve listed three below:

1. Off the Shelf LLM

The low hanging fruit is to subscribe to one of the many public LLM services and deploy Generative AI chatbots across the organization as productivity helper to the hundreds of content creators across an organization. The risk is low, and the rewards are potentially great. Content creators will be generating emails, web pages, blogs, code aplenty.

2. Extend LLM With Own Data

The natural next step is to extend the concept to include an organizations’ OWN data. An organization could either host the infrastructure for LLM’s themselves or augment an existing service as the high-level diagram below highlights:

But that’s just half of the solution. A chatbot mechanism is then needed to access the LLM and return responses based on the newly trained data. The high-level diagram below demonstrates the overall concept:

Enhancing an existing model requires more consideration, because there’s not only the question of whether a company should host its own Generative AI infrastructure, but also surfaces issues of data quality and data security. For example, should all employees have access to a GenAI chatbot if the organization’s LLM is trained on corporate code-base, documentation, usage telemetry, HR information, sales data, etc.?

A singular model deployed in this manner could generate answers containing sensitive data to folks not entitled to view it. Consequently, someone in marketing could ask questions about the company’s proprietary source code. Or the same person could ask questions about salaries from HR data. Also, chatbots like Slack and Teams allow group chats, mixing users with different privileges and access. Again, this complicates delivery of a singular Generative AI model. My original chatbot patent had an access and security layer built into the chatbot flow. Therefore, I’d hazard a guess that many corporations and vendors will enforce some sort of question-response access policy as the technology develops and matures.

One other challenge is update methodology. Should the knowledge warehouse be append only, or should old information be deleted, especially as selective pruning of vector databases can be tricky. For example, how do you treat outdated documentation? Do you delete it from the model or just append the new documentation and label it as current? On the one hand having old and new documentation allows for comparative questions to be answered, however answers to general questions based on outdated documentation could be confusing. Perhaps a more specific prompt will lessen this issue. I’ll leave the “prompt engineering debate” to another posting.

3. Embedding Generative AI as Co-pilots

The final place where organizations can quickly benefit from Generative AI is to embed it into their own products as “helpers” or “co-pilots.” Once again, this type of technology is not an original idea. Integratged Development Environments (IDEs) have provided code completion, error detection and refactoring for years, but recently GitHub Copilot took these concepts further by utilizing a large-scale machine learning model to generate swathes of code as a productivity enhancer. Now you can do something similar.

Creating a co-pilot is similar to use case #2 except you train the model on your own code base or business processes and the user interface is a more targeted kind of chatbot. For instance, a social media posting application could provide a ‘co-pilot’ to not only generate interesting posts, but also automate postings at the most optimal time.

Are Chatbots Saved?

I began this post asking the question whether OpenAI and ChatGPT will save the chatbot industry and I believe this is a resounding YES! You could argue that chatbots didn’t need saving and were doing just fine. But the counterpoint is that we’ve not seen the widespread adoption as comparable innovations in the same period. That was until now. ChatGPT has shaken the IT industry to its very foundation and sparked innovation on a scale that we’ve not seen since the invention of the microchip. Chatting with AI agents is going to be a dominate method of interacting with software in the very near future, especially as every vendor races to blend and extend Gen AI into their solutions over the coming six to twelve months.

ChatGPT is natural evolution of the initial promise of the enterprise Chatbot, but it's also so much more.