Retrieval Augmented Generation with Twilio and ChatGPT

K-Optional is one of the leading experts in applying retrieval augmented generation (RAG) to services on Twilio. With Twilio, you can hold automated conversations with your customers and users. And with RAG, you can harness powerful large language models (LLMs) like ChatGPT to derive thoughtful responses from your own data.

What is retrieval augmented generation (RAG)

LLMs make it easy for a machine to hold a human conversation. But since they only “know” what’s in their training data, they won’t provide info that’s relevant to your business. That’s where RAG comes in: it aims to combine the fluency and genius of ChatGPT with the contextual knowledge of your data and process.

RAG is both an art and a science and K-Optional has mastered how to tease out useful data, encode it, and skillfully feed it to an LLM.

Customer support software

When it comes to customer support, messaging seems to have usurped voice. Whether that reflects users’ growing comfort behind the chat interface or simply better economics, you don’t need to look far to see enterprises embrace messaging:

On Delta Air Lines’ help page you will find a support chat but not their phone number
Other airlines like Spirit also provide a WhatsApp support function. By the way, the Twilio SDK has WhatsApp messaging functions.
The tech company Podium helps people replace website contact forms with a mere SMS chat line — such a basic channel and it works well!

Simply put, messaging with Twilio is a great way to provide customer support. And with RAG, you can further automate that support.

Some of our K-Optional’s work with Twilio and RAG

A RAG customer support pipeline for last mile delivery

One of our clients, Getcho, builds courier technology that helps businesses deliver products and documents within an hour.

Customer support is a huge part part of this business because there are so many variables: pick-up points, drop-off points, item sizes, store hours, ETAs, scheduled times and traffic to name a few. Getcho staffs its Twilio support chat with operators who text drivers and customers. As the service has grown, it has become harder for personnel to keep up with all of the orders.

Using Getcho’s job documentation and over 10,000 support conversations, we built a powerful RAG pipeline that now handles much of support. It starts by grouping all chats by their vector embedding. These groups grow each day with the diversity of new chats. Based on an incoming message’s group (or lackthereof), our code locates a response sequence which it supplies to a dynamic GPT prompt.

Since we shipped this software, Getcho has seen three trends: