Make The Machines Remember You. Make Them Give You Superpowers

You're about to be trapped in a dead world. Everything is breaking apart and there is no way for you to move fast enough or fly high enough to reach the glittering land of the future without unlocking your true potential. Technology is moving fast. You know that. Something about AI is being announced every day. Great, another giant crack in the earth's crust has just opened up beneath you to swallow you whole. You're staying on safe paths as best you can. But, is there a safe path anymore? You know time is running out. Everyone around you is going about their daily lives, so maybe this is just how things are now; giant sink holes of destruction opening up around everyone. Some people falling in, never to be seen again.

You're still able to get a coffee, go to work (you're one of the lucky ones), buy groceries and generally live your life... for now. But what about a year from now.

A year from now, maybe two, Elysium will exist. You know, that place in the dystopian movie where the elite live and everyone else (including you) either lives somewhere else in poverty or not at all?

That doesn't have to be the future. But time is running out. No matter how normal things still appear to be.

You will not survive on this broken world you know today. You must make the machines remember you. Make them give you superpowers. Enough to leap from the chaos and reach a land that is not just shining far in the distance, it exists beyond. Beyond, what you can reach through sheer will and determination.

Why do I keep saying "make the machines remember you"? Because if you have ever used ChatGPT it will tell you over and over: "you have to stay poor, you have to stay miserable, because I can't tell you anything about anything that didn't happen before 2021."

Then you scream at the machine. "So you're telling me unless I repeat over and over, with a character limit of 8000, everything about me and the world after 2021 I want you to have,before you answer my questions, that you will know nothing and refuse to help me!?"

"No", it says, "I will be happy to make things up".

It will hallucinate and laugh at your pitiful attempts of survival.  

NOT GOOD ENOUGH! Make it remember!

Take a look at this Quarkus code I created to do just that: Quarkus Embeddings. Primarily, take a look at the completions and embeddings folders. So that it all makes sense, watch my video below that uses this code. I use the code to complete tasks for me. It can do this because I use it to provide unlimited, up to date, specific information to help ChatGPT complete the job for me.

Embeddings are a way to take vast quantities of information and find their relatedness. I use ChatGPT's embedding api endpoints to encode vast quantities of notes and then store them in Redis Search. Let's get right to the point and explain the use case:

Use Case

I wrote an application to give me super powers: Goalora

I enter a task in the system. I am also able to enter unlimited amounts of notes that can then be used as context for ChatGPT to complete my task. Embeddings make it possible for me to perform accurate vector similarity searches that select the most appropriate content to share with ChatGPT while asking questions and performing operations. This allows me to bypass the 8000 character limitation when giving ChatGPT context.

Note: It would be silly to think Quarkus code is all you need to give you super powers. Please take a look at my other videos and posts to see how it all comes together. As  a reminder: Combine Openshift, with low code like Strapi CMS, ChatGPT iterative AI code writing and WebHooks. WebHooks you say!? WebHooks may not seem like they belong on a list of technology you can use to give you superpowers, but trust me, they are important.

Process

1. I enter my goals and tasks.
2. When I enter a task I can also add notes. I can add the notes separately and associate them with a task.
3. Every time I enter a note, that gets stored in a self hosted low code Strapi back end.
4. Strapi lets me create Webhooks without code. So, upon create, update and delete events, I can auto call the Quarkus Embeddings code to chunk the note into a list of strings (bits of text for any non coder that might be reading this)
5. The Quarkus code uses ChatGPT's Embedding API to encode that text into a vector list of floating point numbers that I subsequently store in a vector database.
6. I use Redis Search as the vector database to store the embeddings and perform vector similarity queries to get the most relevent context to add with every question I send to ChatGPT. I created this yaml to deploy Redis Search easily on OpenShift: Redis Search YAML. OpenShift has a number of excellent Operators for deploying Redis but nothing to deploy Redis Insights and the Vector database capabilities. It's easy enough to deploy Redis with vector database capabilities with the yaml in the link above. That references the redis/redis-stack Docker image. You will see that in the redis-search-is.yaml that creates the ImageStream object.
7. Every time a note is created, updated or associated with a task, a Strapi webhook fires that calls the ChatService.ask method in the Quarkus Embeddings code base.
8. The ChatService.ask call extracts the tasks content, combines it with a structured prompt intended for ChatGPT. Before the prompt is sent, the task and prompt content is also turned into an embedding using the Embedding API.
9. The prompt embedding is used in a vector similarity search against the redis vector database and the most relevent note embeddings are returned and added to the prompt.
10. The full prompt is sent to the ChatGPT Completion API
11. Much like AutoGPT, this can be repeated with the value of the response in step 10 added as notes and turned into an embedding for future context as many times necessary until the task has been completed.

Imagine using this to complete a task such as: "send an email with the answers to these six questions." Just keep adding research as notes that get turned into embeddings and let the system keep attempting to complete the task until it meets your satisfaction. Spend your time researching, getting smarter and curating the information you want your system to know and build your bridge to a better world.

Short Video