AI Assistant for Magento 2 - F(r)iction or Reality

 

Artificial Intelligence (AI) is not a fiction anymore, it is now a tool, a technology that has infiltrated in our everyday lives and possibly used on a daily basis by more and more people. It is likely that almost all of us have used some sort of AI-Driven technology so far, it could be help received from generative AI (ChatGPT, Gemini etc), or an AI-Powered tool such as standard Google search, AI IDE such as Cursor or Copilot in Microsoft products that we use every day.

Without a doubt AI is growing with tremendous rates and possibilities are virtually endless.

As a development company, we're excited to explore how AI technologies can be applied in real business. We've already seen AI based content generators that can help e-commerce shop owners with automated content population, syntax correction, search engine optimization and much more. 

However from a business owner perspective, the real question is:

Can AI help me make more money?
Can AI help me sell more?
Can AI reduce support operations?

 

Cloud vs Local LLM

Most small to medium business do not own AI infrastucture, nor host any local LLM models, let alone trained models. Almost 90% of business have the only option to implement an AI-Driven process using cloud based service such as OpenAI API. OpenAI is the company behind ChatGPT, a large language model (LLM) specifically trained on large sets of text which responds in a human-like manner.

 

Expectations

With the word AI-Driven technology we all have some up-front expectations, with the word AI Assistant, the expectations are even more. The first thing we'd expect from an AI assistant is to act like or close to real human that works in an offline store. We usually ask if they have particular product or if they have a product that can best suit our goals. Till now, online shoppers can use standard keyword based search or filter based search e.g. type a keyword or select multiple options which works perfect and usually provides very accurate results, however it is impossible to express intent and here AI technology can come into handy.

 

Let's discuss some of the possibilities that are now available with AI technologies

 

Semantic search

AI technologies provide new options for implementing semantic search. A semantic search is a special technology that can interpret the meaning of the words and phrases. It is designed to understand intent and context. This can be applied to any content such as product description, search query and more. The concept here is that each text or phrase and can be represented as vectors or so called embeddings. Vectors are arrays of numeric values calculated by the LLM (large language model) based on different factors such as combination of words, relatedness, context and intent. Having texts represented as mathematical vectors allows developers to calculate the similarity between vectors and therefore calculate how close are two texts in terms of meaning. This can be best explained visually in the following diagram. A vector is a mathematical object that has both magnitude (length) and direction. As directed object, by converting texts into vectors, it is possible to compute how identical 2 vectors are by checking if they are both directed in the same direction and how small the angle is between them (cosine similarity)

 

 

Now this opens a huge set of possibilities. What if we have all our product descriptions converted into vectors? And then what if we convert what user is searching for into vector and compare this vector with all our product vectors?

This would in effect result into semantic search, allowing the engine to understand the meaning of our search, then to understand the meaning of product descriptions, compare them and return most similar products. This could in effect return products that do not match exactly to searched query but are semantically related and still be what the user might be looking for.

 

Vector and Embeddings Misconception

Here I'd like to explain some major misconception that often developers are having and it could explain some wierd results when using semantic search. Embeddings/Vectors are not fixed, they depend on the semantic concept. For example, a fox and a rabbit may be semantically close due to both being animals, however if you read a book for a rabbit that likes eating carrots, then the rabit and the carrot may be closer semantically compared to rabbit and a fox outside the context. Another misconception is about accuracy, similarity is determined by calculating meaning, however if you're comparing a keyword and want to find products that match to this keyword, semantic search may not produce accurate results because it's harded for the model to determine intent.

 

Reasoning and RAG

An AI assistant should not only suggest products that may fit user goals but it should be able to tell why they have returned those products. For example, one might be looking for a miter saw that can cut through 7 inch beam. The semantic search may return multiple results related to available miter saws but not all of them may be able to cut 7 inch beam into single cut. The AI assistant should be able to distinguish miter saws that can do the job and show them first as most relative.

To achieve this, we can use a concept called retrieval augmented generation or RAG. The idea here is that we can find a small set of semantically related products from our entire product range, then we can feed an LLM model with this set and ask it to suggest best products that match user's query. The key point here is that we feed a small set of products instead of the entire range, which optimizes performance, reduces AI hallucinations and provides accurate results. The LLM can parse small set of data and augment the data in a way that it best matches what user is asking for.

 

Performance

We all hate to wait for our order in the restaurant and an AI assistant should be no exception. What we expect from an AI assistant is to be fast, accurate and helpful as much as possible. Cloud based solutions are definitely a challenge in terms of performance. OpenAI API could be slow and sometimes take approx. ~ 10 to 20 seconds to complete a request. In Magento 2 however, we have lots of tools and options already available at our disposal which makes it perfect for optimizing the process.

1. Ultra fast search

We can actually achieve ultra fast semantic search by storing product vectors locally into already available database that supports vector types such as Elasticseach 8 or Opensearch. These engines already used natively by Magento 2, allows us to store vector data into designated index and retrieve this data very fast. This allows our AI assistant to show results quickly which is expected by the customer. Only when customer is interested in specific product, we can interact with the slower Cloud based LLM. This should not bother customers as they naturally expect the reasoning answer to not be instant.

2. Intent detection

It is possible to use an LLM model to detect intent. We can ask the LLM to let us know what user's query is about, is it a general question, or an FAQ request or even an action request. The AI assistant can then perform different operation and since it run locally, it can actually interact with Magento's interface giving it ability to add products to cart, generate AI driven discounts and much more. Possibilities are endless.

 

Dependency

Being dependent on cloud-based LLMs has its disadvantages. To search and perform tasks in real time, the AI assistant must constantly communicate with the OpenAI API. This introduces the possibility of outages and can negatively impact performance and response speed.

It is important that the AI assistant clearly communicates to the end user that it is an AI system and that it may occasionally provide incorrect responses—or fail to respond at all. When customers are aware that they are interacting with AI technology, they are more likely to tolerate minor performance issues and occasional inconsistencies.

 

Privacy & Security Concerns

Many are concerned about privacy and security regarding AI however it's really down to what information is accessible to AI agent. To be helpful and act as an AI assistant, the model has to have access to your product data such as product name, price, description etc. but this information is already public, it can be accessed by anyone just browsing your site. Just like in a real offline store, to get a help from a sales person, they have to know the product range, they have to know what the store is selling, where are different types of products located in the store etc.

An AI assistant however should not have access to sensitive information such as personal name, email, basically any PII (personally identifiable information). If you are aming about AI assistant that can track orders, or has access to order information, this should be done only using you own local LLM and cloud based solutions avoided at all costs. It is possible to instruct an cloud LLM model to not save any personal information or not learn from it but it's still a blackbox and one should not realy on this.

 

Conclusion

With all the technologies available now, having a real AI assistant for Magento 2 is no longer a fiction, it's reality. We have proved this with a real production ready AI Assistant for Magento 2 already used by customers.

 

MEET AI BUDDY

We believe that best test is the test of time and until a real product is applied and tested in a production environment, it would be impossible to predict everything. And this is how AI Buddy was born. AI Buddy is our AI-Driven Sales and Marketing Assistant for Magento 2, it's an extension that you can install, configure and run within few minutes

 

Find more about AI Buddy