Abstracting Vectorization to Jump Start AI and RAG Solutions

Amid the news and buzz surrounding generative AI, you’ve likely heard the term retrieval augmented generation (RAG). RAG-enabled solutions allow a generative AI model to retrieve data normally unavailable in its general knowledge and help stem hallucinations. This is useful when data is proprietary, sensitive, or did not exist when the model was trained. By including extra data in the prompt of the model, generative AI provides answers that are more accurate, current, and contextual.

One of the best ways to implement a RAG solution is through vectorization. By vectorizing data, information can be efficiently indexed, searched, and retrieved for use in the response of a Large Language Model (LLM). FactSet possesses an extensive amount of high-quality data, and vectorization helps us quickly identify, access, and serve the relevant information.

Retrieving Data Through Semantic Search

Imagine an open-book test at school. You have all the answers in your textbook, how could you possibly get a question wrong? If you have ever been in this situation, you would probably agree that the limiting factor isn’t having the information but finding it.

Finding specific information in an entire textbook is no easy job—even for AI. In the test example, you might use the textbook’s index of keywords to find pages with relevant information. For generative AI, however, we rely on a better solution: semantic search.

Instead of using a keyword index, we search by semantic meaning. This involves vectorization, which is the process of describing natural language using a series of numbers. Together, these numbers form a vector that accurately defines the initial text.

By using vectors, it becomes possible to numerically compare words and phrases. Additionally, the use of a numerical solution allows us to locate data directly instead of skimming through all possibilities.

This approach allows for a deeper, more nuanced comparison and relationship between words and phrases, as it captures the underlying meaning rather than just surface-level keywords. This way, generative AI models can quickly access the most relevant information when given a prompt.

The Need for Abstraction

Ever since the ChatGPT release in 2022, we have seen the release of hundreds of AI models, each with their own set of parameters. To safely manage them and provide our employees with generative AI as a learning tool, FactSet built Chat in early 2023.

Chat is an LLM-agnostic UI wrapper—meaning the UI works with any conversational Large Language Model—that provides a unified platform to experience all the AI models FactSet supports.

We have gained many benefits from the system, and as a result we approached RAG and vectorization with the mindset of centralization. We realized that by creating a solution for our employees, we could provide a safe, easy, and centralized platform to encourage innovation and rapid prototyping.

Vectorization as a Service

In June 2024, FactSet engineers created a simplified, generic vectorization system that enables our employees to store data for an AI solution to retrieve. For this article, the solution is referred to as Vectorization as a Service (VaaS). All employees need to do to get started with vectorization is provide their data to VaaS. This can be as simple as uploading a file or as complex as pointing VaaS at an entire database cluster of information.

In seconds, employees can ask questions and compare answers using different models and vectorization options. In minutes, users can vectorize entire databases, devise multiple vectorization strategies, and create an interface to make their data discoverable through natural language.

VaaS takes the complexity out of the vectorization process. The illustration below shows how users might interact with VaaS as well as all the complexity VaaS simplifies behind the scenes. 

How FactSetters Use VaaS

The simplest way our employees use VaaS is through our internal LLM wrapper: Chat. With the advent of VaaS, Chat was upgraded to allow users to upload a wide range of files from text documents, to slide decks, to Excel spreadsheets. Files are automatically routed to VaaS where they are rapidly prepared to answer users’ prompts.

However, VaaS is being used to solve greater challenges beyond file exploration. VaaS is enabling teams to build their own knowledge bases. By vectorizing catalogs of information such as technical documentation, learning libraries, or internal ticketing systems, FactSetters are creating a single entry-point to perform universal search across the company.

Imagine a single platform that can fetch information from every source at the company. By democratizing the aggregation and exploration of data throughout the company, VaaS serves as a nexus between FactSetters and the data they need. Since the release of VaaS, employees have created hundreds of specialized knowledge bases to make their information more discoverable.

It’s important to note that since we exclusively use deployed models within our private tenant, we guarantee safety for company data, and our prompts or responses are not used for future model training. Likewise, VaaS observes any confidentiality and authentication restrictions placed on internal data. VaaS can only fetch information the user is authorized to access.

The Impact of VaaS

The most tangible impact of VaaS for us is how accessible it makes preparing data for AI-solutions. To previously ingest data for AI, several manual steps were required. Now it is a fully automated pipeline.

Once VaaS streamlined the process of preparing data, we saw a dramatic increase in the number of tokens processed across the firm. Tokens are the smallest unit of information an AI model can process and are used to construct words, symbols, and punctuation. In the chart below, notice the significant jump in tokens processed starting in September 2024. VaaS first debuted in June 2024. 

VaaS has also driven the centralization of AI-enabled data at FactSet. Centralizing our data has enabled FactSet employees to more easily access information and collaborate while keeping our data flexible for the future.

We have already seen dramatic changes to AI and data best practices over the last few years, and it’s logical to expect more in the future. By using a system like VaaS to keep our data centralized, we ensure that all of FactSet’s data remains efficiently accessible. 

Conclusion

With the rapid evolution of AI solutions and techniques, spending extra development time on DevOps solutions is relevant now more than ever. Through VaaS, FactSet has been able to empower employees of all skill levels to build, maintain, and utilize RAG-enabled AI solutions.

 This article was a collaborative effort between FactSet's AI/ML engineer content group and Development Specialist Evan Murphy. 

This blog post is for informational purposes only. The information contained in this blog post is not legal, tax, or investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.