How we optimized vector comparisons in BBQ with hardware accelerated SIMD (Single Instruction Multiple Data) instructions.

Why do we care so much about comparing vectors?

Comparing query and stored vectors

Where does the performance come from?

How much does SIMD improve things?

Wrapping up

BBQ: Optimizing vector comparisons with hardware accelerated SIMD instructions

Smokin' fast BBQ with hardware accelerated SIMD instructions

Learn about the trade-offs using semantic reranking in search and RAG pipelines.

Retrieval

Semantic Reranking

Cross-encoders

Connection with RAG

Wrapping Up

Semantic search reranking: What it is and how to use it

What is semantic reranking and how to use it?

Understand what binary quantization is, how it works and its benefits. This guide also covers the math behind the quantization and examples.

Introduction

Building the Bit Vectors

Find a Representative Centroid

1 Bit, 1 Bit Only Please

The Catch

The Query

Estimated Distance

Re-ranking

Conclusion

Quantization allows for vectors to be encoded in a lossy manner, thus reducing fidelity slightly with huge space savings.

What are the benefits of binary quantization?

Binary quantization takes each vector and transforms it relative to a prototypical set of sample vectors creating a smaller representation that can be used to approximate distances at great space and computation savings.

How does binary quantization work?

Binary quantization does not really create codebooks and instead compresses each data vector down to a 1 bit representation that is the inclusion in a region around a centroid.  The authors of the ![RaBitQ paper](https://arxiv.org/pdf/2405.12497), that is a large driver for our understanding and implementation of binary quantization, mention how the codebooks in PQ are related to the regions around any given centroid so if it's more intuitive to think of these regions as codebooks then yes.

Does binary quantization create codebooks similar to PQ?

Binary quantization relies on finding regions first.  Query vectors similarly are transformed into this space using the same centroid that allowed transformation into the region that was used to quantize the data vectors.  This is what allows the distance estimation comparison to happen.

Are the query vectors converted to 1 bit prior to comparison with the codebook / regions created from the data vectors?

Yes, if the number of vectors justifies it.  Although, even for millions of vectors, we have found that only one centroid is sufficient for many datasets and models.

Are there multiple centroids?

No, since very few centroids are required they can be maintained in their float32 form without concern for scaling.

Are the centroids converted to a 1 bit representation as well?

We've found so far that we can reduce complexity, both algorithmically and performance-wise, by removing this random transformation.  While some vectors may be favored, traditionally reranking a reasonable set of data, which is required to achieve reasonable recall, seems to negate the need for this in practice.

The author's of the ![RaBitQ paper](https://arxiv.org/pdf/2405.12497) mention transforming the data with a random orthogonal matrix P why do you not include it here?

We've found so far that a single centroid for most production datasets works well.  Additionally when talking about how binary quantization works we can focus on a single centroid at a time.  As you have more vectors, >100m, you may find that recall is impacted.  When using Elasticsearch these large datasets can be pre-partitioned and multiple centroids will naturally be used.

The author's of the ![RaBitQ paper](https://arxiv.org/pdf/2405.12497) mention using multiple centroids but you focus on a single centroid here.  Why only one centroid?

Better Binary Quantization 101: An Introduction

Better Binary Quantization 101

An alternative approach to parsing PDF tables for RAG, overcoming the limitations of highly normalized formats like CSV and JSON

Why This Approach?

Parsing Challenges: A Case Study with SEC FORM 10-Q

Complexity of Table Structure

Loss of Context

Handling Special Characters and Formats

Addressing the Challenge

The Benefits

Key Code Explanation

1. Extracting Text and Tables from PDFs

2. Cleaning and Sending Tables to Azure OpenAI

3. Writing the Final Output

From PDF Tables to Insights: An Alternative Approach for Parsing PDFs in RAG

Learn how to use Eland on Elasticsearch Serverless

NLP in Elasticsearch Serverless

Data frames in Eland

Ingesting test data

Connecting to Elasticsearch

Searching data with Eland

Eland on Elasticsearch Serverless: Searching data with Eland

Using Eland on Elasticsearch Serverless

Learn how to use LLMs to write Elastic Query DSL and query structured data with filters

Problem Statement

Solution

Prompt Design

1. Document Schema

2. Examples

3. Special Cases

4. Leniency and Coverage

Test Procedure

Test 1

Query:

Generated Elastic Query:

Results:

Test 2

Test 3

Discussion

Write Elastic Query DSL to search structured datasets

Automating traditional search with LLMs

How to add an AI summary box along with the search results to enrich your search experience.

Steps

Creating endpoints

Embeddings endpoint

Completion endpoint

Indexing data

Creating the proxy

Creating the component

Updating App.js

Asking questions

Generate AI summaries with Elastic

Adding AI summaries to your site with Elastic

Discover how to use Elasticsearch vector database with AlibabaCloud AI Search, which offers inference, reranking, and embedding capabilities.

Using a base model in AlibabaCloud AI Search platform

Creating an inference API endpoint in Elasticsearch

I. Completion

Step 1: Configure the Completion Service

Step 2: Issue a Completion Request

II. Rerank

Step 1: Configure the Rerank Service

Step 2: Issue a Rerank Request

III. Sparse Embedding

Step 1: Configure the Sparse Embedding Service

Step 2: Issue a Sparse Embedding query

IV. Text Embedding

Step 1: Configure the Text Embedding Service

Step 2: Issue a Text Embedding Request

AI search with Elastic and AlibabaCloud

Use AlibabaCloud AI Search with Elasticsearch vector database

Elasticsearch open Inference API adds support for AlibabaCloud AI Search

An easy approach to create embeddings for and apply semantic GenAI powered search (RAG) to documents as part of the BSI IT Grundschutz using Elastic's new semantic_text field type and the Playground in Elastic.

semantic_text field type

Prerequisites

Configurations in Elastic

Pushing and Indexing “IT Grundschutz” to Elastic

Prototyping search with Playground

Summary and conclusions

BSI IT Grundschutz: Easy approach to GenAI-powered search

Understanding BSI IT Grundschutz: A recipe for GenAI powered search on your (private) PDF treasure

Blogs

Exploring depth in a 'retrieve-and-rerank' pipeline