Explore OpenAI's function calling capabilities and learn how to implement dynamic function calls, including fetching data from Elasticsearch, enhancing the model's real-time data access and complex operation handling.

Introduction to OpenAI function calling

Implementing function calling

Prerequisites

Elastic

OpenAI

Open-Meteo API

Sample data

Python notebook

Import packages

Accept credentials

Function1: fetch_from_elasticsearch()

`def fetch_from_elasticsearch(nl_query):`

Get index mapping

Get reference document

Generate Elasticsearch query DSL based on user query

Parallel function calling

Conclusion

OpenAI function calling refers to the capability of AI models to interact with external functions or APIs, allowing them to perform tasks beyond text generation. This feature enables the model to execute code, retrieve information from databases, interact with external services, and more, by calling predefined functions.

What is function calling in OpenAI?

OpenAI function calling with Elasticsearch

Learn how to ingest data from Snowflake to Elasticsearch using Logstash or a Snowflake Elasticsearch Python Script.

Snowflake credentials<a id="snowflake-credentials"></a>

Elastic credentials

Methods to ingest data from Snowflake to Elasticsearch

Method 1: Using Logstash

Method 2: Using Snowflake-Elasticsearch Python script

Verify data

Snowflake is a fully managed SaaS that provides a single platform for data warehousing, data lakes, data engineering, data science, data application development, and secure sharing and consumption of real-time/shared data.

What is Snowflake?

How to ingest data from Snowflake to Elasticsearch

Go to the Maven Central Repository and download: https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc.

Click on the directory for the version that you need and download the snowflake-jdbc-#.#.#.jar file.

Create a pipeline by creating file sf-es.conf. Add the snippet and replace all credentials.

Follow this step-by-step guide to build a Retrieval Augmented Generation (RAG) system using Gemma, Hugging Face, and Elasticsearch.

Background

How to a RAG system with Gemma, Hugging Face & Elasticsearch

Overview

1. Import packages & credentials

Install required packages

Import all dependencies

Get credentials

2. Preparing documents

Download the sample dataset and deserialize the document

Document chunking

Why is document chunking needed?

Context window limit

Balancing context size

Hallucination

How to chunk data?

3. Index documents

4. Load Gemma model locally using Hugging Face

Why Hugging Face?

Hugging Face login

Initialize the tokenizer with the model (google/gemma-2b-it)

Usage

Create a `text-generation` pipeline and initialize with LLM

5. Create a chain using a prompt template

6. Ask a question

Build a RAG system with Gemma, Hugging Face & Elasticsearch

Building a RAG System With Gemma, Hugging Face & Elasticsearch

Author

Ashish Tiwari

Articles

OpenAI function calling with Elasticsearch

How to ingest data from Snowflake to Elasticsearch

Building a RAG System With Gemma, Hugging Face & Elasticsearch