Model Catalog¶

Our comprehensive model catalog provides a diverse array of models for your selection. To configure your agents to leverage any of these models, please refer to our project configuration guidelines. Below, you will find a list of the models currently supported. We are dedicated to the continuous enhancement and expansion of our model catalog, so please visit this page regularly for the latest updates.

Large Languge Models (LLMs)¶

The list of LLMs that we support are:

meta-llama/Llama-3.1-70B-Instruct
meta-llama/Llama-3.1-70B-Instruct-NIM (powered by NIM)
meta-llama/Llama-3.1-8B-Instruct

Configuring LLMs for Your Project¶

To utilize any of these LLMs in your project, simply update the llm_config within the base_config or within the config section of any utility agents in your project's YAML configuration file. Ensure that the model parameter of the llm_config is set to one of the names listed above.

Using LLMs through Our Inference API¶

You can also directly use any of the models listed above through our inference API. See an example below:

import os

from openai import OpenAI

from air import login

auth = login(
    account=str(os.getenv("ACCOUNT")), # your account 
    api_key=str(os.getenv("API_KEY")), # your API key
)
base_url = os.getenv("AIREFINERY_ADDRESS", "")

client = OpenAI(**auth.openai(base_url=base_url))

# Create a chat request  
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    model="meta-llama/Llama-3.1-70B-Instruct", # an LLM from the list  above
)

Vision Language Models (VLMs)¶

The list of VLMs that we support are:

meta-llama/Llama-3.2-90B-Vision-Instruct

To utilize any of these VLMs, simply update the vlm_config within the base_config or within the config section of any utility agents (which support images e.g.,ImageUnderstandingAgent) in your project's YAML configuration file. Ensure that the model parameter of the vlm_config is set to one of the names listed above.

Embedding Models¶

The list of models that we support for embedding your data are as follows:

intfloat/e5-mistral-7b-instruct
nvidia/nv-embedqa-mistral-7b-v2
intfloat/multilingual-e5-large

Using Embedding Models in Your Project¶

To utilize any of these embedding models in your project, simply update the embedding_config within the base_config or within the aisearch_config section of the ResearchAgent. Ensure that the model_name parameter of the embedding_config is set to one of the names listed above.

Embedding Your Data Using Our Inference API¶

You can also directly use any of the models listed above to embed your data using our inference API. See an example below:

import os

from openai import OpenAI

from air import login

auth = login(
    account=str(os.getenv("ACCOUNT")), # your account 
    api_key=str(os.getenv("API_KEY")), # your API key
)
base_url = os.getenv("AIREFINERY_ADDRESS", "")

client = OpenAI(**auth.openai(base_url=base_url))

# Create an embedding request  
response = client.embeddings.create(  
    input=["What is the capital of France?"],  
    model="nvidia/nv-embedqa-mistral-7b-v2",  # required
    encoding_format="float",  # required
    extra_body={"input_type": "query", "truncate": "NONE"}  # extra_body is required for "nvidia/nv-embedqa-mistral-7b-v2" model
    # where "input_type" can be either "query" or "passage"
)

Compression Models¶

The list of prompt compression models that we support are:

llmlingua/bert

To utilize any of these prompt compression models in your project, simply update the compression_config within the base_config of your project. To learn more about prompt compression, see this tutorial. Ensure that the model parameter of the compression_config is set to one of the names listed above.

Reranker Models¶

The list of reranker models that we support are:

BAAI/bge-reranker-large
BAAI/bge-reranker-v2-m3

To utilize any of these reranker models in your project, simply update the reranker_config within the base_config of your project. To learn more about reranking, see this tutorial. Ensure that the model parameter of the reranker_config is set to one of the names listed above.