Project Guidelines¶

In this documentation, you will find comprehensive guidance on setting up and configuring your projects using the AI Refinery SDK. Specifically, we will discuss the project point of view in AI Refinery and provide detailed guidelines on how to create your custom agent and configure your project. To quickly start using an example project, please see our quickstart guide.

Project POV — Project Point of View in AI Refinery SDK

AI Refinery (service)¶

The AI Refinery service acts as the host-side solution for your project. This comprehensive solution manages the orchestration of various agents, including super agents and built-in utility agents. It also supports and hosts large language models (LLMs) and the essential GPU resources to ensure seamless and efficient agent performance.

SDK Environment¶

Through our AI Refinery SDK, the client side of your project supports extensive customization, enabling you to effortlessly create your own AI solutions. Particularly, the client side of your project features the following:

Project Configuration Schema¶

You can configure your project by using a YAML file. The root of the YAML contains the following sections: base_config, utility_agents, super_agents, orchestrator and memory_config. The template of your project YAML file with their descriptions can be seen below:

`base_config`¶

The base_config and all its attributes are optional. There is a default base_config that is used for all projects. If you provide your own base_config for your project, the default values get overridden.

base_config:
# Optional. There is a default base_config that is used if you do not provide your own base_config.

  llm_config:
    #Optional. 
    model: <An LLM from our model catalog> # Optional. Defaults to "meta-llama/Llama-3.1-70B-Instruct"
    temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
    top_p: <top_p for the LLM inference> # Optional. Defaults to 1
    max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048


  vlm_config:
    #Optional. 
    model: <A VLM from our model catalog> # Optional. Defaults to "meta-llama/Llama-3.2-90B-Vision-Instruct"
    temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
    top_p: <top_p for the LLM inference> # Optional. Defaults to 1
    max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048

  reranker_config:
    #Optional. 
    model: "<A reranker from our model catalog>" # Optional. Note: Defaults to "nvidia/llama-3.2-nv-rerankqa-1b-v2"

  compression_config:
    #Optional. 
    model: "<A compression mdoel from our model catalog>"  # Optional. Defaults to "llmlingua/bert"

  embedding_config:
    #Optional. 
    model: "<An embedding mdoel from our model catalog>" # Optional. Defaults to "intfloat/e5-mistral-7b-instruct"

`utility_agents`¶

The utility_agents is a required section for configuring your project. This section includes all the utility agents, both built-in and custom, that you plan to use. Each utility agent must be listed and configured within this section.

The utility_agents section contains a list where each item specifies the configuration of a single agent. The number of agents in this list will vary based on your project requirements. Below is a generic template that demonstrates how to configure utility_agents with two agents.

For detailed information on all the configurable options of different built-in utility agents, please visit our agent library.

Self-Reflection is a feature for Utility Agents that allows the agent to evaluate and iteratively improve its response before replying to the user query, thereby enhancing the overall quality and accuracy of the final output. It currently supports the Analytics Agent, Critical Thinker Agent, Planning Agent, Research Agent, Search Agent, and Base Utility Agent. Each supported agent has its own default configuration, please refer to its detail page for more information.

utility_agents:
# Required
  - agent_class: <Class of the agent, e.g., AuthorAgent. Must be CustomAgent for your custom agents> # Required. Agent 1 
    agent_name: <A name that your choose for this agent e.g., "Author Agent". For a CustomAgent, the name must be from your executor_dict> # Required. Name of Agent 1
    agent_description: <Description of the agent> # Optional
    config:
    # Optional. Configuration of this agent. 
      output_style: <"markdown" or "conversational" or "html">  # Optional field
      contexts:  # Optional field
        - "date"
        - "chat_history" # the chat history upto a certain number (configured using memory_config) of rounds
        - "env_variable"
        - "relevant_chat_history" # the chat history that is relevant to the current query
      llm_config: # Optional. The LLM the agent should use. Set to the base_config.llm_config if not provided.
        model: <An LLM from our model catalog>
        temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
        top_p: <top_p for the LLM inference> # Optional. Defaults to 1
        max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048
      self_reflection_config: # Optional. Configuration for self-reflection. Only supported for the agents listed above.
        self_reflection: <true or false>  # Whether to enable self-reflection for this agent. Defaults to false.
        max_attempts: <number>  # Maximum number of times the agent can perform self-reflection. Defaults to 2.
        response_selection_mode: <"best" | "aggregate" | "auto">  # Strategy used to generate the final response after reaching max_attempts. Defaults to "auto".
        return_internal_reflection_msg: <true or false>  # Whether to return internal reflection messages. Defaults to false.


  - agent_class: <Class of the agent, e.g., CustomAgent. Must be CustomAgent for your custom agents> # Required. Agent 2 
    agent_name: <A name that your choose for this agent e.g., "My Custom Agent". For a CustomAgent, the name must be a key in your `executor_dict`> # Required. Name of Agent 2
    agent_description: <Description of the agent> # Optional
    config:
    # Optional. Configuration of this agent. 
      output_style: <"markdown" or "conversational" or "html">  # Optional field
      contexts:  # Optional field
        - "date"
        - "chat_history" # the chat history upto a certain number (configured using memory_config) of rounds
        - "env_variable"
        - "relevant_chat_history" # the chat history that is relevant to the current query
      llm_config: # Optional. The LLM the agent should use. Set to the base_config.llm_config if not provided.
        model: <An LLM from our model catalog>
        temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
        top_p: <top_p for the LLM inference> # Optional. Defaults to 1
        max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048
      self_reflection_config:  # Optional. Configuration for self-reflection. Only supported for the agents listed above.
        self_reflection: <true or false>  # Whether to enable self-reflection for this agent. Defaults to false.
        max_attempts: <number>  # Maximum number of times the agent can perform self-reflection. Defaults to 2.
        response_selection_mode: <"best" | "aggregate" | "auto">  # Strategy used to generate the final response after reaching max_attempts. Defaults to "auto".
        return_internal_reflection_msg: <true or false>  # Whether to return internal reflection messages. Defaults to false.

`super_agents`¶

The super_agents section is an optional configuration for your project. If your project requires handling complex tasks that involve multiple steps, you can set up super agents to manage them. For more information about super agents, visit this page.

super_agents: # A list of super agents that handles different complex tasks
  - agent_class: SuperAgent # The class must be SuperAgent
    agent_name: <A name that you choose for your super agent.> # Required. 
    agent_description: <Description of your super agent.> # Optional.

    config: # Required. Configuration of this super agent.
      max_turns: <Maximum number iterations to complete the tasks in the checklist.> # Required. 
      goal: <A high level goal of your super agent.> # Required
      steps: <The steps that should to be followed by the super agent.> # Required
      exit: <The name of the exit agent> # This agent generates the final output once all tasks in the checklist is completed. Must be one of the agents in the agent pool i.e., `agent_list` (see below).

      agent_list: # Required. The list of agents to be added in the agent pool. Each agent listed here must be configured under `utility_agents` in the root of project YAML file.
        - agent_name: <Name of agent 1>  # Requried. 
          requirements: # Optional. If provided, these will be the preliminary tasks that must be completed (i.e., the pre-specified todo list) before the super agent focuses on the main task. 
            - <Task 1>
            - <Task 2>

        - agent_name: <Name of agent 2>  # Required. 
          requirements: # Optional. If provided, these will be the preliminary tasks that must be completed (i.e., the pre-specified todo list) before the super agent focuses on the main task.
            - <Task 1>
            - <Task 2>
            - <Task 3>

      llm_config:
      # Optional. Customized llm config (if you want the super agent to use a different LLM than the on in your base config)
        model: <model_name>

`orchestrator`¶

The orchestrator is a required section for configuring your project. The agent_names of all utility agents and super agents that you want your orchestrator to have access to must be listed under agent_list. You can also optionally configure input guardrails for the orchestrator of your project. The template of your project YAML file with their descriptions can be seen below:

orchestrator:
# Required
  agent_list:
  # Required. All names listed here must be the names of the utility agents you listed in the `utility_agents` section.
    - agent_name: "<Name that you chose for Agent 1>" # Required
    - agent_name: "<Name that you chose for Agent 2>" # Required

`memory_config`¶

The memory_config is an optional section for configuring your project. This configuration specifies the types of memory modules and their parameters, allowing your assistant to store and access different kinds of information. The template for the memory_config can be seen below:

memory_config:  
# Optional
  memory_modules:  
  # A list of memory modules that your project will use
    - memory_name: chat_history  # A unique identifier for the memory module to retain the chat history
      memory_class: ChatMemoryModule  # The class of this memory module
      kwargs:  
        n_rounds: <number of rounds of chat history the memory module should store>

    - memory_name: env_variable  # A unique identifier for the memory module to store environment variables
      memory_class: VariableMemoryModule  # The class of this memory module
      kwargs:  
        variables:  # The list of environment variables along with their values this memory modules should store 
          <env_variable_1>: <value of env_variable_1>
          <env_variable_2>: <value of env_variable_2>

The information stored by the memory modules can be used by any of the agents in your project.

Creating Your Project¶

Once you have your project configuration ready in an YAML file, you are ready to create your project using the Distiller API of the AI Refinery SDK. For example, if your defined your project configuration in a file named "example.yaml", you can use the following code snippet to create your project:

import os

from air import DistillerClient, login
from dotenv import load_dotenv

load_dotenv() # This loads your ACCOUNT and API_KEY from your local '.env' file

# login to AI Refinery with your credentials
auth = login(
    account=str(os.getenv("ACCOUNT")),
    api_key=str(os.getenv("API_KEY")),
)

# Create an DistillerClient
client = DistillerClient()

# Create a project name using your project yaml config file, and a 
# project name of your choice.
client.create_project(config_path="example.yaml", project="example")

Here, we create a project named example. The project name that you choose must conform to our Project Name Conventions.

Creating a Custom Agent¶

A custom agent is a Python function designed to process a string query and return a single output. The complexity of this function can vary widely:

Simple Tasks: A custom agent can be as simple as getting a single response from an LLM based on the query.
Complex Systems: A custom agent can also be designed to handle more intricate operations, such as:
- Gathering information from multiple sources (e.g., utilizing Retrieval-Augmented Generation (RAG) systems)
- Performing data analytics
- Returning a comprehensive response

The template for creating custom agents can be seen below:

async def your_custom_agent(query: str) -> str:  
    """  
    Processes the given query and generates a response.  

    Args:  
        query (str): The input query to be processed.  

    Returns:  
        str: The response from the agent.  
    """  
    # Process the query  
    # Generate a response  
    # You can simply use the query as the input to an LLM to generate the response
    # Or you perform complex operations to generate the response  
    return response

Utilizing the Executor Dictionary¶

A key component of the SDK is the Executor Dictionary. It is a Python dictionary which includes each of the custom agent that you create for your project and the executors/tools required by any built-in agent. If you do not add your project to the Executor Dictionary, you may encounter unexpected errors and the AI Refinery service will not utilize your custom agents.

For example, if the name of the function defined for your custom agent is your_custom_agent, it can be added to the executor dictionary as follows:

executor_dict = {"<A name that you choose for your custom agent>": your_custom_agent}

⚠️ Note: The key in the dictionary for a custom agent must match the agent_name in the project YAML file.

Interacting with Your Project¶

Once you have your project created, and your executor_dict set up with all your custom agents and tools, you can start interacting with your project via the Distiller API.

import asyncio
import os

from air import DistillerClient, login
from dotenv import load_dotenv

load_dotenv()

# Log in with your credentials  
auth = login(  
    account=str(os.getenv("ACCOUNT")),  
    api_key=str(os.getenv("API_KEY")),  
)  

# Create a DistillerClient  
client = DistillerClient()  

# Create a project name using your project yaml config file, and a project name of your choice  
client.create_project(config_path="example.yaml", project="example")  

# Define a mapping between your custom agent to Callable. 
# When the custom agent is summoned by the super agent / orchestrator,
# distiller-sdk will run the custom agent and send its response back to the
# multi-agent system.
executor_dict = {"Custom Agent Example": your_custom_agent}  

async def run_query():  
    async with client(  
        project="example",  # name of project you created  
        uuid="test_user",   # a user id  
        executor_dict=executor_dict  
    ) as dc:  
        responses = await dc.query(query="hi")  # Send a query to AI Refinery  
        async for response in responses:  
            print(response['content'])  # print the received response  

# Ensure you have an event loop to run the asynchronous function  
asyncio.run(run_query())

Here, we interact with the project using an user id test_user. Note that, similar to he project name, the user id must conform to our same conventions below.

Project Name & User ID Convention¶

Please note that the project names and user IDs (such as "example" and "test_user" in the example above) can be any string of your choosing that includes only the following characters:

Letters
Numbers
Hyphens
Underscores