Project Guidelines¶

In this documentation, you will find comprehensive guidance on setting up and configuring your projects using the AI Refinery SDK. Specifically, we will discuss the project point of view in AI Refinery and provide detailed guidelines on how to create your custom agent and configure your project. To quickly start using an example project, please see our quickstart guide.

Project POV — Project Point of View in AI Refinery SDK

AI Refinery (service)¶

The AI Refinery service acts as the host-side solution for your project. This comprehensive solution manages the orchestration of various agents, including super agents and built-in utility agents. It also supports and hosts large language models (LLMs) and the essential GPU resources to ensure seamless and efficient agent performance.

Client Environment¶

Through our AI Refinery SDK, the client side of your project supports extensive customization, enabling you to effortlessly create your own AI solutions. Particularly, the client side of your project features the following:

Custom Tools Integration¶

The AI Refinery SDK empowers you to leverage your own tools, including various APIs, Retrieval-Augmented Generation (RAG) systems, and custom agents. This flexibility allows you to tailor your AI solutions to meet your specific requirements. Here are a few examples of custom tool integration:

Implement your own RAG pipeline using your organization's proprietary data and seamlessly incorporate it into the agentic framework of AI Refinery.
Grant your custom agent access to closed-source data through your own or third-party APIs.

Custom Agent Gallery¶

A key component of the SDK is the Custom Agent Gallery. It is a Python dictionary which includes each of the custom agent that you create for your project. If you do not add your project to the Custom Agent Gallery, you may encounter unexpected errors and the AI Refinery service will not utilize your custom agents.

What is a custom agent?¶

A custom agent is a Python function designed to process a string query and return a single output. The complexity of this function can vary widely:

Simple Tasks: A custom agent can be as simple as getting a single response from an LLM based on the query.
Complex Systems: A custom agent can also be designed to handle more intricate operations, such as:
- Gathering information from multiple sources (e.g., utilizing Retrieval-Augmented Generation (RAG) systems)
- Performing data analytics
- Returning a comprehensive response

The template for creating custom agents can be seen below:

async def your_custom_agent(query: str, **kwargs) -> str:  
    """  
    Processes the given query and generates a response.  

    Args:  
        query (str): The input query to be processed.  

    Returns:  
        str: The response from the agent.  
    """  
    # Process the query  
    # Generate a response  
    # You can simply use the query as the input to an LLM to generate the response
    # Or you perform complex operations to generate the response  
    return response

Next, this agent needs to be added to the custom agent gallery as follows:

custom_agent_gallery = {"<A name that you choose for your custom agent>": your_custom_agent}

The custom_agent_gallery is used as an argument for running the DistillerClient that interacts with the AI Refinery service.

Configuring Your Project¶

You can configure your project by using a YAML file. The root of the YAML contains the following sections: base_config, utility_agents, super_agents and orchestrator. The template of your project YAML file with their descriptions can be seen below:

`base_config`¶

The base_config and all its attributes are optional. There is a default base_config that is used for all projects. If you provide your own base_config for your project, the default values get overridden.

base_config:
# Optional. There is a default base_config that is used if you do not provide your own base_config.

  llm_config:
    #Optional. 
    model: <An LLM from our model catalog> # Optional. Defaults to "meta-llama/Llama-3.1-70B-Instruct"
    temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
    top_p: <top_p for the LLM inference> # Optional. Defaults to 1
    max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048


  vlm_config:
    #Optional. 
    model: <A VLM from our model catalog> # Optional. Defaults to "meta-llama/Llama-3.2-90B-Vision-Instruct"
    temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
    top_p: <top_p for the LLM inference> # Optional. Defaults to 1
    max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048

  reranker_config:
    #Optional. 
    model: "<A reranker from our model catalog>" # Optional. Defaults to "meta-llama/Llama-3.2-90B-Vision-Instruct"

  compression_config:
    #Optional. 
    model: "<A compression mdoel from our model catalog>"  # Optional. Defaults to "llmlingua/bert"

  embedding_config:
    #Optional. 
    model_name: "<An embedding mdoel from our model catalog>" # Optional. Defaults to "intfloat/e5-mistral-7b-instruct"

`utility_agents`¶

The utility_agents is a required section for configuring your project. This section includes all the utility agents, both built-in and custom, that you plan to use. Each utility agent must be listed and configured within this section.

The utility_agents section contains a list where each item specifies the configuration of a single agent. The number of agents in this list will vary based on your project requirements. Below is a generic template that demonstrates how to configure utility_agents with two agents.

For detailed information on all the configurable options of different built-in utility agents, please visit our agent marketplace.

utility_agents:
# Required
  - agent_class: <Class of the agent, e.g., AuthorAgent. Must be CustomAgent for your custom agents> # Required. Agent 1 
    agent_name: <A name that your choose for this agent e.g., "Author Agent". For a CustomAgent, the name must be from your custom_agent_gallery> # Required. Name of Agent 1
    agent_description: <Description of the agent> # Optional
    config:
    # Optional. Configuration of this agent. 
      output_style: <"markdown" or "conversational" or "html">  # Optional field
      contexts:  # Optional field
        - "date"
        - "chat_history"
        - "chat_summary"
      llm_config: # Optional. The LLM the agent should use. Set to the base_config.llm_config if not provided.
        model: <An LLM from our model catalog>
        temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
        top_p: <top_p for the LLM inference> # Optional. Defaults to 1
        max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048

  - agent_class: <Class of the agent, e.g., CustomAgent. Must be CustomAgent for your custom agents> # Required. Agent 2 
    agent_name: <A name that your choose for this agent e.g., "My Custom Agent". For a CustomAgent, the name must be a key in your `custom_agent_gallery`> # Required. Name of Agent 2
    agent_description: <Description of the agent> # Optional
    config:
    # Optional. Configuration of this agent. 
      output_style: <"markdown" or "conversational" or "html">  # Optional field
      contexts:  # Optional field
        - "date"
        - "chat_history"
        - "chat_summary"
      llm_config: # Optional. The LLM the agent should use. Set to the base_config.llm_config if not provided.
        model: <An LLM from our model catalog>
        temperature: <A temparature for the LLM inference> # Optional. Defaults to 0.5
        top_p: <top_p for the LLM inference> # Optional. Defaults to 1
        max_tokens: <Max tokens for the LLM inference> # Optional. Defaults to 2048

`super_agents`¶

The super_agents section is an optional configuration for your project. If your project requires handling complex tasks that involve multiple steps, you can set up super agents to manage them. For more information about super agents, visit this page.

super_agents: # A list of super agents that handles different complex tasks
  - agent_class: SuperAgent # The class must be SuperAgent
    agent_name: <A name that you choose for your super agent.> # Required. 
    agent_description: <Description of your super agent.> # Optional.

    config: # Required. Configuration of this super agent.
      max_turns: <Maximum number iterations to complete the tasks in the checklist.> # Required. 
      goal: <A high level goal of your super agent.> # Required
      steps: <The steps that should to be followed by the super agent.> # Required
      exit: <The name of the exit agent> # This agent generates the final output once all tasks in the checklist is completed. Must be one of the agents in the agent pool i.e., `agent_list` (see below).

      agent_list: # Required. The list of agents to be added in the agent pool. Each agent listed here must be configured under `utility_agents` in the root of project YAML file.
        - agent_name: <Name of agent 1>  # Requried. 
          requirements: # Optional. If provided, these will be the preliminary tasks that must be completed (i.e., the pre-specified todo list) before the super agent focuses on the main task. 
            - <Task 1>
            - <Task 2>

        - agent_name: <Name of agent 2>  # Required. 
          requirements: # Optional. If provided, these will be the preliminary tasks that must be completed (i.e., the pre-specified todo list) before the super agent focuses on the main task.
            - <Task 1>
            - <Task 2>
            - <Task 3>

      llm_config:
      # Optional. Customized llm config (if you want the super agent to use a different LLM than the on in your base config)
        model: <model_name>

`orchestrator`¶

The orchestrator is a required section for configuring your project. The agent_names of all utility agents that you want to use must be listed under orchestrator. The template of your project YAML file with their descriptions can be seen below:

orchestrator:
# Required
  agent_list:
  # Required. All names listed here must be the names of the utility agents you listed in the `utility_agents` section.
    - agent_name: "<Name that you chose for Agent 1>" # Required
    - agent_name: "<Name that you chose for Agent 2>" # Required

AI Refinery SDK¶

Through the AI Refinery SDK, you can create and manage your projects on the AI Refinery server side. Once you complete creating your custom agent and configuring your YAML file, you can use our DistillerClient API to call our AI Refinery service that instantiates and runs your project.