AI Refinery Core Concepts

Distiller

The Distiller framework of our SDK is designed to streamline complex workflows by orchestrating various agents that handle different tasks. Below are the key components of the Distiller and their roles. AI Refinery SDK is equipped with a distiller client api. An overview of our Distiller framework can be seen in the figure below.

Overview of the Distiller Framework

Orchestrator

The Orchestrator is a general-purpose component that routes users' requests i.e., queries to the appropriate agents. It ensures that tasks are assigned to the correct agents based on their capabilities and the nature of the request.

Features

• Request Routing: Directs requests to the suitable agent based on the user query and the description of each agent.
• Task Management: Manages the lifecycle of tasks from initiation to completion.
• Scalability: Efficiently handles multiple requests and scales as needed.

Utility Agents (Built-in & Custom)

Utility Agents are specialized entities designed to accomplish specific tasks such as research, analytics, writing, and more. They come in two main types: built-in agents and custom agents.

Built-in Utility Agents are pre-configured to focus on a single task but can be adjusted by the user to meet specific requirements. Custom Utility Agents are user-defined and leverage proprietary data and tools within the user's own environment. These agents are tailored by the user to perform specific tasks using unique data sets and workflows.

Features

• Single Task Focus: Built-in agents are specialized for a particular task.
• User Configuration: Built-in agents allow user adjustments to better fit project needs.
• Efficiency: Built-in agents are optimized to perform their specific tasks quickly and accurately.
• Customization: Custom agents are fully customizable to use proprietary data and tools.
• Environment Specific: Custom agents operate within the user's environment.
• Flexibility: Custom agents provide the flexibility to define unique tasks and workflows.

Super Agents

A Super Agent is a general-purpose agent that decomposes a complex task into multiple subtasks. These subtasks are then assigned to the appropriate utility agents (either built-in or custom or both). Super Agents are configured by the user to handle specific types of complex tasks.

Features

• Task Decomposition: Breaks down complex tasks into manageable subtasks.
• Custom Configuration: Users can configure Super Agents based on their specific needs.
• Integration: Works seamlessly with other agents to complete tasks efficiently.

Agent Memory

Agent Memory enhances AI agents by enabling them to remember past interactions and data, leading to more personalized and context-aware responses. To ensure optimal performance, correct configuration is critical—misconfigured modules can lead to errors and hinder the AI Refinery service from leveraging memory effectively. For more detailed guidance on Memory modules, please refer to Memory Tutorial session.

Chat History Module

• The ChatMemoryModule stores and manages the history of chat conversations, enabling efficient retrieval and modification.
• Key Features:
  • Stores chat history in a structured format (chat_history.json).
  • Retrieves past messages based on the number of rounds (n_rounds) or a maximum context size (max_context).
  • Allows adding new messages while maintaining an organized chat history.

Relevant Chat History Module

• This module integrates an optimized chat_history retrieval that fetches the most relevant previous conversation from all the rounds in the current session history to generate a concise summary.
• Key Features:
  • Integrates new information with past conversation data.
  • Summarizes previous entries by focusing on overarching themes and summarizing key insights.
  • Ensures clarity and readability, avoiding unnecessary details or redundant references.

Variable Memory Module

• This module manages key-value pairs, allowing for the storage, retrieval, and modification of variables.
• Key Features:
  • Stores structured variables for reuse across sessions.
  • Supports retrieval and updates with context size limits.
  • Facilitates data persistence for improved conversation continuity.

AI Refinery SDK

The AI Refinery SDK is engineered to facilitate the development of AI solutions by leveraging the aforementioned Distiller. An overview of the AI Refinery SDK workflow is illustrated in the figure below. As we can see in the figure, it is comprised of two main components which are detailed below.

Workflow Overview of AI Refinery

Server Backend

The server backend of the AI Refinery SDK is responsible for hosting the Distiller, which includes the orchestrator, super agent, and built-in utility agents. Additionally, it instantiates and hosts user-provided custom utility agents. The server backend also manages and stores user account history in a dedicated database.

User Backend

The user backend of the AI Refinery SDK is equipped with comprehensive APIs to facilitate the creation of custom utility agents and to enable seamless communication with the server backend for queries and requests. Additionally, the user backend is responsible for managing proprietary data that users wish to employ as the knowledge base for their agents.