HumanAgent¶
The HumanAgent is a built-in utility agent within the AI Refinery SDK that facilitates human-in-the-loop (HITL) workflows. It enables the system to collect user feedback at runtime and dynamically incorporate it into the response generation pipeline. For instance, when a research agent produces an initial draft of a research report, the HumanAgent can prompt the user for feedback. If the user requests deeper analysis on a specific topic, the system can update the downstream process to focus more on that area.
Workflow Overview¶
The HumanAgent can be invoked by a Super Agent or the orchestrator and supports the following capabilities:
-
Query Preparation: Queries for user feedback can be generated in two ways:
- Structure Mode: Automatically generated in natural language from a predefined schema. The user-facing query is formatted based on both the schema and the context within the agent pipeline. Each question in the schema specifies:
- the expected response type (options include
bool,str,int,float), - a question description,
- and whether the question is required.
Detailed examples are provided in the following sections.
- the expected response type (options include
- Free-form Mode: The query is a natural-language question, without a predefined schema. It is composed by an upstream agent—an agent at a preceding stage in the pipeline that invokes the HumanAgent.
If no schema is defined, the agent defaults to Free-form Mode.
- Structure Mode: Automatically generated in natural language from a predefined schema. The user-facing query is formatted based on both the schema and the context within the agent pipeline. Each question in the schema specifies:
-
User Feedback Collection: Gathers feedback using a defined input method. By default, input is collected via the terminal, but custom input methods (e.g., a UI interface) can be configured.
- Terminal: Prompts the user for input directly via the command line.
- Custom: Enables integration with customized external input interfaces (e.g., a web UI).
-
Feedback Interpretation (Optional): An interpreter agent can optionally be used to refine structured feedback into natural language responses.
Usage¶
The HumanAgent can be readily integrated into a project by adding the required configuration in the project's YAML file. The agent needs to be listed under the available Utility Agents and then can be selected by corresponding Super Agent or the orchestrator.
Quickstart¶
To quickly set up a project with a HumanAgent in the Structure Mode, use the following YAML configuration. To use a custom input method instead for feedback collection, change user_input_method: "Terminal" to user_input_method: "Custom" and define the customized input method in the corresponding python file.
orchestrator:
agent_list:
- agent_name: "Human in the Loop Advisor"
utility_agents:
- agent_class: SearchAgent
agent_name: "Initial Research Agent"
agent_description: "Performs the first phase of research."
- agent_class: HumanAgent
agent_name: "Human Reviewer"
agent_description: "This agent interacts with the user to get feedback or additional information."
config:
user_input_method: "Terminal" # How the agent collects user feedback
wait_time: 300 # Maximum time in seconds to wait for user feedback
feedback_interpreter: true # Optional. If true, converts structured feedback into natural language
feedback_schema: # Schema definition for structured feedback (required if using Structure Mode)
is_answer_correct: # Question identifier
type: "bool" # Type of expected feedback
description: "Is the answer provided correct?" # Description of the question
required: true # Whether required
need_more_detail: # Question identifier
type: "bool" # Type of expected feedback
description: "Does the answer need more detail?" # Description of the question
required: true # Whether required
optional_comment: # Question identifier
type: "str" # Type of expected feedback
description: "Any additional comments or suggestions" # Description of the question
required: false # Whether required
- agent_class: SearchAgent
agent_name: "Follow-up Research Agent"
agent_description: "Performs additional research based on human input."
super_agents:
- agent_class: FlowSuperAgent
agent_name: "Human in the Loop Advisor"
agent_description: "An advisor that incorporates human feedback into the research process."
config:
goal: "To conduct research, get human feedback, and then write a final report."
agent_list:
# Required. The list of agents to be added in the agent pool. Each agent listed here must be configured under `utility_agents` in the root of the project YAML file.
- agent_name: "Initial Research Agent" # Required.
next_step: # User design. Specifies next steps to run after this agent.
- "Human Reviewer"
- agent_name: "Human Reviewer" # Required.
next_step: # User design. Specifies next steps to run after this agent.
- "Follow-up Research Agent"
- agent_name: "Follow-up Research Agent" # Required. Exit agent that produces the summary output.
To quickly set up a project with a HumanAgent in the Free-form Mode, use the following YAML configuration. To use a custom input method instead for feedback collection, change user_input_method: "Terminal" to user_input_method: "Custom" and define the customized input method in the corresponding python file.
orchestrator:
agent_list:
- agent_name: "Human in the Loop Dinner Planner"
utility_agents:
- agent_class: PlanningAgent
agent_name: "Dinner Planner Agent"
agent_description: "Generates a dinner plan."
- agent_class: HumanAgent
agent_name: "User Feedback Agent"
agent_description: "Asks for user feedback on the proposed dinner plan."
config:
user_input_method: "Terminal" # How the agent collects user feedback
wait_time: 300 # Maximum time in seconds to wait for user feedback
- agent_class: PlanningAgent
agent_name: "Dinner Planner Refinement Agent"
agent_description: "Refine the dinner plan with human feedback."
super_agents:
- agent_class: FlowSuperAgent
agent_name: "Human in the Loop Dinner Planner"
agent_description: "Plans a dinner with initial proposal and refinement after human feedback."
config:
goal: "To generate dinner plan, give an initial plan, get user feedback, and then write a final plan."
agent_list:
# Required. The list of agents to be added in the agent pool. Each agent listed here must be configured under `utility_agents` in the root of the project YAML file.
- agent_name: "Dinner Planner Agent" # Required.
next_step: # User design. Specifies next steps to run after this agent.
- "User Feedback Agent"
- agent_name: "User Feedback Agent" # Required.
next_step: # User design. Specifies next steps to run after this agent.
- "Dinner Planner Refinement Agent"
- agent_name: "Dinner Planner Refinement Agent" # Required. Exit agent that produces the summary output.
Template YAML Configuration for HumanAgent in Structure Mode¶
You can use the following template to create your own HumanAgent in the Structure Mode in your project:
utility_agents:
- agent_class: HumanAgent
agent_name: <Name of the Agent> # Required. The name of the agent
agent_description: <Description of the agent> # Optional. The description of the agent
config:
user_input_method: "Terminal" # How the agent collects user feedback
wait_time: 300 # Maximum time in seconds to wait for user feedback
feedback_interpreter: true # Optional. If true, converts structured feedback into natural language
feedback_schema: # Required
is_answer_correct: # Question identifier
type: "bool" # Type of the expected feedback to the question
description: "Is the answer provided correct?" # Description of the question
required: true # Whether required
Template YAML Configuration for HumanAgent in Free-form Mode¶
You can use the following template to create your own HumanAgent in the Free-form Mode in your project:
utility_agents:
- agent_class: HumanAgent
agent_name: <Name of the Agent> # Required. The name of the agent
agent_description: <Description of the agent> # Optional. The description of the agent
config:
user_input_method: "Terminal" # How the agent collects user feedback
wait_time: 300 # Maximum time in seconds to wait for user feedback