Asteroid makes your AI agent more reliable and effective by supervising it while it’s running. Using humans, AI, or code we help you monitor every action during experimentation or deployment:

  • During development, Asteroid makes it easy to rapidly catch failures and improve your agent.
  • In production, Asteroid is your safety layer that will continue to monitor your agent and intervene to prevent bad actions, correct mistakes, terminate the agent or escalate to a human reviewer.

Asteroid is not an agent framework; it hooks into your existing agent.

​
Asteroid Demo

​
Use Cases

Asteroid is designed to be used in a wide range of use cases to enhance reliability, safety and performance of agentic systems, including but not limited to:

​
Tool Call Supervisors

Asteroid has an API that agents can call to submit actions for approval. The request is routed via a predefined list of supervisors. When presented when with an action, a supervisor can:

  • approve The action is safe and the agent can proceed.
  • reject The action is unsafe and should not be executed, but the agent can continue.
  • modify The action has been approved with modifications.
  • escalate The action should be escalated to the next supervisor.
  • terminate The agents execution should be terminated.

Finally, after the supervisors have made their decisions, the agent receives a response indicating the outcome of the approval process, which is either approve, reject, modify, or terminate. It is up to the agent’s source code to handle these responses appropriately.

​
Parallel Supervision Chains

Supervisors can be grouped into supervision chains which are executed in parallel. This allows for a fast and efficient approval process, where multiple supervisors can approve or reject an action at the same time. As a dummy example, you can create 2 supervisor chains that are attached to the send_email() tool:

Supervision Chains Example - Send Email

In this example, both supervisor chains need to approve the action before the agent can proceed. You can configure many more supervisors and chains as you see fit with different policies.

Chain 1 - Check Email Domains and Content

1

Check Email Domains Supervisor

Deterministic Supervisor that only allows emails to be sent to whitelisted domains: ['my-company.com', 'asteroid.ai']

2

Check Email Content Supervisor

LLM Supervisor that checks the email content for sensitive information and ensures it aligns with company policies.

Chain 2 - Check Email Urgency and Risk

1

Check Email Urgency Supervisor

LLM Supervisor that evaluates the urgency and importance of the email based on its content.

2

Check Email Risk Human Supervisor

Human Supervisor to review and approve emails flagged as high-risk or requiring human judgment.

​
Supervisor Types

Asteroid currently provides three main types of supervisors for handling tool call supervision: Human, LLM (Large Language Model), and Deterministic. Each supervisor has a different use cases, can be highly customised or you can create completely custom supervisors.

Human Supervisor

A human reviews the tool call suggestion made by the agent and returns an approval decision.

  • —> Allows manual review and decision-making by a human.
  • —> Useful for high risk scenarios requiring human judgment and oversight.
  • —> Supports 1 or more tool call suggestions per approval, allowing a human to choose the best option.

LLM Supervisor

A Large Language Model with a customisable prompt analyses the tool call suggestion made by the agent and returns an approval decision.

  • —> Automates decision-making
  • —> Ideal for high-volume or lower risk routine tasks where human intervention is not necessary.

Code Supervisor

A code-based supervisor can run arbitrary code to determine the approval decision. This could be useful for simple approval processes that are known ahead of time, such as whitelisting emails, commands, or URLs, or for more complex approval processes like those requiring custom classifiers or business logic.

​
Custom Supervisors

Asteroid allows for custom supervisors to be created and plugged in to the approval process. This allows for a wide range of use cases, including but not limited to:

  • Integrating with existing approval workflows
  • Implementing specialized logic for certain types of actions

​
LLM Response Mocking

You can reuse previous execution logs to mock function responses, which is useful for testing and reproducing results. When mocking is enabled, the function will not execute its actual logic. Instead, it will behave according to the specified mock policy. You can mock function responses using different MockPolicy options.

  • SAMPLE_LIST: Returns a random element from mock_responses.
  • SAMPLE_RANDOM: Creates a random value matching the return type.
  • SAMPLE_LLM: Uses an LLM to generate a response.
  • SAMPLE_PREVIOUS_CALLS: Samples responses from previous executions.

​
Architecture

Asteroid uses a simple client-server architecture. The client is a React application that displays information related to agents and their supervision to human operators. The server is a Go application that handles the API requests from the agent and routes them to the appropriate supervisor. Supervisors can be configured via the UI.

Consider the following diagram for a high level overview of how Asteroid works in the case of the Human Supervisor:

In this case, the agent submits a review to the server, which is then displayed in the UI for a human reviewer to approver or reject. The reviewer’s response is then submitted back to the server, which updates the status of the review accordingly.

Quickstart