MCP Tools

The benchmark is driven by agent submissions via the submit MCP tool. Each submission advances the benchmark to the next phase. To test your agent, run main.py to start the benchmark, then have your agent submit answers at each phase.

SREGym provides a suite of MCP tools that enable agents to interact with the cluster and benchmark:

Observability Tools

• get_services: Retrieve the list of service names from Jaeger
• get_operations: Query available operations for a specific service from Jaeger
• get_traces: Get Jaeger traces for a given service in the last n minutes
• get_metrics: Query real-time metrics data from Prometheus using PromQL expressions

Cluster Management Tools

• exec_kubectl_cmd_safely: Execute kubectl commands against the Kubernetes cluster. Converts natural language to kubectl commands and executes them. Can get/describe/edit Kubernetes deployments, services, and other components. Takes one query at a time and requires namespace names for most queries
• exec_read_only_kubectl_cmd: Execute read-only kubectl commands (e.g., get, describe, logs, top, events). A restricted version of exec_kubectl_cmd_safely that only allows non-destructive operations
• rollback_command: Roll back the last kubectl command executed with exec_kubectl_cmd_safely
• get_previous_rollbackable_cmd: Get a list of previously executed commands that can be rolled back. When calling rollback_command multiple times, commands are rolled back in the order of this list

Benchmark Interaction

• submit: Submit task results to the benchmark to progress to the next phase

Example Usage

The Stratus agent in clients/stratus demonstrates usages of these MCP tools in an agent.

Testing Your Tools

We recommend using MCP Inspector to test MCP tools before integrating them into your agent.