MCP Servers for Agentic Vehicle Control

Language models are good at intent and bad at protocol. Connecting one to a vehicle the naive way means letting it emit MAVLink and hoping: no schema, no bounds, no notion of which commands are even legal in the current flight mode. The failure modes are not subtle when the actuator is a motor.

The context needed to decide well also lives somewhere else entirely — what the terrain under the vehicle looks like, what the camera can see, what the vehicle is doing right now. Without those, an agent is reasoning about a vehicle it cannot perceive.

Four MCP servers, each owning one domain and exposing typed tools instead of a free-form API surface. MAVLinkMCP speaks MAVLink over TCP and UDP and covers the real control set: arm and disarm, takeoff, land, return-to-launch, flight-mode changes, NED position control in absolute, relative and body frames, GPS waypoint navigation, home-position and EKF-origin management, and data-stream rate configuration.

The decision that matters is the split between tools and resources. Telemetry — position, attitude, IMU, battery, VFR HUD, raw GPS — is exposed as resources, which the model reads without side effects. Anything that changes vehicle state is a tool, with an explicit signature. That boundary is what makes the system safe to hand to an agent, and it does not come for free with the protocol.

The other three servers supply context. terrainMCP answers elevation questions against DEM data — point elevation, profiles along a path, area statistics. visionMCP runs YOLOv8 detection and video tracking with GPU acceleration and multi-model support. cameraMCP handles GStreamer RTSP streaming, recording, snapshots and multi-camera auto-configuration.

On top of that sits an experiment harness, because “the agent flew it” is not a result. It runs a fixed command array against an ArduPilot SITL instance in Docker, sweeping three axes: model, compute tier (cloud, local, edge) and prompt strategy (zero-shot and two engineered system prompts). The model list spans Claude Opus 4, Sonnet 4 and Haiku 3.5, GPT-4.1, 4o and 4o-mini, DeepSeek-V3 and Gemma 2 down to a 2B edge variant.

Per command it records time to first response, time to first action, end-to-end duration, prompt and completion tokens, cost, the tools actually called against the tools expected, a tool-match score, task success, and the number of LLM turns — with a smart-plug power monitor alongside so the edge tier can be compared on energy rather than only on latency. Analysis scripts emit the figures and LaTeX tables directly.

A vehicle an assistant can operate through a stable, inspectable interface, with each capability installable on its own — the servers are separate Python packages rather than one monolith, so a deployment takes only the surface it needs.

The harness is built and smoke-tested against SITL — the runner, instrumentation, power monitoring and analysis path all execute end to end — but the full model-by-tier-by-prompt matrix has not been run yet, so there are no comparative numbers to report. What exists today is the measurement rig, not the measurement.

The pattern is more useful than the code. The interesting engineering in agentic systems is not prompting, it is deciding what the model is allowed to touch and writing that down as a typed boundary. All of it assumes the agent will eventually try the wrong call, and is shaped so the wrong call is either unavailable or obviously wrong.