Autonomy integration

Why this matters

A flight controller is excellent at holding attitude and position hundreds of times a second, but it lacks the power to run AI models or build maps. That work moves to a companion computer, such as a Jetson or Raspberry Pi board that talks to the flight controller continuously, like a co-pilot giving directions to a pilot who still holds the controls. If the two disagree about direction, or the co-pilot falls silent in mid-air, the consequences can be serious. This module practises connecting the two sides correctly and safely.

The whole course uses one hypothetical case: an autonomous survey drone with a companion computer that detects obstacles with a camera and a range sensor, tested from SITL through to the field. The Python code of every module can be downloaded from /downloads/uat-322/.

A clear division of work

• The flight controller estimates state with its EKF, controls position and attitude, and always keeps authority over failsafes, whatever the companion computer commands
• The companion computer perceives the environment, plans paths and sends setpoints (desired position, velocity or yaw) to the flight controller
• The ground station is used to configure, monitor and abort missions

There are two main links:

ArduPilot also has a built-in ROS 2 interface (AP_DDS). The PX4 documentation recommends ROS 2 Jazzy on Ubuntu 24.04. With the MAVSDK library for Python, note that the mavsdk package version 4 is a new API that differs from older examples; the older examples using mavsdk_server have moved to the mavsdk-grpc package, so pin versions to match the documentation you read.

Directions must agree

ROS uses ENU (x east, y north, z up) under REP 103, but PX4 uses NED (x north, y east, z down). Send a ROS position to PX4 without converting it, and “climb 3 metres” becomes “descend 3 metres”.

import math


def enu_to_ned(east, north, up):
    return north, east, -up


def yaw_enu_to_ned(yaw_enu):
    yaw = math.pi / 2 - yaw_enu
    return math.atan2(math.sin(yaw), math.cos(yaw))   # wrap into −π to π


print("target NED:", enu_to_ned(10.0, 5.0, 3.0))
for deg in (0, 90, 180):
    print(f"yaw ENU {deg:>3}° -> NED {math.degrees(yaw_enu_to_ned(math.radians(deg))):6.1f}°")

target NED: (5.0, 10.0, -3.0)
yaw ENU   0° -> NED   90.0°
yaw ENU  90° -> NED    0.0°
yaw ENU 180° -> NED  -90.0°

When the companion computer falls silent

In PX4 Offboard mode, the companion computer must send a continuous proof-of-life signal of at least 2 Hz, and setpoints must be streaming before switching into the mode. If the signal is lost for longer than COM_OF_LOSS_T (default 1.0 s), PX4 leaves Offboard and does what COM_OBL_RC_ACT specifies (Position mode by default). In ArduPilot Guided mode, velocity commands should be re-sent every second, and the vehicle stops if none arrives for 3 seconds.

stamps = [round(0.1 * k, 1) for k in range(31)] + [round(4.5 + 0.1 * k, 1) for k in range(11)]
LOSS_T = 1.0            # s, the default of COM_OF_LOSS_T
print("setpoints in the first second:", sum(s < 1.0 for s in stamps))

last, failsafe, i, t = None, None, 0, 0.0
while t <= 5.5:
    while i < len(stamps) and stamps[i] <= t + 1e-9:
        last = stamps[i]
        i += 1
    if last is not None and failsafe is None and t - last > LOSS_T:
        failsafe = t
    t = round(t + 0.05, 2)
print(f"failsafe at {failsafe} s (stream stopped after 3.0 s)")

setpoints in the first second: 10
failsafe at 4.05 s (stream stopped after 3.0 s)

The latency budget

Every stage from camera to flight controller takes time; the total is the latency, during which the drone keeps moving.

stages = {"camera frame": 33, "detector": 45, "planner": 12, "DDS link": 4, "FC control": 3}   # ms
total = sum(stages.values())
speed = 5.0   # m/s
print(f"total latency {total} ms; at {speed} m/s the drone moves {speed * total / 1000:.2f} m before reacting")

total latency 97 ms; at 5.0 m/s the drone moves 0.48 m before reacting

The stage times are assumptions; measure them on your own hardware (module 4). The budget shows how much safety distance to allow, and which stage to shorten first; here the detector takes longest.

Module lab

Common mistakes

Summary

• The flight controller keeps stability and failsafe authority; the companion computer perceives, plans and sends setpoints
• PX4 connects to ROS 2 through uXRCE-DDS, and both PX4 and ArduPilot support MAVLink
• ROS uses ENU and PX4 uses NED; convert both position and yaw
• Offboard needs at least 2 Hz and exits when the signal is lost for longer than COM_OF_LOSS_T
• The latency budget shows how far the drone travels before reacting

Check your understanding

1. What is the ENU point (east 4, north −2, up 6) in NED?
2. What is an ENU yaw of 45° in NED?
3. At least what rate of setpoints does PX4 Offboard mode require?
4. With 120 ms total latency at 6 m/s, how far does the drone travel before reacting?
5. Why should failsafes live in the flight controller rather than the companion computer?