The right simulator for the right question
Updated: 5 days ago
The core idea behind the Badb project is that UAVs and UGVs need to be able to operate with a higher degree of autonomy - less reliant on receiving external signals from satellites (GNSS/GPS) and human operators, and with less need to transmit, making it easier for them survive in contested spaces.
The human operator retains authority over objectives, but the machine needs to acquire a capacity for perception and navigation, and to make small tactical decisions on how best to meet the operator's objectives. On a technical level, this requires training a lot of machine learning models, which in turn requires data: in my last Badb post, I explained the value of simulation in shortening our learning loop: although real-world data is vital, we should not have to wait for an adversary to show us something new before learning how to respond. Simulations let us train AI models for perception, navigation and control, and allow us to test the full software stack in the data centre, before we fly - in this field, "software crash" is not a metaphor.
So, what do we mean by simulation?
A simulation might be a spreadsheet, a stripped-down 2D world for learning pursuit tactics, or a photorealistic forest with vehicle physics, cameras and controls. Our rule is simple: decide what we need to learn, then make the simulation "as simple as possible, but no simpler".
Strategy: one step from a spreadsheet
At Rigr, we have put two small "arcade" simulations online: an air-defence allocation simulator and a convoy-protection simulator. They are interactive and deliberately abstract, more an aid to discussion than a means of answering serious questions.
One asks how to divide limited air-defence resources between fixed sites and mobile units. The other asks how to protect a small armoured patrol from drone attack in close terrain. Move the sliders to see how earlier warning, concentrated attack or a shorter sensor horizon changes the outcome.
These scenarios are hugely simplified, with entirely imaginary geographies, sensors and platforms, which makes them easy to explain, and also allows us to run tens of thousands of simulated engagements in seconds. Closer to animated spreadsheets than digital twins, nevertheless very simple models can help us explore the binding constraints in questions of strategy or procurement; it can help us figure out what kind of drone we should be simulating in more detail.
Tactics: simple worlds for fast learners
Machines can also learn in simple simulations. The accompanying video shows a stripped-down 2D pursuit environment. An interceptor must keep a Shahed-type one-way attack drone inside its field of view. Here, the target has a rear-facing defensive camera; once the Shahed sees the interceptor, it can manoeuvre to break the lock.
The graphics are rudimentary because they are not part of the simulation, strictly a "for the humans only" playback after the fact. The agents never see them, just the read-outs from sensors we can simply assume for the purposes of the simulation. The simulator includes only the core "facts" - relative speed, manoeuvrability, fields of view - that the machine needs so as to learn how to be good at pursuit (or evasion - we train both strategies at once, since you need a clever "prey" if you want to train a smart "hunter").
We can vary sensor range and resolution, camera geometry, speed and turning performance while training pursuit and evasion policies. A learning system can experience - needs to experience - an enormous number of encounters. We can also ask whether a wider field of view compensates for poorer manoeuvrability - so the outcome might be a smarter interceptor, or it might be a design change to the physical platform.
Synthetic worlds: where appearance and physics matter
This is what most people picture when they hear simulation: a rich 3D world built with much the same technology as a high-end computer game.
We choose the vehicles, place every tree and structure, set the weather and time of day, and control the cameras. Because the environment is wholly synthetic, we know the ground truth: every object's position and motion, and what each sensor should see.
These environments are useful when both appearance and physics are part of the question. We can generate training data, test perception, navigation and control, reproduce awkward edge cases, and change one variable at a time.
Captured worlds: real places, virtual missions
The most ambitious simulations we are building in Badb Loop combine realistic vehicle and sensor behaviour with detailed 3D captures of wide areas.
Instead of something resembling a boreal forest, we can use aerial and ground surveys to "import" an actual area of intended operations. Trees and buildings occupy their real positions, are a really close match in size and shape for the real versions, and the scene can appear photorealistic. The real world has a complexity and richness that is hard to capture when we try to build the world ourselves.
This lets us exercise perception, navigation and control together in a reconstruction of a real place. For GNSS-denied visual navigation, a system must relate what its camera sees to mapped terrain. A generic forest, however convincing, cannot test whether a camera view matches a particular map.
Interestingly, it is quite possible to capture an entire country in this way, a real advantage for a defending army, whose UAVs/UGVs/USVs can then fight and re-fight scenario after scenario on the home ground, in a simulation that is within 10s of centimetres of reality: machines really can acquire local knowledge, and exploit the details of the terrain to maximum effect.
There is an interesting inversion: simulations for strategy, force structure and procurement may be the simplest - you can do it in a spreadsheet; those for the lowest-level engineering questions may be the most complex.
The map is not (quite) the territory
Modern simulations are extraordinary: we can capture every road and lake, even every tree in every forest in an entire country, and build a virtual test range that includes every metre of ground we need to protect. We can run millions of scenarios and produce marvellous charts.
However, there is always a "reality gap" - real sensors meet glare, fog, snow, vibration and noise, real actuators fail to live up to their simulated counterparts, our real adversaries turn out to be more cunning or more erratic than their simulated selves. So, field testing is not a final validation step, but part of the loop, making sure our simulations aren't castles in the clouds. So, the Badb consortium continues to conduct regular field tests in participating countries.
