ACHIEVEMENTS

Highway Chauffeur Project

For PSA: from 2015 to 2019

Prototyping of level 3 autonomous vehicles on separated roadway lanes. Among the first open-road experiments in France.

Our team was in charge of all non-control software aspects: data capture and fusion, decision-making, display. Very ambitious guidelines guided our design, with the goal of achieving a system:

  • Affordable: with a multitude of affordable sensors (4 layer LiDARs, cameras, radar, ultrasonic sensors) instead of a few expensive sensors (like the Velodyne),
  • Lean: 100% onboard processing and decision-making, energy-efficient (consumer-grade CPU/GPU, no 4G/5G),
  • Expert: using explainable technologies and without the need to build/annotate a driving database (no machine learning),
  • Transposable to several prototypes with different sensor architectures,
  • Able to use Semantic HD Mapping,
  • Human-like in its behavior (frequent overtaking, politeness, insertion negotiation) while ensuring comfort and compliance with the highway code (Inter-Vehicular Distance, signs, no overtaking on the right).

Press:

SAM Project

For PSA / STELLANTIS - 2019 to 2023

A continuation of the Highway Chauffeur project.

SAM (Safety and Acceptability of autonomous driving and Mobility) project brings together a consortium of actors in response to the Autonomous Road Vehicle Experimentation (EVRA) call for projects launched by the agency for the environment and energy management (ADEME).

The work involves prototyping level 3 and 4 autonomous shuttles between business areas (mainly highways and some semi-urban areas around arrival zones) on the La Defense / Roissy-Charles-de-Gaulle and Massy / Dourdan routes.

Press:

ACQUISITION AND REPLAY

Recording data during open road experiments and the ability to analyze/play them back is a critical issue in the design of any autonomous prototype. We are developing a tool that addresses this issue in a generic enough way to be used on all prototypes, whether they are trains or cars. It diagnoses the present/absent sensors, robustly records received data, and makes them available in shared memory for real-time exploitation by third-party software, which in turn can produce data to include in the recording. A software library also allows playback of the recorded data.

Acquisition

A robust and independent application to diagnose, collect, and provide real-time data from all types of sensors.

  • Multi-source (TCP/UDP, CAN, cameras, and micro USB),
  • Agnostic (driving autonomous cars, autonomous trains, detection of abandoned packages, ...),
  • Collection and provision of streams in shared memory,
  • Monitoring of sensors (presence/absence) and streams (received/consumed),
  • Backup of streams, with the possibility of incorporating custom packets.

Replay

Recording is good. Replaying is better!

  • Integration into the processing application executed during driving,
  • Deterministic replay, in real-time or accelerated mode,
  • Overlay of fusion/decisions from the driving with those calculated in real-time,
  • Semi-automated processing (blurring, automatic annotations, statistics),
  • Indexing and search in all recordings,
  • Timeline with notes and important events,
  • Bugtracking (tracking, assigning, and editing annotations).

HD MAPPING

A good understanding of the road is necessary for anticipating the intentions of other road users. Sensors do not provide sufficiently accurate information or at a sufficiently long distance, unlike HD Mapping. The use of real-time centimeter-precision HD Mapping required the development of a lightweight mapping format, efficient processing functions, display shaders, as well as conversion/generation/manual input/correction tools. Our mapping tools are generic enough to be used for both autonomous cars and trains, as well as in our urban simulations.

High-Definition Semantic Mapping

Our mapping is based on HERE HD LiveMap, which is automatically converted into our format:

  • Lightweight, suitable for real-time operation
  • Smooth, thanks to the conversion of polylines into polybeziers, and smoothing of curves without compromising accuracy
  • Semantic, which includes types of markings and lanes, connectivity between lanes, speed limits, signs, barriers, etc.
  • Real-time usable, for map-matching, route calculation, trajectory prediction, etc.

Cartography Editor

Unavailable or non-updated map portions can be created or corrected using our editor.

  • Precise, using satellite data (Google, Mappy) or multiple LiDAR scans,
  • Assisted input, by generating polybeziers with continuous curvature passing through the entered waypoints, and cloning polybeziers with constant offset,
  • Semantic, all attributes of the roads can be entered or inherited from the junctions with the base map,
  • With automatic validation of the consistency of the junctions between the entered and non-entered parts,
  • Proven on several tens of km, including tolls, construction zones, airport terminals, tunnels, etc...,
  • Automatable to create map data in our format from OpenStreetMap data.

Streaming

  • Display of a simplified, non-semantic version of the entire map,
  • Dynamic loading from disk of semantic tiles necessary for the immediate route.

POSITIONING

Positioning in HD mapping relies on multiple sources (GPS, camera lines, inertial sensors, etc.). The challenge is to produce smooth positioning (comfortable trajectory tracking) while remaining faithful to the sensors and robust to potential malfunctions (for example: loss of GPS signal when passing under a bridge, erased lines on the road, etc.).

Multi-sensor positioning

The use of lines captured by the camera allows to compensate for the imprecisions of GPS or map data.

  • Temporal tracking of position, altitude, angle, curvature, speed, acceleration, and jerk,
  • Multimodal management to make use of absolute information (GPS), inertial information, or local corrections based on relative information (e.g. camera lines),
  • Temporal coherence to properly integrate signals received in disorder or with a large delay,
  • Interpolation/Extrapolation of tracked variables to obtain information at all times, based on a simplified model (constant acceleration and curvature over a short time).

Robustness and consistency

The level of confidence that can reasonably be given to a sensor can suddenly vary depending on the environment. SpirOps is therefore developing a system:

  • Adaptive, to the presence/absence of specific sensors, and to their relative and absolute precision,
  • Robust, by temporarily ignoring sensors whose provided data dynamics are clearly inconsistent,
  • Consistent, by ensuring that the different variables do not contradict each other (velocity, acceleration, curvature corresponding to what could be derived from positions).

FUSION

An accurate reconstruction of the environment is essential for an autonomous vehicle to make the right decisions. Since sensors have different properties, none of them provides an absolute truth and our job is to make the most of each sensor. This involves analyzing the strengths and weaknesses of different technologies, filtering out their flaws, implementing inter-sensor redundancy when necessary, and ensuring the physical consistency of reconstructed vehicles and their maintenance when they go out of perception zones or are masked by other objects.

Multi-sensor processing and fusion

The requirements of the project led us not to use the high level data provided by each sensor but rather to exploit the lowest level data possible in order to minimize over-aggressive filtering or imperfect tracking, and to make arbitrations as late as possible, thus exploiting the maximum amount of information (better understanding of the road situation, vehicle maneuvers, concordances/divergences between sensors).

  • Aggregation of raw data from LiDARs, cameras, radars, ultrasonic sensors,
  • Intelligent filtering of points/impacts outside the road,
  • Filtering of artifacts related to rain or low sun angles,
  • Creation and tracking of oriented boxes from points/impacts.

Sensor redundancy

Maximizing the use of each sensor to track:

  • The classification of the targets using cameras,
  • The speed of the targets using radars,
  • The contours of the targets using LiDARs,
  • The history of each target to refine its dynamics, predict its trajectory, and improve tracking,
  • The shadow areas to increase the consistency of target appearances/disappearances.

Prediction of vehicle lane change

Anticipating driving decisions can save precious milliseconds:

  • to react,
  • to compensate for the possible absence of turn signals.

DECISION

Autonomy comes from the ability to make decisions, that is, to arbitrate between several possible choices, between several conflicting objectives. When driving, constant trade-offs are necessary between comfort, speed, itinerary compliance, politeness, while ensuring a solid foundation of respect for the rules of the road. In the field of autonomous vehicles, it is particularly important to be able to explain these trade-offs and to offer guarantees on the areas of operation of the decision-making systems.

SpirOps's Drive Oriented Technology

Our patented decision-making technology is perfectly suited for creating a driver AI.

  • Division into mutually independent desires and inhibitions (we drive on the right when possible, we don't cross a solid line, etc.),
  • Respect of the important rules of safety, while taking into account criteria of comfort,
  • Emergence of complex behaviors from a multitude of simple rules,
  • Explainability of each decision in real time,
  • Ease of debugging thanks to determinism and display tools for calculation chains,
  • Flexibility to create multiple driving styles or to add/disable/interchange elementary behaviors without compromising the whole (e.g. driving on the left in England).

SIMULATION / TEST

Ensuring that all the components mentioned above maintain their quality, even when significant modifications are made, is a very important challenge to ensure that the project progresses in the right direction. We develop tools that allow us to check, for each change, the performance on each of the above themes and to easily reproduce any regressions discovered.

Ground truth

Ground truth editor for non-regression tests.

  • Semi-automated input of ground truth boxes and their tracking
  • Comparison to fusion result to track missed/invented surfaces

Build Server

To prevent regressions, we have for each commit:

  • A monitoring of quality indicators,
  • Warnings in case of regression,
  • Direct integration into the source control tool,
  • Advanced display of test results and steps to reproduce them.

Simulation Editor & Behavior Tester

Assists in maintaining existing behaviors and prototyping new ones.

  • From a replay (real situation) or from generated data (imagined situations),
  • Adding or modifying vehicles and behaviors,
  • Expected car's behavior editor for automated non regression testing.