Energieeffiziente Längsführung vernetzter und automatisierter Fahrzeuge mittels Reinforcement Learning

  • Energy-efficient longitudinal control of connected and automated vehicles using reinforcement learning

Wegener, Marius; Andert, Jakob Lukas (Thesis advisor); Abel, Dirk (Thesis advisor)

Aachen (2022)
Dissertation / PhD Thesis

Dissertation, Rheinisch-Westfälische Technische Hochschule Aachen, 2022


This thesis investigates the application of reinforcement learning for the realization of predictive and efficient longitudinal control of automated vehicles in urban traffic. A simulation environment is built to train and test reinforcement learning agents under conditions as realistic as possible. The simulation environment uses the microscopic traffic simulator SUMO and the framework FLOW to integrate reinforcement learning algorithms. As part of this dissertation, the environment is extended to include a powertrain model as well as a probabilistic driver model to enable a realistic evaluation of energy savings in a stochastic environment. The "Alleenring" in the city of Aachen is defined as a reference scenario to evaluate the effectiveness of the learned driving strategy. The parameterization of the simulation scenario is validated with measurements of the real route so that the evaluation of the driving strategy can be performed under realistic conditions. In a simplified training environment, reinforcement learning agents are trained with different reward functions and subjected to an initial evaluation. Thus, a reward function is identified that achieves a better resolution of the trade-off between travel time and energy demand than a rule-based reference strategy. A detailed analysis of the strategy shows that the reinforcement learning agents can predictively and efficiently steer the vehicle through the training scenario. The highest energy savings can be achieved when traffic density is low and traffic signals along the route of the ego-vehicle are unfavorably switched. By transferring the learned strategy to the realistic reference scenario "Alleenring", an energy saving of 25,6% is achieved with a reduction of the average speed by 6,8%. It is shown that reinforcement Learning is suitable for learning efficiency-increasing driving strategies on its own and that those strategies can also be transferred to new environments. Finally, the effects of the driving strategy on the immediate environment in urban traffic are considered. Here it can be shown that with a market penetration of 5%, there is only a small impact in the form of minor reductions in energy consumption and average speed. Through an adaptive reward function that takes following vehicles into account, however, this effect can be compensated, but this leads to a reduction of the potential for the ego-vehicle. If there are several reinforcement learning agents in the same road network an increase in travel times for all road users can be observed with market penetrations greater than 20%. With a higher market penetration of connected and automated vehicles, driving strategies must be developed that explicitly avoid such effects.