2025

Abstract: Existing object detectors encounter challenges handling domain shifts between training and deployment, particularly under poor visibility conditions like fog and night. Cutting-edge cross-domain object detection methods use teacher-student frameworks and compel teacher and student models to produce consistent predictions under weak and strong augmentations. In this paper, we reveal that manually crafted augmentations are insufficient for optimal teaching and present a simple yet effective framework named Adversarial Defense Teacher (ADT), leveraging adversarial defense to enhance teaching quality. Specifically, we employ adversarial attacks, encouraging the model to generalize on subtly perturbed inputs that effectively deceive the model. To address small objects under poor visibility conditions, we propose a Zoom-in Zoom-out strategy, which zooms-in images for better pseudo-labels and zooms-out images and pseudo-labels to learn refined features. Our results demonstrate that ADT achieves superior performance, reaching 54.5% mAP on Foggy Cityscapes, surpassing the previous state-of-the-art by 2.6% mAP.

Abstract: Cameras and LiDAR are among the most commonly used sensors for autonomous driving applications; often, a combination of both is used. Although both sensors have attracted considerable interest, camera-based approaches are the most mature, thanks partly to industry efforts to optimize the underlying processing pipelines. In contrast, LiDAR-based perception methods still rely on the research community’s efforts to develop and support the processing libraries that make the methods possible. This imbalance between the perception methods of the two sensor modalities often results in a high GPU computational load and long inference times for the processing of LiDAR point clouds, which may be too demanding for the embedded devices in autonomous vehicles. In this work, we explore point cloud sub-sampling using prior knowledge to reduce the computational burden of LiDAR processing. In particular, we show that with perfect prior knowledge, the accuracy of modern 3D object detectors is not affected even when large portions of point clouds are removed. For practical applications, we instead use 2D object detections of surrounding cameras to determine relevant regions. In extensive experiments, we demonstrate that our proposed framework can reduce the computational load and inference time of 3D object detectors while maintaining high detection performances. Keywords: Camera-based Prior Knowledge, Efficient Perception, LiDAR 3D Object Detection, Point Cloud Sub-sampling

Abstract: Neuromorphic computing, inspired by the structure and functionality of the human brain, offers a transformative potential for advancing automated driving systems. This review examines the role of neuromorphic computing in overcoming current limitations in AI-based perception systems, particularly with respect to energy efficiency, real-time processing, and robustness. Using spiking neural networks and event-driven architectures, neuromorphic systems enable more efficient computation than traditional AI models, which require significant computational resources and power. This work explores specific scenarios where neuromorphic computing outperforms traditional methods and highlights how neuromorphic hardware can improve data integration, reduce power consumption, and increase the reliability of automated driving systems. This review concludes that neuromorphic computing is not only a viable alternative, but a superior approach for future advances in automated driving technology, offering a path to more efficient and adaptive systems.

Abstract: Safety is the most critical aspect to address for the real-world deployment of robotic platforms, such as autonomous driving systems. While learning-based approaches like reinforcement learning have gained popularity for managing real-world complexity, they often lack transparency and safety awareness. In this paper, we aim to advance the development of safety-aware AI agents by presenting a framework for estimating collision probability distributions, which can be integrated into the decision-making process of reinforcement learning agents. To this end, we provide a thorough definition and motivation for incorporating safety awareness, highlighting its importance for reliable and interpretable decision-making. Finally, we demonstrate how these collision probabilities can be effectively integrated into decision-making by incorporating them into a value function, enabling safety-aware reinforcement learning.

Abstract: Road traffic injuries are the leading cause of death for people aged 5-29, resulting in about 1.19 million deaths each year. To reduce these fatalities, it is essential to address human errors like speeding, drunk driving, and distractions. Additionally, faster accident detection and quicker medical response can help save lives. We propose an accident detection framework that combines a rule-based approach with a learning-based one. We introduce a dataset of real-world highway accidents, featuring high-speed crash sequences. It includes 294,924 labeled 2D boxes, 93,012 labeled 3D boxes, and track IDs across 48,144 frames, captured at 10 Hz using four roadside cameras and LiDAR sensors. The dataset covers ten object classes and is released in the OpenLABEL format. Our experiments and analysis demonstrate the reliability of our method.

Abstract: One of the challenges for the assessment of automated driving systems (ADS) is the definition of reasonable release thresholds. Scenario-based reference models, like the ”competent and careful human driver” introduced in UN regulation No. 157, can be integrated into a larger scenario-based testing process within a safety assessment program for ADS. This article extends the goal structuring notation (GSN) developed in the VVM project by a practically applicable methodology to derive scenario-based “competent driver” models from human reference driver data, which can serve as scenario-based assessment criteria. Based on an established role and procedures for safe on-road testing, the in-vehicle fallback test driver (IFTD), including presence of an in-cab safety conductor (SC) and adhering to a variety of safety management controls, is used as a human reference driver representing a competent and careful driver. The model development methodology is piloted using three collected on-road driving datasets.

Abstract: This study evaluates the HMIs developed for the operation center of the CityBot as part of the BMDV-funded Campus FreeCity project. The dispatcher and remote operator HMIs were assessed in a field study to determine acceptance, perceived control and safety, workload, usability, and immersiveness. Standardized questionnaires and interviews were employed to gather data from 20 participants with minimal experience in remote operation technologies. Results for the dispatcher HMI indicated high acceptance, good usability, and low to moderate workload, with participants appreciating the overview map and color-coded vehicle status indicators. Suggestions for improvement included better error message handling, enhanced live video feeds, and shortcut functions to reduce user effort. For the remote operator HMI, acceptance and usability ratings were moderate, with higher workload compared to the dispatcher role. Participants highlighted challenges in vehicle control and spatial awareness. Recommendations included augmented reality features and additional auditory feedback to improve navigation.

Abstract: Research on take-overs in SAE-Level 4 automated driving is crucial for ensuring safety during the utilization of these driving functions, as take-overs may still occur, e.g., when certain traffic situations require it. Central questions revolve around the vehicle system’s decision procedure in these situations to request the human to take over and possible alternatives. The decision bases on the required take-over time in dependence of the current non-driving related activity and the time left until the system boundary is reached. The presented models HoMoTo and SAM are tools to support this decision and to enhance safety during the take-over process.

Abstract: This paper demonstrates the potential of implementing the lateral Motion Sickness Dose Value (MSDV) model in motion planning to reduce motion sickness in automated vehicles. In our study, we evaluate motion sickness ratings of participants experiencing two different trajectories: a standard trajectory, with larger lateral accelerations, and a comfort trajectory, with reduced lateral accelerations. We fit the MSDV model to the motion sickness ratings of passengers who experienced the standard trajectory and use this fitted model to assess the comfort trajectory regarding motion sickness. A comparison of this MSDV model-based assessment with the passenger’s motion sickness ratings of the comfort trajectory shows that a comparable fit with minimal loss in accuracy is achieved. Thus, our study demonstrates that the MSDV model can be used to predict motion sickness symptoms. These findings highlight the potential of utilizing the MSDV model in trajectory planning, as a reduced MSDV correlates with reduced passenger symptoms.

Abstract: Scenarios play an important role in the development and release of automated vehicles (AV), as they form the basis for system development, testing and ensuring safety in various situations, including potentially hazardous ones. However, there are a variety of definitions and applications of scenarios, often with different accuracy requirements. The relationships between the different scenario concepts in use case definition, Hazard Analysis and Risk Assessment (HARA), Safety Analysis and Risk Assessment (SARA), and testing often remains vague. This paper aims to develop a consistent definition and taxonomy of scenarios and assign them to different systems engineering processes.

Abstract: With Automated Valet Parking (AVP), a vehicle is guided to perform a parking usecase. AVP systems can be integrated either in the vehicle or in the infrastructure of the parking facility. Obtaining both vehicle-centered and infrastructure-based systems offers a redundant AVP solution. However, developing each system separately is time-consuming and prevents the knowledge transfer. Therefore, leveraging existing AVP systems in the development process is required. In this work, we introduce a systematic approach to develop an infrastructure-based AVP system by adapting an existing vehicle-centered system. We followed the V-model to define the development stages, which facilitates the traceability of the complete process.

Abstract: Designing, assuring and releasing safe automated vehicles is a highly interdisciplinary process. As complex systems, automated driving systems will inevitably be subject to emergent properties, i. e. the properties of the overall system will be more than just a sum of the properties of its integrated elements. Safety is one example of such emergent properties. In this regard, it must be ensured that effects of emergence do not render an overall system that is composed of safety-approved sub systems unsafe.
The key challenges in this regard are twofold: Regarding the interdisciplinary character of the development and assurance processes, all relevant stakeholders must speak a common language and have a common understanding of the key concepts that influence system safety. Additionally, the individual properties of system elements should remain traceable to the system level. Model-Based Systems Engineering (MBSE) provides an interdisciplinary mindset, as well as methods and processes to manage emergent system properties over the entire system lifecycle. By this, MBSE provides tools that can assist the assurance process for automated vehicles. However, concepts from the domain of MBSE have a reputation for not being directly accessible for domain experts who are no experts in the field of Systems Engineering.
This paper highlights challenges when applying MBSE methods to the design and development of automated driving systems. It will present an approach to create and apply domain-specific SysML profiles, which can be a first step for enhancing communication between different stakeholders in the development and safety assurance processes.

Summary: The transition from traditional hardware-centric vehicles to software-defined vehicles is largely driven by a switch to modern architectural patterns of software, including service orientation and microservices. Automated driving systems (ADS), and even more so, Cooperative Intelligent Transport Systems (C-ITS), come with requirements for scalability, modularity, and adaptability that cannot be met with conventional software architectures. The complexity and dynamics of future mobility systems also suggest to employ ideas of the event-driven architecture paradigm: distributed systems need to be able to detect and respond to events in real-time and in an asynchronous manner. In this paper, we therefore stress the importance of data-driven event detection in the context of ADS and C-ITS. First, we propose a classification scheme for event-detection use cases. We then describe a diverse set of possible use cases and apply the classification scheme to a selection of concrete, innovative examples. Last, we present a modular event detection software framework that we publish as open-source software to foster further research and development of complex C-ITS use cases, but also for robotics in general. The framework is published at github.com/ika-rwth-aachen/event_detector.

Abstract: In this paper, we enhance the BEVFormer for 3D object detection by using a new multiscale cross-attention approach. We show that our proposed architecture provides an improved performance and requires less computation. We introduce a layer-wise upscaling of the BEV grid features and design them to align with the image features of matching spatial resolution. Moreover, we reduce the number of parameters of the initial BEV grid to prevent overfitting. The proposed enhancements are vital for making automated driving systems more efficient and reliable.

Abstract: For the real-world deployment of automated vehicles general trajectory planning methods are required. The most versatile planning approaches, such as dynamic programming, consider many distinct options, which increases their computational effort significantly. Therefore, previous works often use heuristic search or significantly limit the amount of behavior options. However, due to recent advances in graphic processing units (GPUs) the available computational resources have increased tremendously. Thus, this paper reevaluates the use of the versatile dynamic programming method under consideration of recent hardware. Additionally, an exemplary implementation and evaluation on challenging scenarios such as unsignalized intersections or unprotected turns is provided. The source code will be released as part of our trajectory planning library under https://github.com/uulm-mrm/tpl.

Abstract: Using HD maps directly as training data for machine learning tasks has seen a massive surge in popularity and shown promising results, e.g. in the field of map perception. Despite that, a standardized HD map framework supporting all parts of map-based automated driving and training label generation from map data does not exist. Furthermore, feeding map perception models with map data as part of the input during real-time inference is not addressed by the research community. In order to fill this gap, we present lanelet2_ml_converter, an integrated extension to the HD map framework Lanelet2, widely used in automated driving systems by academia and industry. With this addition Lanelet2 unifies map based automated driving, machine learning inference and training, all from a single source of map data and format. Requirements for a unified framework are analyzed and the implementation of these requirements is described. The usability of labels in state of the art machine learning is demonstrated with application examples from the field of map perception. The source code is available embedded in the Lanelet2 framework under https://github.com/fzi-forschungszentrum-informatik/Lanelet2/tree/feature_ml_converter.

Abstract: We study the Out-of-Distribution (OoD) generalization ability of three SotA trajectory prediction models with comparable In-Distribution (ID) performance but different model designs. We investigate the influence of inductive bias, size of training data and data augmentation strategy by training the models on Argoverse 2 (A2) and testing on Waymo Open Motion (WO) and vice versa. We find that the smallest model with highest inductive bias exhibits the best OoD generalization across different augmentation strategies when trained on the smaller A2 dataset and tested on the large WO dataset. In the converse setting, training all models on the larger WO dataset and testing on the smaller A2 dataset, we find that all models generalize poorly, even though the model with the highest inductive bias still exhibits the best generalization ability. We discuss possible reasons for this surprising finding and draw conclusions about the design and test of trajectory prediction models and benchmarks.

Abstract: This paper presents an AI-based solution to promote level-compliant behavior in automated vehicles. By integrating driver monitoring and human machine interaction, the system detects and addresses non-compliant behavior through multimodal interactions. A user study with 61 participants evaluated the system's usability and effectiveness, revealing high comprehensibility and usability scores. The AI agent, trained using reinforcement learning, adapts to various driving contexts and user states, ensuring safe and comfortable interactions. The findings highlight the importance of adaptive, multimodal approaches in enhancing driver compliance and safety in partially automated driving.

Abstract: Self-assessment plays a critical and important role toward safe and robust autonomous driving. Current self-assessment approaches in this area focus on individual modules at specific positions within the autonomous driving stack. The literature lacks a unifying framework to combine various self-assessment information. Hence, this work provides a comprehensive self-assessment framework for autonomous driving stacks, combining and unifying existing selfassessment methods. For this framework, we propose using subjective logic as an interface to standardize the output of self-assessment modules. This allows the combination of different modules and their use in subsequent processing modules. Our approach can be deployed to existing autonomous vehicle software stacks without imposing any requirements on their functional parts, enabling easy integration. With this framework, we are aiming to contribute to the improvement of safety and reliability in autonomous driving.

Abstract: Connected driving simulators are effective tools for collecting driving behavior data. This work proposes an experimental design to create natural driver interaction scenarios, verified through a pilot and main study. The results, showing an 88% interaction success rate, demonstrate the framework’s effectiveness and offer practical insights for researchers in driving behavior and human factors research

Abstract: Modular automated driving systems commonly handle prediction and planning as sequential, separate tasks, thereby prohibiting cooperative maneuvers. To enable cooperative planning, this work introduces a prediction model that models the conditional dependencies between trajectories. For this, predictions are generated by a microscopic traffic simulation, with the individual traffic participants being controlled by a realistic behavior model trained via Adversarial Inverse Reinforcement Learning. By assuming various candidate trajectories for the automated vehicle, we generate predictions conditioned on each of them. Furthermore, our approach allows the candidate trajectories to adapt dynamically during the prediction rollout. Several example scenarios are available at https://conditionalpredictionbysimulation.github.io/