Leap Icon

Continuously Learning, Adapting and Improving:
A Dual-Process Approach to Autonomous Driving LeapAD


Anonymous Author(s)

Abstract

Autonomous driving has advanced significantly due to sensors, machine learning, and artificial intelligence improvements. However, prevailing methods struggle with intricate scenarios and causal relationships, hindering adaptability and interpretability in varied environments. To address the above problems, we introduce LeapAD, a novel paradigm for autonomous driving inspired by the human cognitive process. Specifically, LeapAD emulates human attention by selecting critical objects relevant to driving decisions, simplifying environmental interpretation, and mitigating decision-making complexities. Additionally, LeapAD incorporates an innovative dual-process decision-making module, which consists of an Analytic Process (System-II) for thorough analysis and reasoning, along with a Heuristic Process (System-I) for swift and empirical processing. The Analytic Process leverages its logical reasoning to accumulate linguistic driving experience, which is then transferred to the Heuristic Process by supervised fine-tuning. Through reflection mechanisms and a growing memory bank, LeapAD continuously improves itself from past mistakes in a closed-loop environment. Closed-loop testing in CARLA shows that LeapAD outperforms all methods relying solely on camera input, requiring 1-2 orders of magnitude less labeled data. Experiments also demonstrate that as the memory bank expands, the Heuristic Process with only 1.8B parameters can inherit the knowledge from a GPT-4 powered Analytic Process and achieve continuous performance improvement.

LeapAD Architecture

In LeapAD, the Scene Understanding module mimics human driving attention, analyzing surrounding images and selectively focusing on key objects influencing decision-making. It provides detailed descriptions and behavioral reasoning of these objects, facilitating informed subsequent decisions. The Analytic Process then uses an LLM to accumulate experience in driving analysis and decision-making, conducting reflections on accidents. The experience is stored in memory and transferred to a lightweight language model, forming our Heuristic Process for quick responses and continuous learning.

The Heuristic Process in LeapAD performs closed-loop decision-making using a few-shot strategy. It is designed for enabling instant decision-making within the vehicle. The Heuristic Process relies on the knowledge transferred from the Analytic Process to make quick and efficient decisions. This lightweight language model ensures rapid responses and adaptability in various driving scenarios, maintaining a high level of performance with minimal computational resources.

The Analytic Process in LeapAD is designed for thorough analysis and reasoning. It processes more complex scenarios and builds a comprehensive memory bank of high-quality driving decisions. The Analytic Process accumulates experience through continuous learning and self-reflection, analyzing accidents, and updating the memory bank. This accumulated knowledge is then transferred to the Heuristic Process through supervised fine-tuning (SFT), ensuring the entire LeapAD continuously improves and adapts to new driving environments and challenges.

Case Studies

BibTeX

@article{anonymous2024leapad,
    title={Continuously Learning, Adapting and Improving: A Dual-Process Approach to Autonomous Driving}, 
    author={Anonymous},
    year={2024},
}