Cognitive Driving Enhancement: A Multimodal Interaction Framework

Driving is the coordinated operation of mind and body for movement of a vehicle, such as a car, truck, or bus. Driving, being considered as an everyday activity for most people still has an issue of driver safety. Over the 20 years from 1980 to 2000, the number of licensed driv-ers in the U.S. has increased 23.7%, from about 154.0 mil-lion to 190.6 million. Total annual mileage traveled an-nually in the U.S. increased 28.9% from 1990 to 2000 and reached 2,767 billion miles in 2000 [1]. Over the years, we have seen emerging technology for safer driving.

Elec-tronic stability control, collision avoidance systems, in-telligent speed adaptation, and vehicle tracking systems can all help mitigate the threats to drivers [2].

Great im-provements have been introduced to many aspects of modern cars, from better engines and chassis construc-tion to higher vehicle stability, better wheels and tires, and better overall crash protection. Unfortunately, the total number of fatal crashes is still increasing despite the safety improvements in road and vehicle design. Mo-tor vehicle-related fatalities increased from 33,186 in 1950 to 42,387 in 2000 [3].

One could argue that the increase in fatalities could be attributed to the increase in the num-ber of cars on the road, resulting in a normal correlation. Nonetheless, the problem seems to be more complex than just a correlation. In this paper, we analyze the threats to driver safety and the growth of speech recogni-tion with respect to automotive domain. In addition, we propose a solution to minimize driver distraction by introducing multimodal interaction and comparing the command hit rate when using single mode speech inter-action and dual mode speech with text tip.

Get quality help now

Sweet V

Verified

Proficient in: Distracted Driving

4.9 (984)

“ Ok, let me say I’m extremely satisfy with the result while it was a last minute thing. I really enjoy the effort put in. ”

+84 relevant experts are online

Hire writer

Speech recognition (SR) has recently been evolved as an alternative mode of driver-car interaction and is as-sumed to be a panacea to driver distraction. Past re-search neither fully supports nor contradicts the above assumption. Most attempts of introducing voice-controlled cockpit focus on reaching perfection in SR using natural language processing (NLP). In our re-search, we review some inherent flaws of in-car speech recognition and present a rich multi-modal interaction design to mimic real human-human interaction. Our multiple modes are designed to work in the specific con-text of automotive domain. The speech interactive modes are intended to work collectively to enhance driver interaction within the cockpit while avoiding the inherent problems of traditional speech recognition.

DRIVER DISTRACTIONS

Driver distraction, in simple words is anything that al-lows the driver to take hands off the steering wheel or eyes off the road or the mind unfocused [28]. This ap-proach is generic and needs to be looked at in more de-tails. One of the main challenges in driving is the coordi-nation between body and mind on one side, and the car controls as well as the road dynamics on the other side. This makes the true understanding of driver distraction. If we consider the difference between novice and expert drivers, we can identify the level of coordination be-tween them as low (for novice driver) versus high (for expert driver). Figure 1 shows Four main phases that a typical driver would go through:

At the very beginning of learning how to drive, the driv-er will go though a very low level of coordination. The driver will have hard time trying to coordinate their physical activities between managing the gas, breaks, steering, car movements as well as other car controls. No attention would be paid to additional control or even the raod. Drivers would often start this phase in an empty space not shared with other drivers until they gain basic coordination to move to the next phase.

In this phase, the driver will gradually gain better basic coordination and start paying some attention to the car movements. In this phase, drivers would be able to focus on road signs as well as other drivers. Eventually, driv-ers will be able to reach a sufficient level of coordination to pass a driver’s test and obtain their driving licence.

At this stage, drivers would be allowed to drive on pub-lic roads alone or initially with a mentor. This stage will see progressive coordination and more attention to the road and other vehicles as well as road signs. This phase can extend to a longer time interval than the first two phases and will be mainly for the driver to establish a strong driving experience.

This phase moves the driver into an expert. While the borderlines between the first 3 phases can be general defined in concrete terms, the borderlines between phas-es 3 and 4 is blury at best. Over several months or years, the driver becomes an accomplished driver and reaches a high level of experience. This experience relied heavily on the perfection of coordination between the driver physical activities and the road dynamics. The physical coordination eventually becomes a second nature to the driver due to its complete predictive and systematic na-ture; very similar to walking. While toddlers first attempt to walk seem daunting tasks, we eventually walk without thinking as our brains automate the process. On the con-trary, the second type of coordination, the attention to road dynamics remains a major challenge due to its non-systematic and unpredictable nature. Roads and other drivers are continuously changing and often have an ex-ception or a surprise once in a while. Therefore, unlike physical coordination, the mental coordination stemming from the focus on the road and driving dynamics remain unautomated. This type of mental coordination is proba-bly the main reason for driver distraction, and a major contributor to automobile accidents.

Unfocused driving can be due to fatigue, ageing, alcohol or distraction. According to a study released by the Na-tional Highway Traffic Safety Administration (NHTSA) and the Virginia Tech Transportation Institute (VTTI), 80% of crashes and 65% of near-crashes involve some form of driver distraction [28]. Since 2009, the U.S. De-partment of Transportation (USDOT) has launched a va-riety of creative campaigns to raise awareness about the dangers of distracted driving such as “One Text or Call Could Wreck It All” [2]. In fact, we can identify two main groups of contradicting factors. Our initial analysis shows these two distinct groups of risk factor will possi-bly have opposing effect on the number of accidents. The net effect remains to be known or even clearly identifia-ble.