Audi is showing off its clever self-driving tech these days at CES in Las Vegas with a new Q7-based deep learning concept featuring highly advanced artificial intelligence developed in collaboration with Nvidia. To better demonstrate the vehicle’s AI capabilities, the vehicle drove itself without any human input on a specially designed open area.
It did so by relying on a front-mounted camera featuring a two megapixel resolution that captured the road up ahead and then sent the gathered information to Nvidia’s processing unit, which was effectively in charge of controlling the Q7’s steering. To get where it is today, the Q7 deep learning concept was initially fitted with extra training cameras in order to get acquainted with the route and its surroundings while the vehicle was being controlled by a human driver.
With subsequent demonstrations, the Q7 was able to learn more and more about how to drive itself and now it can figure out on its own what to do when there’s a temporary traffic signal in front, for example. Audi believes artificial intelligence plays a very important role in the development of piloted driving technologies and that’s why it has teamed up with Nvidia to boost progress.
The fruits of the collaboration between the two in AI will be seen in the all-new A8 flagship slated to debut later this year. Set to feature Audi’s new MIB2+ (Modular Infotainment Platform) powered by Nvidia’s Tegra K1 processor, the fullsize sedan will benefit from the second-gen Audi Virtual Cockpit set to merge onboard and online information. MIB2+ is going to work together with the central driver assistance controller (zFAS), which will be making its series debut in the new A8. Further down the line, Nvidia’s X1 processor is going to be implemented.
For its new range topper, the automaker has also joined forces with Mobileye to implement image recognition using deep learning methods, also a first. The fourth-gen A8 (D5) will be the first production Audi to feature a traffic jam pilot function, which will allow the car to “take over full control at times.” More advanced levels of autonomy will be implemented in the next decade, with the ultimate goal being to engineer an entirely self-driving Audi.
Meanwhile, the next A8 is likely heading to the Frankfurt Motor Show in September.
Gallery: Audi Q7 deep learning concept
AUDI AND NVIDIA JOIN FORCES TO BRING Q7 DEEP LEARNING CONCEPT TO CES 2017
Piloted Audi SUV study realised through collaboration with microchip partner NVIDIA highlights the development of Artificial Intelligence
- Demo drives at the Consumer Electronics Show in Las Vegas: Audi Q7 deep learning concept shows off its machine learning proficiency
- Close cooperation with market leaders from the electronics sector: Audi working on piloted driving with NVIDIA and Mobileye
Ingolstadt, January 5, 2017 – Artificial intelligence (AI) is a key technology for piloted driving – that’s why Audi and strong partners from the electronics industry are jointly developing game-changing know-how in the field of machine learning. At the Consumer Electronics Show (CES) in Las Vegas Audi is presenting the Audi Q7 deep learning concept, a piloted driving car made possible thanks to collaboration with NVIDIA.
In conjunction with the CES keynote address by NVIDIA, Audi is demonstrating the intelligence of the Q7 deep learning concept on a specially designed, variable open area for piloted driving. The car orients itself by means of a front camera with 2 megapixel resolution, and the camera communicates with an NVIDIA Drive PX 2 processing unit, which in turn controls the steering with high precision. The high-performance controller is specially engineered for piloted driving applications.
Serving as the core of the software are deep neural networks that experts from Audi and NVIDIA have trained specifically for autonomous driving and recognition of dynamic traffic control signals. Beginning with a human driver at the wheel, the Audi Q7 deep learning concept gained a limited familiarity with the route and the surroundings, by means of observation and with the help of additional training cameras. That established a correlation between the driver’s reactions and the occurrences detected by the cameras.
During the subsequent demonstration drives the car is then able to understand instructions, like from a temporary traffic signal, interpret them right away and act as the situation requires. When a corresponding signal appears, the concept car immediately changes the driving strategy and selects either the short route or the long one. The design of the system is so robust that it can even cope with disturbance variables such as changing weather and light conditions. It masters its tasks day and night, and even in direct sunlight or harsh artificial light.
The learning methods used for the Audi Q7 deep learning concept are essentially very much like those of deep reinforcement learning. This method was the underlying principle behind the Audi presence at the Conference and Workshop on Neural Information Processing Systems (NIPS), an AI event held in Barcelona in December. There, the neural networks – which are similar to the human brain – were also trained for a particular application. While the 1:8 scale model car at NIPS learned how to park through trial and error, during the training runs the network of the Audi Q7 deep learning concept receives concrete data it finds relevant – in other words, it learns from the driver.
Pushing the boundaries of AI
Artificial intelligence is a game-changing key technology for piloted driving – of this Audi is convinced, which is why it is working closely with the leaders in the electronics industry. Together with its partners, Audi is evaluating various approaches and methods for machine learning. The aim is to always find the optimal method for the specific application being studied. Collaborative efforts by companies in the IT and automotive industries are also of tremendous value for future implementation in concepts and production cars.
With its impressive systems expertise, NVIDIA is considered the worldwide semiconductor industry’s biggest, most capable player. Audi has been working with the manufacturer since 2005. The Audi A4 was using an NVIDIA chip as early as 2007, and two years later NVIDIA technology allowed the Audi A8 to achieve a new dimension in visual displays. The Modular Infotainment Platform (MIB), which was introduced in 2013, featured the Tegra 2 processor from NVIDIA. And the MIB2 followed in the Audi Q7 in 2015, running with an NVIDIA T 30 processor.
The platform’s next level of development is the MIB2+ – which is premiering this year in the new generation of the Audi A8. Its key element is the Tegra K1 processor, which makes new functions possible and has the impressive computing power needed to support several high-resolution displays – including the second-generation Audi virtual cockpit. Onboard and online information will merge, making the car part of the cloud to a greater degree than ever. Together with the MIB2+, the central driver assistance controller (zFAS) in the new Audi A8 is also making its series debut. The K1 processor is also on board and in future the X1 processor from NVIDIA. Audi and NVIDIA are planning to intensify their long-standing partnership by combining NVIDIA’s development environment expertise for AI applications with Audi’s wealth of experience in the area of vehicle automation.
Another Audi key partner is Mobileye, whose image processing chip is also integrated in the zFAS. The high-tech Israeli company is the world leader in the field of image recognition for automotive applications. Mobileye is already supplying a camera for use in a range of Audi models – the Audi Q7, the A4/A5 series and the new Q5 – and the product’s image processing software can recognise a large number of objects. These include lane markings, vehicles, traffic signs and pedestrians. Today, defining the characteristics needed to clearly classify objects is still done manually.
2017 Audi A8 – the next level of piloted driving
In the new Audi A8, Audi and Mobileye are demonstrating the next level of development – with image recognition that uses deep learning methods for the first time. This significantly reduces the need for manual training methods during the development phase. Deep neural networks enable the system to be self-learning when determining which characteristics are appropriate and relevant for identifying the various objects. With this methodology the car can even recognize empty driving spaces, an important prerequisite for safe, piloted driving.
The traffic jam pilot function will be offered in a series production model for the first time in the new A8. This is the first piloted driving function in series production that will enable the driver to let the vehicle take over full control at times. With this step the stage is set to begin the next decade with higher levels of automation in a growing number of driving situations.