Seamless Autonomous Mobility lets a person pilot an autonomous vehicle through a hazard, and all of the models learn from it.
Nissan was an early innovator in autonomous technology, especially after Carlos Ghosn’s original plan to have the system ready by 2020. At this year’s Consumer Electronics Show, the CEO is going into detail about some of the solutions that could eventually make people unnecessary behind the wheel.
Machines aren’t great at problem solving, and Nissan is tackling that issue with its Seamless Autonomous Mobility technology on demonstration at CES. Rather than letting a computer figure out the problem, the company is handing things over to a person who would remotely drive the vehicle.
For example, at an accident with police on the scene a human driver would easily be able to identify first responders and react to any of their signals about driving around the collision. According to Nissan, a driverless vehicle “cannot and should not, reliably judge what to do by itself” in this same situation. Instead, Seamless Autonomous Mobility would contact a human at a remote site who would be able to see the data from the model’s lidar, cameras, and other sensors. This person would then take control long enough to get the vehicle out of the situation.
Seamless Autonomous Mobility would also send the temporary human drivers commands to any other driverless cars in the area. Then, the vehicles would know how to avoid the hazard rather than having the person at the home base need to take control of each one of them. Over time, the self-driving models would learn from these experiences and would need external help less often.
Nissan foresees Seamless Autonomous Mobility as a backbone for driverless delivery vehicles in the future, and the company is partnering with the Japanese company DeNA to begin developing self-driving, electric commercial models for carrying packages. They’ll start the first test of it later in 2017 in Japan and begin evaluating the models in Tokyo in 2020.
The automaker isn’t only focusing on autonomous driving, though. Ghosn is also outlining using Microsoft Cortana as an in-vehicle personal assistant. The system would adapt to different driver preferences, and integration with the tech giant’s Azure cloud service would allow for more advanced navigation, over-the-air updates, and remote vehicle monitoring.
Gallery: Carlos Ghosn drives Infiniti Q50 autonomous prototype
On January 5, 2017 during his CES keynote Nissan Chairman of the Board and Chief
Executive Officer Carlos Ghosn announced several advancements as part of the Nissan
Intelligent Mobility blueprint for transforming how cars are driven, powered, and integrated
into wider society. Highlights included:
The launch of Nissan’ breakthrough Seamless Autonomous Mobility (SAM)
system. Developed from NASA technology, SAM partners in-vehicle artificial
intelligence (AI) with human support to help autonomous vehicles make
decisions in unpredictable situations and build the knowledge of in-vehicle AI.
This technology will enable millions of driverless cars to co-exist with human
drivers in an accelerated timeline. It is part of Nissan Intelligent Integration.
Leading a Renault-Nissan Alliance engagement, Nissan and Japanese internet
company DeNA will begin tests aimed at developing driverless vehicles for
commercial services. The first phase of testing will begin this year in designated
zones in Japan, with a focus on technology development. By 2020, Nissan and
DeNA plan to expand the scope of their tests to include the commercial usage of
driverless technology for mobility services in the Tokyo metropolitan area.
A new Nissan LEAF will be coming in the near future. The model will be
equipped with ProPILOT technology, enabling autonomous drive functionality for
single-lane highway driving. This new LEAF will build on the company’s
leadership in electric vehicles, which includes more than 250,000 Nissan LEAFs
sold worldwide since 2010. The new LEAF represents the next chapter of Nissan
The Renault-Nissan Alliance is continuing its partnership with Microsoft to build
the next generation of connected car technologies. The keynote included a
demonstration of how Microsoft’s personal assistant technology Cortana can
make driving more efficient and enjoyable. Cortana is one of the technologies
that the Alliance and Microsoft are exploring together.
A new partnership with 100 Resilient Cities - Pioneered by The Rockefeller
Foundation (100RC). 100RC is a global non-profit working to help cities build
resilience to physical, social, and economic challenges. Together, Nissan and 100RC will help cities lay the groundwork for autonomous drive, electric vehicles,
and new mobility services. Nissan is 100RC’s first automotive platform partner.
Nissan Intelligent Mobility
The world is facing serious challenges such as climate change, traffic congestion, road
fatalities and increasing air pollution. Nissan is committed to addressing these
challenges by making transportation safer, smarter, and more enjoyable, with the
ultimate goal of achieving zero-emissions and zero-fatalities on the road. Nissan
Intelligent Mobility is the roadmap.
Guided by the vision of mobility for all, Nissan is implementing these innovations by
bringing them from luxury segments to compact high volume models and ensuring
everyone has access to the benefits. Nissan is making cars exciting partners for all of
Nissan Intelligent Mobility encompasses three core areas of innovation:
Nissan Intelligent Driving helps to give customers more confidence through enhanced
safety, control, and comfort for everyone on board. The building blocks for autonomous
driving are already built into Nissan cars with extensive set of advanced safety features
including Intelligent Around-View Monitor and Intelligent Lane Intervention. Autonomous
drive technologies can already be found in certain Nissan vehicles today, including the
Nissan Serena, the first model produced for the mass market to feature ProPILOT
technology. Nissan has plans to extend this technology to more models in Europe,
Japan, China and the United States, with 10 models to be launched by 2020 by the
Nissan Intelligent Power makes driving more exciting for customers by continuing to
reduce emissions and increase fuel economy. Nissan is committed to a holistic
approach to achieving zero-emission mobility by making internal combustion engines
more efficient and putting more advanced technologies into our electric vehicles. We
continue to advance a variety of powertrain technologies under Nissan Intelligent
Power, which are most suitable to the different market segments and different regions
across the world. We have a diverse range of EV-based technologies in our portfolio in
addition to 100% electric vehicles, these technologies include e-Power (series-hybrid)
and fuel cell electric vehicles. Each new technology supplements the portfolio, but does
not supplant other technologies.
Nissan Intelligent Integration keeps customers more connected by conveniently
linking Nissan cars to the wider society. Nissan is helping to shape a sustainable
ecosystem enabling cars to interact with people, other cars and road infrastructure.
This approach will eventually lead to remote vehicle operation, reduced traffic jams,
more efficient car-sharing, and improved energy management.
Nissan Intelligent Mobility is not about removing humans from the driving experience.
Instead, it’s about building a better future for Nissan customers where cars are their
partners, and where drivers are more confident and more connected.
Seamless Autonomous Mobility: The Ultimate Nissan Intelligent Integration
Advances in artificial intelligence are making vehicles smarter, more responsive, and
better at making decisions in a variety of driving environments. But we are still not at a
point where autonomous vehicles can know exactly how to handle unpredictable
situations. This is one of the roadblocks to realizing a fully autonomous future for
driving. The solution is Nissan’s Seamless Autonomous Mobility system or SAM.
During CES, Nissan conducted a live demonstration of the system in operation using a
link-up to NASA Ames Research center. The demonstration of the drive showed just
how SAM will work in reality.
SAM will ensure a seamless mobility system in which millions of autonomous cars can
operate safely and smoothly. SAM can help cars to navigate unforeseen situations that
occur on city streets, such as accidents, road construction, or other obstacles.
Here’s how it works: imagine an autonomous vehicle is moving through city streets and
comes across an accident, with police using hand signals to direct traffic, perhaps
across double yellow lines and against traffic lights. The vehicle cannot and should not,
reliably judge what to do by itself.
Vehicle sensors (LIDAR, cameras, radars) can tell the car where obstacles are, the
traffic light state, and even recognize some hand gestures, but human judgment is
required to understand what other drivers and pedestrians are doing and decide on the
appropriate course of action.
With SAM, the autonomous vehicle becomes smart enough to know when it should not attempt to negotiate the problem by itself, as in this instance. Instead, it brings itself to a
safe stop and requests help from the command center. The request is routed to the first
available mobility manager – a person who uses vehicle images and sensor data
(streamed over the wireless network) to assess the situation, decide on the correct
action, and create a safe path around the obstruction. The mobility manager does this
by “painting” a virtual lane for the vehicle to drive itself through. When the policemen
wave the vehicle past, the manager releases the car to continue on by itself along the
designated route. Once clear of the area, the vehicle resumes fully autonomous
operations, and the mobility manager is free to assist other vehicles calling for
As this is all happening, other autonomous vehicles in the area are also communicating
with SAM. The system learns and shares the new information created by the Mobility
Manager. Once the solution is found, it’s sent to the other vehicles.
As the system learns from experience, and autonomous technology improves, vehicles
will require less assistance and each mobility manager will be able to guide a large
number of vehicles simultaneously. There are several factors that will determine how
many managers are necessary: for example, how busy the zone is, and what service
the vehicle is providing, whether it’s for robo-taxis, robo-shuttle, or a robo-delivery
NASA’s Visual Environment for Remote Virtual Exploration (VERVE) software, used to
visualize and supervise interplanetary robots, was the starting point for Nissan’s SAM
platform. NASA’s robots use autonomous technology to avoid obstacles and calculate
safe driving paths through unpredictable and uncertain environments. Where the
environment makes autonomous decision-making difficult, NASA supervisors draw the
desired route and send to the robot for execution.
Back on Earth, SAM is not for just Nissan vehicles, but for all vehicles.
"Our goal is to change the transportation infrastructure," said Maarten Sierhuis, former
NASA scientist and director of the Nissan Research Center in Silicon Valley. "We want
to reduce fatalities and ease congestion. We need a huge number of vehicles out there.
What we are doing at Nissan is finding a way so that we can have this future
transportation system not in 20 years or more, but now."
“This is not only a demonstration of the transfer of space technology to industry, but also
the application of their research back to our space technology, with additional uses for
our unmanned aircraft systems research. This is a perfect example of technology
literally driving exploration and enabling future space missions,” said Eugene Tu, Center
Director, NASA Ames Research Center.
The backbone of SAM is human/machine teaming. The goal is not to remove the human
from the system, but rather to use the human intelligence more strategically to support a
larger system of autonomous mobility – and to help improve the artificial intelligence of
the vehicles in real-time.
SAM makes it possible for society to reap the benefits of the mass introduction of
autonomous vehicles. In any single day, autonomous vehicles will encounter thousands
of situations that should not be resolved autonomously. Without SAM, these vehicles
will be stranded, causing traffic congestion, creating a public nuisance and failing to
reach their destinations. SAM permits autonomous vehicles to seamlessly integrate into
existing transportation infrastructure and society. But it’s more than a luxury – SAM is a
necessary component of any system with autonomous vehicles. Without a technology
like SAM, the full integration of autonomous vehicles into society will be difficult.
SAM will also benefit companies that wish to deploy fleets of commercial autonomous
vehicles, including delivery companies, taxi services and transportation systems.
Announcing the New Nissan LEAF: The Next Chapter in Nissan Intelligent Power
On stage at CES, Ghosn announced plans to launch a new Nissan LEAF, with ProPILOT
technology, enabling autonomous drive functionality for single-lane highway driving. The
new LEAF is coming in the near future and represents the next chapter of Nissan
The new LEAF will build on Nissan’s industry-leading position in electric vehicles (EVs).
In 2010, Nissan was the first carmaker to introduce an all-electric vehicle to the mass
market. Today, the Nissan LEAF is the world’s best-selling EV with more than 250,000
units sold and more than 3 billion kilometers travelled, a distance to get to Saturn and
back. In the process, the LEAF has prevented the emission of 497,227 tons of CO2.
That is the equivalent of the emissions made by more than 52,000 homes a year in the
To advance zero-emission mobility today, Nissan is proactively focusing on vehicle
electrification by diversifying electric powertrains and fuel systems to adopt multiple fuel
and energy sources to meet different market and customer needs. In addition to
Nissan’s core EV technology, Nissan has a diverse range of EV-based technologies in
its portfolio. These technologies include e-Power (series-hybrid) and e-bio fuel cell
electric vehicles. The new Nissan Note, also available with e-POWER powertrain,
launched in Japan in the fall of 2016, became the best-selling model in the market in
Nissan’s commitment to zero-emission mobility extends beyond making a great vehicle.
Nissan is helping to shape a sustainable ecosystem with a holistic approach - from
building a robust charging infrastructure to recycling batteries to introducing the ‘vehicleto-grid’
Nissan has worked with local governments, corporations and other entities to deploy
charging infrastructure and encourage the adoption of EVs. To-date, there are more
than 13,600 CHAdeMO quick chargers installed globally. In the U.S., Nissan is involved
in DRIVETHEARC, a corridor of electric vehicle fast-charging stations spanning from
Monterey to Lake Tahoe in California, promoted by Japan's New Energy and Industrial
Technology Development Organization (NEDO). DRIVETHEARC aims to increase the
ease of long-distance EV travel while studying EV use and driving patterns.
EV batteries can do more than just provide power for driving – they can also be used as
energy storage devices. To this end, Nissan is promoting EVs as clean mobile energy
units. Integration of EVs into society will help energy distribution across the grid, and
vehicle-to-home (V2H), vehicle-to-building (V2B), and vehicle-to-grid (V2G) solutions
have already been adopted in many markets such as Europe, the U.S. and Japan.
In 2012, Nissan introduced its “LEAF to Home” system in Japan, allowing drivers to
supply a house with the energy stored in a Nissan LEAF battery. Users can charge the
Nissan LEAFs at night when electricity is cheaper, and then use that electricity as the
daytime power source for a household. This way, the system helps to alleviate power
consumption during peak periods when demand is highest and most expensive. It can
also be used as a backup power supply for blackouts and emergencies.
Nissan has also been testing a V2B system at the Nissan Advanced Technology Center
in Japan since 2013. In this project, six LEAFs contribute to a decrease in electricity
costs. Nissan’s new Europe HQ in Paris will be partly powered by V2B and V2G
technology when it opens in spring 2017.
Today about 4,000 households in Japan are utilizing EVs to manage home energy use,
and thousands of EVs are powering buildings in the U.S., Japan and Europe.
For example the Hawaiian island of Maui, Nissan LEAF owners volunteered to
participate in a unique project which explored the possibilities of combining smart grid,
renewable energy and electric vehicle technologies into a single comprehensive energymanagement
solution. Residents use renewable energy from wind and solar sources to
power their vehicles. In return, they use energy stored in their EV to manage the energy
of the island. About 600 LEAF owners participated in the project and Nissan, along with
other partners, are using the information to inform technology development and policy
Elsewhere in the US, Nissan is involved with a variety of V2G and V2B activities. For
example, Nissan has been a long-term partner with the Department of Defense on
multiple grid-based projects at Los Angeles Air Force Base (California), Fort Hood
(Texas), and Joint Base Andrews (Maryland). Combined, approximately 30 LEAFs have
been deployed at these bases to demonstrate the technical and market viability of EV
participation on the grid. Similar programs are underway between Nissan and other
organizations around the US, including universities and utilities.
Nissan is also involved with commercializing V2H technology in the US based on
market success in Japan. In this context, V2H would provide a homeowner with
emergency power during outages and, potentially, a means of storing solar energy for
use later in the day or at night. As part of its commercialization effort, Nissan
demonstrated the V2H technology to a variety of US audiences in 2016, including to the
Finally, Nissan is also helping to extend the “second life” of the EVs’ lithium-ion
batteries. In Europe, through the xStorage project, in partnership with Eaton, consumers
can save money by drawing energy from the sun and the grid, and then sell it back to
energy companies. Meanwhile, xStorage for business allows organizations with high
energy consumption to manage their energy usage and to power their business in a
more sustainable, smarter way.
For example, in November 2016, Nissan and Eaton announced a ground-breaking 10-
year deal with Amsterdam Arena – the world-famous entertainment venue and home of
Ajax Football Club - to provide back-up power from secondhand Nissan LEAF batteries.
The xStorage-building system will help to ensure the lights never go out at the
renowned 55,000-seat stadium, which has played host to numerous high profile
concerts and sporting events over the years.
What’s Ahead for Nissan Intelligent Driving: Driverless Testing for Commercial
Leading a Renault-Nissan Alliance engagement, Nissan and Japanese internet
company DeNA will begin tests aimed at developing driverless vehicles for commercial
services. The first phase of testing will begin this year in designated zones in Japan,
with a focus on technology development. By 2020, Nissan and DeNA plan to expand the
scope of their tests to include the commercial usage of driverless technology for mobility
services in the Tokyo metropolitan area.
The Alliance will leverage its car building expertise and advanced autonomous drive
knowledge to build and provide the prototype vehicles, which will also be electric. DeNA
will, in turn, provide its expertise in creating online and mobile user experiences to build
and lead the information technology systems to provide a mobility service platform.
This is the first time the Alliance has announced a development plan of autonomous
vehicles, including driverless technology. Now, the Alliance has tests targeted at all
levels of autonomous drive in the United States, Europe, China and Japan.
Nissan’s strategy to make driverless vehicles a reality is based on four stages. Stage
one is autonomous drive for single-lanes on the highway. This level of autonomous drive
technology was introduced last August through the ProPILOT system on the Nissan
Serena family minivan in Japan. When activated, it helps to keep the car centered by
reading lane markers, measuring the distance between your car and the vehicle in front
of you, and providing steering assistance. Sixty percent of customers who have
purchased this model, which is one of the leaders in the segment, have already chosen
this option in Japan. Nissan will be bringing this technology to other models, including
the Qashqai in Europe in FY2017.
Stage two is autonomous drive on multilane highways. This functionality will allow the
vehicle to merge and change lanes autonomously and it is expected to be available in
2018. Stage three is autonomous city driving, which Nissan expects to be available in
2020. The fourth and final stage is fully autonomous and driverless vehicles a reality.
Connected Cars: Innovations in Intelligent Driving & Integration
Mr. Ghosn announced today at CES that the Renault-Nissan Alliance is continuing its
partnership on the development and deployment of advanced connected technologies,
such as Microsoft Cortana, an in-vehicle virtual personal assistant. With features such
as Cortana speech analytics, drivers can benefit from advanced in-vehicle voice
recognition and intuitive human machine interface (HMI).
Cortana will allow the vehicle to adapt to personalized driver settings, even
understanding different driver preferences in a shared vehicle, almost making it feel like
The Renault-Nissan Alliance will develop and launch new connected services and
applications that make it easier for people to stay connected to work, entertainment and
social networks. It will also offer vehicle-centric services that can simplify and enhance
engagement with the car through usage-based information, remote access, remote
diagnostics and preventive maintenance.
The Renault-Nissan Alliance and Microsoft signed a global, multiyear contract focused
on vehicle connectivity and connected services in September 2016.
The companies will co-develop a mutual vision for connected cars focused on
improving the customer experience by making driving more intuitive, intelligent
The two companies will work together to develop next-generation connected
and mobility services for cars using Microsoft Connected Vehicle Platform,
which is built on Azure, Office 365, Cortana and other intelligent cloud services
provided by Microsoft.
The partnership will enable Renault-Nissan to develop a single global solution
providing connected and mobility services for customers across all Alliance
Next-generation connected services for cars will be delivered using Microsoft
Connected Vehicle Platform, which is built on Microsoft Azure, one of the company's
intelligent cloud offerings. This will improve customer experience via advanced
navigation, predictive maintenance and vehicle centric services, remote monitoring of
car features, external mobile experiences and over-the-air updates.
The Renault-Nissan Alliance selected Azure in part because of its enterprise-grade
security, Microsoft's rigorous commitment to compliance, and its unlimited scale. In
addition, Azure supports multiple operating systems, programming languages and tools,
providing flexibility and choice to build a common platform to deploy services to both
The New Sound Management Technology by Bose
Today’s drivers face various distractions, leading to thousands of injuries and fatalities
annually. To contribute to a safe and enjoyable driving experience, Nissan introduced at
CES the new sound management technology developed by Bose that brings order to the
ever-expanding non-entertainment audio landscape inside a vehicle cabin. This system
increases situational awareness for drivers and helps enhance overall safety on the road.
The Bose new sound management technology is intended to help drivers better process
and react to the increasing amount of audible information produced by today’s and next
generation’s cars, such as safety prompts, navigation signals, vehicle system alerts,
Bluetooth phone calls, and text-to-speech messages.
Bose’s new technology utilizes UltraNearfield headrest speakers and Bose proprietary
algorithms to place non-entertainment signals in virtual spaces around the driver, where
it intuitively makes the most sense.
A left-turn prompt will be heard by driver’s left ear. A Bluetooth phone call can be delivered
in a way that’s primarily heard by the person receiving the call to be less distracting to the
other passengers. Drivers can customize some of the sounds and directions.
Nissan Partners with 100 Resilient Cities to Build Urban Resilience
To support the policy environment and planning needed to integrate these technologies
into the world’s cities, Ghosn also announced a new partnership with 100 Resilient
Cities, Pioneered by The Rockefeller Foundation (100RC). 100RC is a global non-profit
working to help cities build resilience to physical, social, and economic challenges.
Together, Nissan and 100RC will help cities lay the groundwork for autonomous drive,
electric vehicles, and new mobility services.
As Nissan takes steps to build the car of the future – and the car company for the future
– Ghosn acknowledged that the policy and regulatory environment will be critical to
enable the advancement of these technologies. Increasingly these debates are being
played out on the stage of the world’s growing cities.
“The biggest transformations will not take place inside our vehicles, or even inside our
companies,” Ghosn said. “Rather, they will take place on the stage of the world’s cities.
From population growth, to the increase in elderly populations, to the stresses on
transportation infrastructure, cities are facing challenges that could be solved, in part, by
mobility solutions. To align technology, policy, and planning, automakers and cities must
work as partners.”
Nissan becomes the first automotive company to join 100RC’s Platform Partners, a
group of more than 90 leading companies, non-profit organizations, and other experts
providing free tools and services to the 100 member cities of the network -- to support
their resilience-building efforts.
This partnership has the potential to transform city streetscapes, making urban
environments more livable and improving safe mobility – hallmarks of a resilient city.
“Cities are ever-more on the forefront of tackling the world’s biggest problems – from
shocks due to a changing climate to stresses due to rapid urbanization and globalization,” said 100RC President Michael Berkowitz. “The partnership between
100RC and Nissan will begin priming cities for new automotive technology, while
creating better mobility for citizens, and building long-term resilience to the shocks and
stresses cities may face.”
Nissan and 100RC will convene city leaders and Chief Resilience Officers from cities
throughout the network for workshops to discuss the pressing issues cities face in
planning for future mobility technologies and services. Based on these discussions,
Nissan will develop a report on initial findings and opportunities, which will be available
to all cities in the network.
Additionally, Nissan will work with selected cities to test and pilot emerging mobility
technologies, such as autonomous drive, driverless cars, electric vehicles and charging
and vehicle-to-grid infrastructure, to enable cities to better plan for their adoption on a
In addition to the immediate challenges of infrastructure and sustainability, there will be
additional opportunities to ensure new technologies are to the benefit of all
communities, especially those not currently connected by public transportation.
More than 50 percent of cities that belong to 100RC have identified transportation and
sustainability as key issues in developing their resilience strategies. Nissan has a record
of working with cities and regions to test and integrate new technologies, and today’s
announcement adds to this legacy. As a Japanese automaker whose manufacturing
plants were affected by the 2011 Japan earthquake, Nissan also understands the
importance of resilience-building, and the role of business to help ensure communities
are able withstand a range of crises.
“We invite others to join us, as well, from tech partners to e-commerce companies, ridehailing
and car-sharing platforms, and social entrepreneurs who can help us to test and
develop new vehicles and services, and make sure everyone has access to the latest
technologies and services that bring value to their lives” said Ghosn.