Along a digitally connected route through the center of Berlin used by diplomats and other visiting foreign dignitaries (Protokollstrecke in German), research institutes and industry partners are preparing for the nationwide rollout of a concept for highly automated driving. The greatest challenge, however, is less making cars smarter and more connecting their surroundings on the road with each system.
Autor: Thomas van Zütphen Fotos: Oliver Krato, Fraunhofer FOKUS
“Ding!” Anon Mall does not let what is presumably a received text message on his cell take even one second of his attention. For the 23-year-old, distraction is not an option, especially now, while he is driving. The student works at the DAI Laboratory at the Technical University of Berlin. DAI stands for “distributed artificial intelligence” - an institute that operates probably the most sophisticated test bed for autonomous driving in Germany.
But autonomous driving, as is soon to be possible here in Berlin at what is called “Level 5,” is only possible through a combination of vehicle sensors and external sensor information. This makes it necessary to develop and monitor from end to end connected IoT landscapes, including 5G network technology and mobile edge computing. “Without managing this complex and safety-related system, autonomous driving won’t be doable in practice,” explains Jörg Tischler, Vice President of Customer Solutions for Connected Mobility at T-Systems. “In other words, the de facto situation is this: A car cannot see around the next corner by itself.” Yet that also can be essential for the three cars currently part of the DAI project.
While the path from Anon Mall leads past the Charlottenburger gate (on the right) via the Großen Stern to the Brandenburg Gate, an autonomously driving car records traffic situations much more complexly than a human could.
3,350 miles, 1.4 million cars
During his brief noontime drive from the Charlottenburg quarter of Berlin to the Brandenburg Gate, Mall could definitely leave his hands in his lap but would always be ready to jump in himself in case of doubt. Should his car respond in a manner other than Mall would think reasonable. Now, for example, when turning from Ernst-Reuter-Platz onto Strasse des 17. Juni. With up to 43,000 cars per day, this traffic circle is one of the hotspots of the capital’s traffic the linchpin of the digitally connected Protokollstrecke (DIGITNET-PS) in Berlin.
“A self-driving car whose technology can master these kinds of traffic circles can drive anywhere in the world,” Dr. Sahin Albayrak is convinced. And that is exactly one of the goals of DIGITNET-PS. The professor is head of the Chair of Agent Technologies in Business Applications and Telecommunication at the Technical University of Berlin. He is also the founder and director of the DAI Laboratory and a founding member of Deutsche Telekom Laboratories (T-Labs). It is no accident that the newspaper Berliner Morgenpost recently speculated that, on Dr. Albayrak’s watch, “traffic history could soon be written on Strasse des 17. Juni between Ernst-Reuter-Platz and Brandenburg Gate”: With DIGITNET-PS, the first step is to use project vehicles dedicated to a limited radius of motion to turn autonomous driving into a highly reliable matter of course. What will also be studied is how the results can be rolled out over longer segments, streets, boroughs and, at the end of the day, all of Berlin with its 3,350 miles of roads. A city where, according to the Berlin Senate Department for Urban Development and Environment, 1,417,866 vehicles were registered as of the start of this year. More than anywhere else in Germany. And that does not even include the trucks, passenger vehicles, and motorcycles that come into the city every day.
Almost 15 percent of Berlin's 250,000 street lamps are gas-powered. The World Monuments Fund WMF has put the ensemble, which is regarded as unique worldwide, on its watch list.
This makes a rollout quite the task; most autonomous driving projects are currently being conducted by manufacturers and Tier 1 suppliers. Based on their product roadmap and portfolio, they are testing their sensor technologies, developing algorithms and thus setting the foundation for legislation and product approvals. Collaborations and industry partnerships for future data platforms are moving forward. However, many of these research and development projects result in isolated solutions. “To realize practicable use cases,” Tischler explained, “connected IoT landscapes for multiple partners, multiple vendors, and multiple technologies must emanate from these isolated domains of experience and working models. Therein lies the challenge.”
To reach nationwide autonomous driving of the future, millions of sensors in cars, edge device’s and infrastructure elements will have to generate enormous amounts of data. This has to be managed and interpreted as a network. The problem: “Most standard monitoring and operating tools are neither suitable for simultaneously processing extremely large amounts of data nor capable of drawing the right conclusions from confusing, conflicting, or missing information from cars and infrastructures,” Tischler further explained. Technically implementing connected IoT landscapes therefore requires artificial intelligence (AI), anomaly detection, and learning algorithms. In collaboration with T-Systems, appropriate concepts should include compliance and safety aspects alongside technical factors such as stability, availability, and performance. To tackle these challenges in the DIGINET-PS project, DAI Laboratory currently employs around 120 research assistants and students. Mall is one of them.
Between data protection and monument protection
He drives his car east, crosses the Landwehr Canal and passes what is currently the only gas lantern museum in the world. Berlin and its lanterns, that’s a part of history unto itself - how else could it be worth its own museum?
„A car cannot see around the next corner by itself.“ JÖRG TISCHLER, Connected Mobility, T-Systems
A part of history, nevertheless, to which the DIGITNET-PS project is adding another chapter. It is also an example of “the greatest problems in IT-based and communication-based innovation projects are sometimes not so much about data protection”, reports Dr. Albayrak. Sensors and cameras in cars and infrastructure do not record personal information and, furthermore, provide only pixelated video frames. In addition, the Berlin Data Protection Commissioner and various Senate agencies have been included in the project. “A veritable hurdle is, instead, the protection of monuments,” Dr. Albayrak revealed. This is because the several hundred lanterns Anon passes as he continues toward the Victory Column are all under a preservation order. “Right now,” said Dr. Albayrak, “we’re in talks with the boroughs over retrofitting the lanterns so we can use them to house our sensors. We would replace their innards without altering their appearance.” Yet experience shows talks of this kind can take a while.
For this reason, the test infrastructure has included a broad range of players from industry, research, government, and society since the start of the project. And the university’s plans have received major support from the government, in particular. The test bed is being funded by the Federal Ministry of Transport and Digital Infrastructure as part of its “Automated and Connected Driving on Digital Test Beds in Germany” policy. The Governing Mayor of Berlin, Michael Müller, and the senators of the state government of Berlin have also pledged their assistance to the research project. “That’s very important to us,” emphasized Dr. Albayrak. However, the infrastructure on the inner-city streets is a matter of the local boroughs. This even includes federal highways like the one Mall takes to quickly get to the Great Star. Where the B 2 and B 5 meet Spreeweg, Hofjägeralle, and Altonaer Strasse is where well over 40,000 cars daily make the traffic circle around the Victory Column into a second trouble spot along the four-kilometer DIGINET-PS Protokollstrecke. Here is where the project is examining the question of how much connectivity and how many sensors are needed when 40,000 vehicles need to communicate with their immediate surroundings or with one another at different (peak) times every day. And not just when they - as Mall did on this day - are driving under clear blue skies, but also in twilight, virtually at night or in fog, or when it is snowing or raining. And when, as in other test beds for autonomous driving in DIGINET-PS, it is not just cars on the move, but also bicycles, pedestrians, wheelchairs, skateboarders. And what actually happens when the power goes out in Charlottenburg? Do the cars have to pull over? What does a fallback solution look like in terms of real incident management?
Everyday instead of lab conditions
The variety of questions DIGINET-PS needs to answer makes one thing clear: Here, in the heart of Berlin, since March of last year, every facet of automated and autonomous driving is being tested under real traffic conditions and in extreme traffic situations like nowhere else. Given the special features of the innovation site of Charlottenburg in particular, with its numerous university facilities, the Protokollstrecke can also serve as a reference solution for complex traffic situations in other urban spaces. One of the specific aspects of the project is to develop a “smart city” reference architecture. The goal of the project, which will run until summer 2019, is initially to develop a flexible and integrated solution for all of the technologies involved in being able to map highly automated driving in complex, inner-city traffic and simulate it in an open test environment. This requires the design of an open and scalable platform on which the various partners in this growing ecosystem can jointly develop new applications for connected and automated mobility.
State of emergency. In a city where 1.4 million cars share the streets, something like “unrestricted travel” is considered unusual.
“Our Protokollstrecke is very complicated and that is what distinguishes it,” rejoices Dr. Albayrak every time he has a glance at “his” project from his window in the DAI Laboratory on the 15th floor of the former Telefunken building: A 2.5 mile, six-lane thoroughfare that virtually connects his office to the Brandenburg Gate, passing through numerous parks on either side. It is not only the view of the “project owners” that this “is one of the most demanding stretches in the world.” Yet - with an emphasis on growth - work continues. For Dr. Albayrak, one thing is certain: “Automated driving will be an everyday occurrence for us in 10 years at the latest.” The expert expects demand for fully or at least partially automated vehicles will grow substantially in the coming years. “On the one hand, society is getting older and older,” Dr. Albayrak said. On the other hand, there are many of today’s youth who would very consciously forego a driver’s license and a car of their own because they can get almost anywhere in the city with a bicycle or public transportation. Even they would be potentially interested in a self-driving car. The market is there. The next step would be to run the test route from Nürnberger Strasse along Kurfürstendamm to Adenauerplatz, then to Charlottenburg Castle and back to Ernst-Reuter-Platz to over various segments such as residential, shopping and office areas. By 2021, Dr. Albayrak and his team, along with partners such as the Fraunhofer Institute, ADAC, BVG, VW, IHV, and T-Systems hope to expand the test route to around 10 miles. However, to do this, additional information sources well beyond the basic sensor equipment on today’s new cars are needed. For one to train cars for autonomous driving. Second, tens of thousands of sensors are needed along the routes to increase the visibility of self-driving cars through digitized and intelligent roads. “For everything to be able to communicate with everything else,” summarized Dr. Albayrak, “requires the standardization or a wide array of protocols and the use of suitably powerful software stacks, among other things.”
Research with on-road tests
At this time of day, Mall makes it through the occasionally seven-cane traffic circle of the Great Star without issue. A quick stop at Café Viktoria to get a latte to go? Even today, the cameras along the route help the largely self-driving cars of DAI Laboratory find a parking space. Using algorithms, the system can determine and promptly notify the vehicle where a parking space is available. The cars themselves use both their own cameras as well as sonar and IR technologies (impulse response), and laser scanners to optically gauge distance and speed as well as remotely measure atmospheric pressure. Instead of radio waves like radar, the LIDAR system in cars uses laser beams. In an on-board unit (OBU) in Mall’s car, all data from external and internal information sources is compiled to reliably steer the vehicle along the most sensible, energy-saving, and safest route as traffic increases. As Mall can make out the Brandenburg Gate while driving by the Berlin Wall Memorial and the Soviet War Memorial, all vehicle, traffic, and environment information from his route converges 1.9 miles behind him - as it has been doing since the start of his journey - in a cloud. Essentially like a digital twin of what Mall’s car is “experiencing” outside in real traffic. And not only that: at DAI Laboratory, data center-based future technologies for improving traffic flow are being developed to conserve the environment and create value.
With the DAI Labor in Berlin, Prof. Dr. Sahin Albayrak heads one of the world's leading research institutes in the field of agent technology.
ACT (Agent Core Technologies), IRML (Information Retrieval & Machine Learning), NGS (Next Generation Services), COG (Cognitive Architectures & Usability), NEMO (Network & Mobility) and SEC (Security) - these six competence centers are where DAI Laboratory is consolidating its numerous areas of research. At a control center, Dr. Manzoor Khan, Director of the Future Communication Systems Competence Center at DAI Laboratory and the project manager, can track all DIGINET-PS processes. How is the car responding? How continuously is the car being supplied with data? How is it interacting with infrastructure? And how quickly does it receive the necessary data and correlate it with historical data to send back the most accurate forecasts possible? In this project, it is just three cars. After a nationwide rollout, however, it will be an entire fleet of cars. But what information is currently relevant for what vehicle where? How can I protect the cars from data overkill? Questions one person alone could never answer.
Bringing intelligence to the streets
This is why part of the open test environment of DIGINET-PS also includes, according to Dr. Khan, carmakers such as Volkswagen, Daimler, and BMW “being able to supply complete traffic data for, say, an entire month in a packet of some kind. This, in turn, makes it possible for the OEMs to continue researching and working on their own autonomous driving applications in these virtual, but one-to-one authentic traffic scenarios.” For Dr. Khan, DIGINET-PS has become his “baby” at least to the same extent as for the founder of the institute, Dr. Albayrak. And both are driven by the same thought: If we just get the chance to extract external information sources from an intelligent environment out of the cars and use it, we will be able to really achieve the highest level of autonomous driving.
Even if they “only” look like a tridirectional camera, the physical sensors of the DIGINET-PS protocol route can perform up to ten additional sensor functions.
DIGINET-PS will get one step closer to this with the new 5G cellular phone network standard starting next year. But one of the most important requirements for this is safety: if, for example, at a later stage of the project with automated and connected vehicles, scenarios such as automated shuttle and convoy trips downtown are implemented. Dr. Albayrak, together with public transportation company Berliner Verkehrsbetriebe, is already planning to use two busses “equipped with window-size displays, so passengers can follow everything the bus is currently seeing”. And that it sees more than the man behind the wheel, who will of course still be there at the start. Such confidence-building companions are intended to help citizens literally go along. Get on board with the system, so to speak. “A single sensor already gives us several different types of data and scores of information,” confirmed Dr. Albayrak. “To secure their communication - from the street to the car, from the street to the infrastructure, from the street to the cloud, back in real time, with one another and, depending on relevant, supplied to all stakeholders in a dedicated manner,” stated Tischler, “makes safety an absolutely essential requirement of autonomous driving.”
Safe in data and road traffic
Meanwhile, Mall entertains not the slightest doubt that he will make his scheduled meeting time and place right near the Brandenburg Gate. A short trip, yes, but in these few minutes, Mall and his car have driven by dozens of cameras, sensors, and antennas - installed on buildings, by the roadside, and on traffic lights - all of which transmit essential information to his car. This includes information on the course of the road, traffic jams, construction zones, as well as environmental values such as nitrogen oxide emissions. For Mall, however, even more importantly: “Every movement the vehicle makes is under the premise of ‘safety first’.”
“The point of most concern,” explained Tischler, “is that the integrity of the data remains intact, which is extremely important for autonomous driving, and data manipulation is not possible despite any associated attack vectors. The security of data transmission, as well as its speed, latency, bandwidth, uplink/downlink - these are all aspects that need to interact and be monitored and protected on the technical end.” Then comes data protection and data governance/privacy, i.e., the questions: Who owns the data? Who can interpret it? What kinds of new business models can be established from the combination of vehicle data, driver data, user data, and public data? And how do those responsible for data traffic and road traffic deal with legal and ethical problems in the future? For example, the question of how long this kind of vehicle and traffic data needs to be retained?
“It is crucial for trust in and acceptance of virtually every concept surrounding autonomous driving that these questions are convincingly answered for every road user,” Tischler confirmed. And after all: according to a study by the Bertelsmann Foundation, 39 percent of participants in an EMNID survey in Germany could see themselves using a self-driving car.
The DAI-Labor performs research and development with the Technische Universität Berlin, in order to provide solutions for a new generation of systems and services. The DAI-Labor tries to apply and test these solutions in real environments and ...