You’ve been working on automated and connected vehicles for a long time. What issues do you see ahead of us in these fields that need to be addressed?

Connected and automated vehicle technologies are not new and the promise that they can improve the safety and efficiency of the transport system has been known for the last 20 to 30 years. But there are barriers to using these technologies. Some of these barriers are political, regulatory, or legal, but some are truly technical and scientific. I’m not trying to remove any policy barriers. Our goal is to resolve some of these scientific issues so these technologies can be deployed. But to some extent our work also helps the regulatory body and the government to design better policies so that they can facilitate the deployment and development of these technologies.

We have designed the Mcity Autonomous Safety Assessment Program in the hope of providing the government a blueprint for ensuring the safety of autonomous vehicles before they are deployed. That could help both autonomous vehicle developers and the public to gain confidence. Right now, there’s a lot of potential but also a lack of trust in autonomous vehicle safety. When a Cruise robotaxi had a collision in San Francisco, that not only impacted the company but also dramatically affected the field itself. To realize the promise of automated and connected vehicles, technology companies like Commsignia and Cruise need other parts of society to support their efforts, such as government agencies from the perspective of policy makers.

What direction do you see that might work?

Recent reports show how important safety testing is for automated vehicles, which accounts for half of the development cost. There are three major approaches: testing in public is the most broadly used approach, but there are two issues with that. If the vehicle isn’t safe enough, crashes can occur. The other is that safety-critical situations are still rare. For a large proportion of the miles driven, vehicles do not encounter any challenging situations. The second approach is to try to solve these issues in simulations. There are still problems with this. Either it’s not realistic enough, or it’s inefficient and wastes a lot of computing hours.

We designed and developed a naturalistic driving environment simulator based on a machine learning model to test the safety performance of vehicles. For the simulator, we have trained the background agents so that we know at what time, which agent will perform which maneuver, and with what probability. Basically performs as a bad driver to challenge the vehicles. We did this in a scientific way to guarantee the unbiased nature of the simulation and testing. We then combine the simulator with the physical Mcity test facility to challenge real vehicles with these virtual agents and see how they respond.

Will you also test connectivity solutions with agents? How does connected vehicle technology fit into this?

Mcity
The University of Michigan created Mcity to bring together the diverse expertise and resources required to realize the potential of emerging mobility technologies, and their commercial and economic viability. Mcity is a public-private partnership and operates the world’s first purpose-built proving ground for testing the performance and safety of connected and automated vehicles and technologies under controlled and realistic conditions. 

UMTRI
The University of Michigan Transportation Research Institute (UMTRI) was founded in 1965 as a global leader in transportation research and a partner of choice for industry leaders, foundations, and government agencies. UMTRI’s multidisciplinary research includes short and long-term projects in areas involving social and behavioral analyses, accident data collection, traffic safety analysis, and standards development and testing, as well as the deployment and evaluation of new safety and mobility technologies.

From the vehicles’ perspective, connectivity serves as sort of a sixth sense. Onboard sensors have limitations in range and in line-of-sight. With connected vehicle technology, drivers can receive additional information about their surrounding driving environment, because other vehicles broadcast their location, speed and status as well. The infrastructure can also send information to the vehicles and use the information coming from connected vehicles to manage traffic flow as well as improve safety and efficiency. It can be mutually beneficial.

The other benefit is related to vulnerable road users, such as bicyclists and pedestrians. Oftentimes there’s miscommunication between vulnerable road users and vehicle drivers that could lead to crashes. V2X has the potential to improve communication between them.

How do you plan to use your results?

Our vision is to build this testing ground for different sectors throughout the ecosystem so they can come and work collaboratively to test out their systems. There’s an intersection in the Mcity test facility that is equipped with multiple sensors from a variety of vendors. We have multiple connected vehicle roadside units. We also have vehicles with onboard sensors. It’s a controlled and closed environment where vendors and OEMs can safely evaluate their systems and identify issues. Whether their roadside perception sensors produces accurate results. Whether a V2X message can be sent real-time or with a delay. Whether the messaging had any packet loss. We are able to help them identify issues.

Our other goal is to deploy these technologies in the real world. For example, the U-M Transportation Research Institute (UMTRI) has been awarded grants from Federal Highway and USDOT to deploy this technology through the Ann Arbor Connected Environment and Smart Intersections Project. Autonomous vehicle developers can use real-world intersections to leverage information from sensor data sharing messages and incorporate that with their motion planning and path planning algorithms and see how these will help.

What’s the benefit of this program in your work as a professor?

Actually, education and workforce development is our number one goal. We do research that pushes the boundaries, but more importantly, we train the next-generation workforce. We have programs like the Perot Jain TechLab at Mcity program in which we invite startup companies in the field of connected and automated vehicles to work with groups of five to six students on specific projects. A lot of these are technology development projects that are solving specific problems. It has been very successful with several students being hired by these companies following graduation.

The US Department of Transportation has developed and is continuously expanding its V2X Deployment Plan to foster the use of connected vehicle technologies. According to the USDOT, V2X is one of the most effective tools to achieve the Vision Zero goal of zero road fatalities. As part of this effort, the third edition of the V2X Summit was held in Ann Arbor on the University of Michigan campus, where industry stakeholders were asked for their feedback. (If you are looking for the October 2023 version of the plan, click here.)

“The industry needs to educate more, train more, and share their experiences”

– that was one of the key messages of Laszlo Virag at the panel discussion. “I encourage IOOs and cities who want to save lives, to reach out to cities that are bigger and have experience to learn from them.” 

A roadmap for value creation would also be useful, and not just for the future. V2X is already creating value today, there are real problems that it can solve – said Laszlo Virag. Anyone starting a deployment with short term goals should be prepared to ensure that all the values are well understood by stakeholders. So, for example, when building a priority solution, they can be sure that later on there could be a camera or lidar based detection integrated with V2X for vulnerable road user safety. 

Many other applications create additional values. V2X can be around in ten, twenty or even thirty years. From a technological point of view, it is absolutely feasible. We just need to spread the word.

The industry could also use a “cheat sheet” for deployments, a more comprehensive list of specific actions. A list of things to do and things to avoid, to ensure interoperability, security, not forgetting a good support team, maintenance and getting the deployment into the operational phase. There are such huge differences between deployments that some may not even be called deployments.

“I truly believe that the US Department of Transportation and this community have already done a great job. The structure of the deployment plan and the way it links the various stakeholders to specific actions is great. Now we just need to push the accelerator.”

“We are now in a good place as our RSUs are operating reliably in the field. We’ll probably start getting requests from our in-house safety studies teams and try to use some of the data to analyze traffic saturation and what happened with the pedestrians.” – Kyle Higgins told us. 

The thing to know about ITS in Florida is that they use the SunGuide Traffic Management Center solution to establish connection with V2X equipment and to operate a vast digital roadside infrastructure. There are various applications attached to SunGuide that help sending out V2X messages. These include general roadway alerts, wrong way driver alerts and construction area information for V2X enabled fleets operating in Florida. SunGuide also helps capture all the data coming back from connected vehicles.

“There is another platform that’s hosted by the FDOT central office called the DEP, Data Exchange Platform, and we’re forwarding every single message that we’re getting off of our RSUs. We also have a pilot project at University of Central Florida (UCF) in which they will receive decoded V2X data to do scientific studies on them.”

“I am very excited about the future. It would be great if our timing engineers, for example, could see when vehicles in a lane of a three-lane road are traveling 20% faster at a certain time on a regular basis for some reason. Of course, we have that kind of data if there’s loop detection, and radar detection, and it’s set up properly, and if you can get the data out of it at all.” – said Kyle Higgins.

When you consider that connected vehicles send raw data about themselves ten times every second using BSM messages alone, and how little information we get from loop detectors, MVDs or point-shoot radars in comparison, you can see that the advantage of V2X is huge.

Connected vehicles provide granular data and get straight to the point. I see the potential, but penetration has to be there – said Kyle Higgins. “Once that’s there, that’ll be golden.”

Utah has decades of experience in building Intelligent Transport Systems. What advice do you have for those with less developed digital assets?

Agencies have limited funds and the public expects us to use those funds to repair broken roads and fix potholes, fix guardrails, and I fully understand that. We hope to demonstrate that there’s value in V2X. From a safety standpoint we need the automakers to deploy V2X in their cars, but we’re trying to show that the mobility benefits are great.

Deploying V2X requires good traffic signal systems. You can’t hang a V2X roadside unit on a signalized intersection and broadcast signal phase and timing messages unless the signal controller is capable of providing you with this information.

“As a first step, it’s worth finding some funding to upgrade the traffic signal system. That will improve the traffic flow anyway, and the public will notice that.”

While you are doing that, spend a little money on backhaul connections, so you can communicate with these intersections to make them efficient. If you do that first, you are setting the stage and getting yourself ready to deploy V2X. That’s what I would tell my colleagues in other states and cities: pick a small area, a couple of corridors in a big city where you have challenges moving traffic and improve those corridors. Retime traffic signals to make them better and traffic will flow better. That would generate some interest and potentially get you a little more funding. You can do that without lots of money on a small scale.

How did UDOT get involved in digital development?

We have a very broad deployment of fiber optics. We started deploying fiber in about 1998, and pretty quickly had an expectation that any major construction project needed to include fiber, even if there was nothing to connect it to yet. As a result of that we have fiber communications all over the state, including very rural areas. It allows us to deploy cameras in places where we need them and use variable message signs to provide information. We also have weather stations in a variety of places, so our meteorologists can watch and predict weather. That gives us better and more reliable information, so travelers can plan better, and our maintenance crews know when they need to be ready to plow snow or deal with ice on the highway. It all makes a difference in road safety.

How did the “digital first” mindset come about so early?

We’ve been very lucky in our state department of transportation to have very consistent and very innovative leadership. While many state DOTs get a new leader every four years when a new governor arrives, our executive directors often have tenures of a decade or more. They’ve established a culture of forward thinking, finding better ways of doing things, which allows us as an organization to think outside the box. So it’s a top-down leadership culture that’s built on innovation and forward thinking. Many agencies do not have this benefit. Secondly, our state legislature recognized in the late 90s that good business growth requires good transportation. They decided that we need additional funds to make sure that our transportation system is well maintained and growing along with our population. The third benefit is that the population of our state is also very young and growing.

You have a bus lane project in Ogden where you use dedicated lanes and V2X-based Transit Signal Priority as well. Why do you use both of these?

The Utah Transit Authority operates the bus system, and they recognized that better bus service will attract ridership and keep more cars off the road. They are always looking for ways to improve their bus service. They got some experience from another Transit Signal Priority (TSP) project we did together four years ago, and decided to include this in their new Bus Rapid Transit (BRT) line in Ogden.

The project starts at the Front Runner commuter train station, moves up through the city, passing by a major hospital and to Weber State University, the 4^th largest university in Utah. Parts of the BRT line have dedicated lanes, but much of the corridor does not. We use V2X TSP to improve bus performance through these intersections. To accomplish this, the bus must communicate with the signal and coordinate the phase with the movement of the bus.

What are the most important goals of ITS America, and where does vehicle-to-everything or V2X fit in?

ITS America is the only national association that focuses on the integration of technology into our transportation system to support a safer, greener and smarter future for all. Our role is to be the leading national voice on transportation technologies to integrate them into our everyday life. As we have seen with mobile phones, real-time information to enable transportation is everywhere and the public is adaptable to accepting these innovations.  Vehicles are increasingly equipped with more and more technology and connectivity. We are guiding and leading this transition to ensure that technology is being integrated in a way that supports communities and our safety goals.

That’s where V2X comes in, very prominently. It is perhaps the number one tool we have to achieve our Vision Zero safety goals. V2X can also advance our climate goals, sustainable mobility, equity and access in our systems. We believe that technologies like V2X – while always putting safety first – help advance all of those goals. Even something as simple as the original Transit Signal Priority has a real equity component to make transportation more accessible and work better for people.

ITS America has a long history…

…going back some 30 years when we were founded. Originally, we were called IVHS, Intelligent Vehicle Highway Systems, because ITS technology was focused on highways and vehicles. Now ITS America as an organization and the solutions out there cover every mode of transportation. More and more technological tools are being developed that can help solve the problems and challenges we face in transportation. Fatalities are increasing – up to 43,000 per year, which doesn’t include all of the nonfatal crashes that result in serious injury. We know that 29 percent of greenhouse gas emissions come from the transportation sector, and ITS technologies help manage demand and optimize the system. Electrification alone is not going to be enough to offset all the energy used in transportation and resulting emissions. A lot of ITS technologies are now being used in the transit space and others, for on-demand mobility and seamless payments, to make transit more reliable, accessible, and attractive to customers. These are the things I think about when I talk about the digital transformation of transportation.

What was the biggest change in technologies that moved forward these efforts?

I think it is the real-time data coming from sensors, connected vehicles, mobile phones and other inputs. Transportation agencies don’t have to rely on a few years old historical crash reports to figure out where the most dangerous intersections are, they can actually see them using real-time data and determine fixes. Some of those fixes are going to be things like deploying V2X or other technologies, and some of those fixes are going to be a physical improvement in a particular area. I think that the consumer expectation of having reliable and real-time information in the palm of their hands, whether it’s a map application or transit information, has really pushed a change forward in the industry.

On the V2X front, with the FCC approval of the joint waiver, you can see that all the players in the industry are aligned around C-V2X and the remaining 30MHz of dedicated spectrum. We have a use or lose situation. We’re seeing a maturity in the alignment in the industry, and also a maturity of the technology, where everyone has gone through the testing and piloting phase and is ready for actual deployments, and they need to happen, today.  At the same time, a massive increase in road fatalities in the country has made it very clear: doing the same thing over and over is a recipe for continued tragedy. There are a host of reasons to work together to scale up V2X deployments to reduce fatalities and crashes for all road users.

What are your expectations for the coming years that would show this dramatic shift in the mindset?

We’ll be working with the federal government on a reference architecture for digital infrastructure, because we need an interoperable system. We don’t want digitalization to happen in silos in different places across the country, we want a national roadmap. Ultimately, I hope the federal government seriously considers the work we release as a starting point for a national roadmap, so that everyone understands how and where to invest, so that these investments do not become obsolete. These are investments for the future.

We’re going to produce a draft strategy and document this summer.  It will take time to reach a full consensus on the document, but I’d really love to see this happen in the next year. Timing is critical as the money from the infrastructure bill is available now and it’s important that we are able to capture those investments. We want to make sure that agencies feel confident making those investments in the digital layer.

Coming back to V2X, there are a number of grants where V2X is an eligible activity and of course we support and promote this, but we would love to see proposals including V2X or interoperable connectivity prioritized in grants. We would like to see this happen not only with smaller technology grants, but also with larger grants like RAISE, which allows for larger scale deployment.

What should people interested in V2X look for?

ITS America just released a national V2X deployment plan covering both the infrastructure and automotive side to inform the development of USDOT’s plan on national interoperable connectivity. Through our committees and working groups, companies and organizations can provide their input, and they can help us promote this plan to US DOT and others. We will also have opportunities over the next year to get involved and help contribute to the national dialogue.

Do you have specific plans about the FCC C-V2X waiver decision?

Our V2X committee will continue to engage on this. The joint waiver has been approved, but it’s important to know that there are other individual waivers still being held back, and we’re working very hard to get those approved by the FCC. There is no reason to hold those back. We want to make sure that all of the waivers that have been filed are approved so that these entities can move forward with deployments across multiple states and jurisdictions.

What will make these rules final?

The FCC has a regulatory process that they have to go through. It’s called the second report and order, which is the ultimate vehicle for the FCC to publish final rules for the spectrum. We are working with our members and through our regulatory channels to provide input into the FCC process so that we can continue to advocate for final rules that support our members’ positions. But it’s all part of the FCC’s rulemaking process.

Is it possible that they want to see deployments all over the United States to be convinced?

The FCC regulatory process is their own process, so I can’t comment on how quickly it will go or not. We are pushing for quick deployments to show that the industry is hungry and moving everything to get this technology to scale.

For those who are not familiar with V2X and the work of the Car2Car Communication Consortium, how would you describe its activities? What was the challenge that brought Car2Car to life?

Car2Car was the organization that put the letter C into C-ITS, the collaboration part of Intelligent Transport Systems.

Traditionally, cars didn’t have any contact with the outside world. The only connection between the car and the road operator was the tire to the tarmac. Car manufacturers started to use radars and camera systems, to individually help drivers, but they wanted to go beyond that. As the need for collaboration between vehicles emerged, the industry needed common specifications and technologies, so vehicles could understand each other.  

The work started with research. There was a frequency band, and we had to figure out how to use that. In mobile networks, the operator controls the network, but V2X is more similar to wifi. We needed a higher degree of interoperability, and in road safety applications we couldn’t allow a long setup before vehicles could start negotiating.

The first results have been brought into standardization, as the European Commission has provided funding to get standards out of the work. The research phase has continued with new developments since then, it’s not a one-shot thing. A third phase is also needed, there are always questions about how standards should be interpreted. In profiling, we define exactly how we will use the standards, exactly what we mean by hard braking and other events. In addition to our internal work, we hold regular bi-weekly meetings with road operators in C-Roads to coordinate the fine details and figure out what is important for road operators and the industry in order to align our priorities.

What was the most important landmark in V2X?

The real important landmark was the launch of the Volkswagen Golf 8. Until that point we had a lot of demonstrations and pilots. You can demonstrate a lot of things, but the moment somebody starts to put things in commercial products, that is where you move from dream to reality.

Is Car2Car mainly focusing on Europe?

At the moment Europe is the only place where we have large scale V2X deployments. Europe has a clear lead in V2X technologies. 

What are your future expectations with V2X?

If you look at our roadmap, we are moving towards providing the underlying support for fully automated driving. If I have to describe V2X to people who are not familiar with the technology, I ask them to imagine sitting in a car, while somebody paints all the windows black and then they have to drive. We need to get all the information the driver normally could observe into the car. Some of these can be received from cameras, radars and lidars. But, sooner or later, you will get into an unforeseen situation where you need to negotiate. Just what human drivers do in day to day traffic. When there’s a doubt who is to drive first, you make eye contact. That is the basic element of why you need communication.

You can get more details with better sensor technologies, but you cannot share information or negotiate. A classic example is when you are driving on a country road only one car wide, where you have these wider meeting points. The one that is closer to this meeting space has to reverse. Now, if both of you believe that the other car is closer, how do you get out of this situation?

What is the role of 5GAA in the development of V2X technologies?

5GAA was created a little bit more than 6 years ago, and we bring together the telecom and automotive sector, mainly to contribute to the advancement of end-to-end connected vehicle services. Beyond the commercial interest of our members, we want to provide social benefits with safer, smarter and more environmentally friendly solutions. We are active in all regions, because our members are global players, and it was always clear for us that 4G and 5G are the right technologies to focus on when we are talking about global telecommunications solutions for automotive.

We are looking at two different kinds of communications. The connectivity to the mobile network is a commercial service by the operators. It brings broad connectivity to different cloud services, to the Internet, and any other services that can be accessed through the Internet. The other one is the short range communication between different vehicles, between the vehicles and the infrastructure, and eventually also including pedestrians. It is low latency and independent of any mobile network operators. These two radio interfaces clearly complement each other. 

What brought 5GAA to life 6-7 years ago?

There were two different challenges. In 2016, the auto industry started real deployments of LTE based connectivity in cars, and this connectivity started to contribute to road safety with warnings such as local hazard warnings. This safety related traffic information, generated by the vehicles, was sent to the mobile network and generated warnings to the relevant vehicles that were passing by. This has really triggered the market and we clearly see that 4G and 5G are becoming the reference technology for connectivity in our cars.

In the meantime, the 3GPP, responsible for global mobile standards, started working on verticals around 2016-2017. Its focus was on how to design 4G amd 5G radio interfaces to serve the specific requirements of specific markets. One of these verticals was transport, but it was clear that the automotive industry is the real target. In order to make it work, telecommunications and automotive professionals had to start understanding each other. 

As a funny anecdote from these early days: people could talk about the problems they face on their network, and they would mention traffic, congestion, collisions, etc. After a while they would realize that some of them were thinking about radio networks, while the others about the road networks. All these terms exist in both sectors, on the mobile network and the road operator side, but, of course, they don’t mean the same.

What would you call the greatest achievement in the life of 5GAA?

At the time we started, it was not clear at all that all vehicle manufacturers would engage fully with 4G and 5G connectivity. We really established 5G as the reference communication for the whole industry, everywhere in the world. In short range connectivity, we managed to get from standardization to deployments in less than three years. By 2020, we had the first full scale deployment on the Chinese market, and now we are engaging with the US, South Korea and other places in the world. In Europe, the story is a little bit different because policy makers have been favoring ITS-G5. But 5GAA members are quite unanimous, they want to go beyond what ITS-G5 can do and make 5G-V2X the technology of choice.

In China, we also started to learn what works well on a large scale with a balanced use of mobile and short range radios. What makes big differences in connectivity, over-the-top cloud services, EDGE computing, and designing interfaces that are exchanging the right data at the right time for different road users. 

What are your future expectations for V2X?

First: making sure to completely embrace the opportunities related to the mobile networks. Second: to make sure that 5G V2X is market ready within the next few years. We are talking about 2025-2027 for mass market readiness. Then, the inclusion of other access technologies within the 5G family. One of them is the use of non-terrestrial networks, satellite communication, that would make sure that all parts of the world are covered, even where a terrestrial service does not make a commercial sense.

You may have already created awesome pictures and some clever paragraphs in generative AI services like ChatGPT and Dall-E, as we have created some fun ideas of what a bicycle airbag or other road safety device would look like. We truly appreciate the mind blowing creativity at the intersection of art and transport, whether it’s done by computers or humans. For the latter, our favorite is the Bull’s Head made by Pablo Picasso.

The capabilities of artificial intelligence can be used in many other areas of transport technology, with less focus on entertainment and more on safety. Here are some examples of areas where the industry prefers to use AI.

1. Scene recognition
2. Crash likelihood prediction
3. Traffic optimization

Scene recognition is a complex process including object detection, tracking and classification. With semantic segmentation smart sensors label each pixel according to the object to which it belongs, such as “sky”, “vehicle” or “pavement”. 

GPUs and TPUs brought massive parallel processing capabilities to road safety sensors and cameras to accelerate the training and inference of neural networks that are built on top is basic computer vision methods. Accurate and rapid detection of a large number of objects is key for the V2X ecosystem. Our Roadside Units need reliable information to formulate V2X messages about what’s happening on the road, so connected cars can display relevant alerts in front of the drivers. This helps in dealing with critical situations.

Roadside and on-board sensors provide data with different levels of confidence. The information therefore needs to be validated using multiple entities to make sure they see the same thing we see.

Crash likelihood detection with AI is a whole new level in V2X. Once we have understood the scene with the help of scene recognition, we need to use this information to predict and prevent potential collisions. One such measure could be to send a V2X alert to the driver about the need to change the speed of the vehicle.

Artificial intelligence algorithms for predictive modeling can build on a wide range of information sources. To predict the probability of a crash, they can analyze data from on-board and roadside sensors, cameras and other sources such as traffic patterns, weather conditions and driver behavior. V2X has the unique ability to provide real-time information about an emergency braking well ahead of us in our lane, or if there has been a collision on our route.

AI also plays an important role at the macroscopic level, in traffic optimization. Only V2X-enabled vehicles and the connected smart infrastructure can provide a realistic traffic environment description for real-time traffic management. AI-algorithms can adjust traffic signals based on actual traffic conditions, road closures, historical traffic data and weather information to reduce congestion and the risk of collisions. 

Besides managing traffic lights, intelligent traffic management systems are able to recommend optimal routes for delivery trucks, taxis and buses to reduce travel time, save fuel, and improve the overall efficiency of public services. V2X-based traffic management can enable cooperation between vehicles over a larger area, which a single self-driving car would not be able to do on its own.

Ultimately, these applications of AI can be just as entertaining as a computer-generated image: we can live in a healthier environment and be safer, spending less time on the road.