Understanding The Opportunities for AI-Cameras and LiDARs for Smart Road Infrastructure
As the Consumer Electronics Show (CES) in January sparked a new wave of autonomous vehicles (AVs) coming to the automotive market in the next few years, much focus as of late has been on the technology of these vehicles themselves. However, the technology embedded in road infrastructure is also beginning to see more conversation between service providers and municipalities. […] LiDAR can provide complementary and sometimes overlapping value with cameras, however there are still several safety critical edge cases where LiDAR’s technology does not perform well (e.g., heavy rain and snow, granular classification), and where cameras have been proven to handle better. Moreover, today’s LiDAR technology remains expensive to deploy at scale due to its high unit price and limited field of view.
As the Consumer Electronics Show (CES) in January sparked a new wave of autonomous vehicles (AVs) coming to the automotive market in the next few years, much focus as of late has been on the technology of these vehicles themselves. However, the technology embedded in road infrastructure is also beginning to see more conversation between service providers and municipalities.
With advancements in artificial intelligence (AI) and 5G network connectivity, smart-road infrastructure technology offers the promise of being added to many different roads, bridges, and other transit systems across the U.S. in hopes of improving real-time traffic analytics and tackling the most challenging road safety and traffic management problems. One technology at the center of this discussion is on the present-day use of AI-enhanced cameras and tomorrow’s promise of LiDAR technology.
Artificial Intelligence Enhances Camera Sensing Performance
Today, there are hundreds of thousands of traffic cameras deployed in the U.S. alone, and even millions more when CCTV cameras are considered. They are mainly used for road monitoring and basic traffic management applications (e.g., loop emulation). However, bringing the latest advancements of AI to these assets can immediately improve basic application performance and unlock more advanced software applications and use-cases.
AI and Machine Learning deliver superior sensing performance over traditional computer vision techniques found in legacy cameras. They enable more robust, flexible, and accurate detection, tracking and classification of all road users with algorithms that can automatically adapt to various lighting and weather conditions. In addition, they allow for predictive capabilities to better model road user movements and behaviors, and improve road safety. Agencies can immediately benefit from AI-enhanced cameras with applications such as road conflict detection and analysis, pedestrian crossing prediction and infrastructure sensing for AV deployments.
LiDAR Technology Cannot Fully Replace Cameras
LiDAR can provide complementary and sometimes overlapping value with cameras. However, there are still several safety critical edge cases where LiDAR technology does not perform well (e.g., heavy rain and snow, granular classification), and where cameras have been proven to handle better. Moreover, today’s LiDAR technology remains expensive to deploy at scale due to its high unit price and limited field of view. As an example, it would take multiple LiDARs at a hefty investment to be deployed in a single intersection, where just one 360-degree AI-camera can be a more cost-effective solution.
For many budget-focused communities, AI-cameras remain the proven technology of choice today. Over time, as the cost of LiDAR technology moderates, communities should evaluate augmenting their infrastructure with such sensors.
Eventually, Sensor Fusion Will Drive Strong Results
When the cost of LiDAR technology eventually sees an anticipated reduction, it will be viewed as a strong and viable addition to the AI-enhanced cameras that are being installed today. Similar to autonomous vehicles, sensor fusion would be the go-to approach for smart infrastructure solutions and would allow to maximize the benefits of both technologies.
See the table below for a comparison of relative performance of Cameras vs. LiDARs.
|
Feature |
Legacy Camera |
AI-powered Camera1
|
LiDAR |
AI-powered Camera and LiDAR Fusion |
|
Challenging lighting conditions (e.g., low light, glare) |
Low |
Medium |
High |
High |
|
Adverse weather conditions (e.g., snow, rain, fog) |
Low |
High |
Medium |
High |
|
Localization |
Low |
Medium |
High |
High |
|
Classification |
Low |
High |
Medium |
High |
|
Affordability |
High |
Medium |
Low2 |
Low2 |
2. Expected to improve with time
The use of a cost-effective and performing AI-powered camera today, combined with the great potential of LiDAR in the coming years, would help communities and municipalities achieve a win-win scenario today and tomorrow.
At the end of the day, the goal is clear in improving overall traffic flow and eliminating road crashes and fatalities, but the technology and implementation strategy has to be right in doing so. The technology monitoring our roads needs to change too, thus calling for the consideration of AI-powered cameras today with the promise of LiDAR tomorrow.
About the Authors:
Dr. Georges Aoude and Karl Jeanbart are co-founders of Derq, an MIT spinoff powering the future of connected and autonomous roads, making cities smarter and safer for all road users, and enabling the deployment of autonomous vehicles at scale. Derq provides cities and fleets with an award-winning and patented smart infrastructure Platform powered by AI that helps them tackle the most challenging road safety and traffic management problems.
Why the U.S. Infrastructure Bill Can’t Just Be About Building New Roads & Bridges
As U.S. President Joe Biden recently signed the Infrastructure Investment and Jobs Act into law, many outdated state and local roads, bridges and transit systems will be improved, not only to keep up with consumer demand, but also to provide increased safety in reducing crashes and fatalities. […] The good news is that smart infrastructure has already been on the radar as well as an area of focus for many communities around the U.S. For example, the city of Fremont in California has teamed up with CT Group and Derq to deploy AI intersection analytics systems as a key component of a safe and smart corridor project along a nine-mile section of Fremont Boulevard.
Why the U.S. Infrastructure Bill Can’t Just Be About Building New Roads & Bridges
As U.S. President Joe Biden recently signed the Infrastructure Investment and Jobs Act into law, many outdated state and local roads, bridges and transit systems will be improved, not only to keep up with consumer demand, but also to provide increased safety in reducing crashes and fatalities.
This Bipartisan Infrastructure Deal will [1] boost transit funding for communities all over the country by an average of 30% and will also help transit agencies reduce the current maintenance backlog by 15% and replace more than 500 aging subway, light rail, and commuter rail cars. It also aims to reduce traffic crashes impacting pedestrians and cyclists through a “Safe Streets and Roads for All” program.
Infrastructure Investments Must Go Beyond New Roads and Bridges
While this is the largest investment in American infrastructure in generations and marks an inflection point for American transportation, building and upgrading is simply not enough in the technological landscape we live in, where intelligent and autonomous transportation technology creates opportunities and has hopes of helping the U.S. achieve vision zero. It is one of the more revolutionary plans aimed at eliminating deaths and severe injuries due to road traffic as well as unsafe infrastructure.
Smart Infrastructure Must Play A Significant Role
Instead of just investing in traditional infrastructure or simply rebuilding highways and bridges, “smart infrastructure,” or the technology designed to enable safer, more connected and efficient roads need to be at the forefront of the future of the transportation ecosystem. From improved traffic and pedestrian safety to less congested roadways and lower CO2 emissions, and eventually city-wide autonomous vehicles deployments, the future of transportation is rooted in smart infrastructure [2].
Several AI companies have created this focus on smart infrastructure, as platforms have been specifically developed, using real-time advanced analytics, to have “eyes” and “brains” on the road infrastructure. These platforms allow for greater visibility overall for all road users, making the transportation not only safer, but more comfortable and with even better performance. For example, they can monitor the trajectories and predict the intents for all vehicles, cyclists and pedestrians in the field of view of traffic sensors, creating a comprehensive understanding of road user behavior that helps identify and predict potential conflicts or collisions and in-turn dangerous spots on the roads. This will also result in less-congested roads. For an average U.S. citizen, congestion costs 99 hours of their time and US$1,377 each year [3]. Smart infrastructure can prevent traffic backups by adjusting traffic signals when needed. Finally, the autonomous vehicle (AV) industry is realizing that smart infrastructure is an important piece of the puzzle when it comes to accelerating the ability to deploy more AV routes in different cities and countries in a safe and scalable way.
Smart Infrastructure Already Beginning in Some Areas
The good news is that smart infrastructure has already been on the radar as well as an area of focus for many communities around the U.S. For example, the city of Fremont in California has teamed up with CT Group and Derq to deploy AI intersection analytics systems as a key component of a safe and smart corridor project along a nine-mile section of Fremont Boulevard [4]. In Michigan, the DOT teamed up with Cavnue and other regional partners to develop a major connected and autonomous corridor between Detroit and Ann Arbor, starting with connected buses and expanding to additional types of CAVs [5]. These points of emphasis serve as a baseline from which the new Infrastructure Bill can build upon. In fact, the Infrastructure Investment and Jobs Act has already prompted deployment of several transportation technologies in programs such as “the Vulnerable Road User Research” and “Congestion Relief Program” programs [6].
Innovative Technologies at the Heart of Smart Infrastructure
The new infrastructure bill offers signs of progress in leveraging advanced AI and data analytics to smart infrastructure buildouts. For example, the bill establishes the Safety Data Initiative where the DOT can conduct projects, award grant, and also use other strategies that leverage new data visualization, sharing, and analytic tools that Federal, State, and Local entities can use to enhance surface transportation safety.
In order to truly build an infrastructure transportation network that serves as a global model, investments in US smart infrastructure cannot just be pilot trial programs. This technology must be central to the development of a nationwide transportation network that paves the way for the future of intelligent and autonomous mobility. As much as it is exciting to see a new and improved U.S transportation infrastructure now in the works, it is significant to realize that none of these roads will be more efficient and provide the utmost safety standards without smart infrastructure leading the way. By utilizing AI startup companies’ tested and proven technologies, road users across the country will see more reliable transit service, drive on smarter roads, and walk feeling much safer on the streets, creating more ways for people to get to work, to play, to access healthcare, and to visit friends and family.
About the Author:
Dr. Georges Aoude is CEO and co-founder of Derq, an MIT-spinoff powering the future of connected and autonomous roads, making cities smarter and safer for all road users, and enabling the deployment of autonomous vehicles at scale. Derq provides cities and fleets with an award-winning and patented smart infrastructure Platform powered by AI that helps them tackle the most challenging road safety and traffic management problems.
Sources:
2. https://visionzeronetwork.org/about/what-is-vision-zero/
3: https://inrix.com/press-releases/2019-traffic-scorecard-us/
5: https://www.michigan.gov/mdot/0,4616,7-151-9620_95066-537201--,00.html
6: https://www.congress.gov/bill/117th-congress/house-bill/3684
Is Smart Infrastructure the Key to Vision Zero?
Vision Zero is one of the newer strategies looking to eliminate deaths and severe injuries due to road traffic and unsafe infrastructure. First successfully implemented in parts of Europe, the strategy has recently gained traction in North American cities in addition to the direction of the new administration.
Is Smart Infrastructure the Key to Vision Zero?
Vision Zero is one of the newer strategies looking to eliminate deaths and severe injuries due to road traffic and unsafe infrastructure. First successfully implemented in parts of Europe, the strategy has recently gained traction in North American cities in addition to the direction of the new administration [1]
Since the inception of roadways and traffic, most drivers were conditioned to think fatalities on the road were inevitable, but the reality is that there are now many resources that can prevent these tragedies. The key is to take a proactive, preventative approach that prioritizes traffic safety as a public health issue. With more than 40,000 people killed in crashes in the US in 2020, something needs to start changing immediately. The significant loss of life exacts a tragic toll. Moreover, pedestrian deaths in 2020 increased by 21 percent from 2019 – even with fewer vehicles on the road that year [2]. Not only is there personal loss, but also deep community impacts including economic costs and emotional trauma as well as increasing taxpayer spending on emergency response and long-term healthcare costs.
Committing to Change with New Strategies and New Technology
Vision Zero specializes in showing a different approach to traffic safety. It is much different than the traditional approach used today in that it starts with the ethical belief that everyone reserves the right to experience safe use of the roadway, and it is the shared responsibility of policy makers, system designers and technology experts to ensure safer roadways. The traditional approach of traffic pushes responsibility onto an individual, however with system designers working together to create smart infrastructure through the new strategy of Vision Zero, various different aspects of the roads and traffic will ultimately become safer. Vision Zero also differs from the traditional approaches by acknowledging the many factors that go into creating safe mobility in communities. Collaborations are integral between traffic planners, engineers, and policy makers alike. Committing to Vision Zero means unlearning old ways of the roads and making room for new strategies to better the transportation infrastructure we use every day. This means that system designers and policy makers are expected to improve the roadway environment and policies in addition to other related systems in order to dramatically lessen the severity of crashes [1].
Advanced technologies including video analytics solutions use artificial intelligence (AI) and machine learning approaches to analyze large quantities of video streams in real-time to then provide actionable insights on complex traffic situations including road hazards, congestion and traffic collisions. For example, companies like Waycare, Derq and Applied Information provide software that analyzes behavioral patterns of vehicles, pedestrians, and traffic flows from existing traffic infrastructure in real-time to identify and predict potential road incidents. These advanced video analytics can then activate a pedestrian blinking sign to alert a distracted driver from colliding with a pedestrian about to cross the road. They can also alert a city operator so emergency responders can be dispatched to the location of the incident if a collision is detected. Furthermore, real-time AI analytics provide additional context in the form of incident identification, near-miss heatmaps and accurate traffic counts to traffic engineers and operators, which ultimately allows them to better understand traffic patterns and proactively improve the safety of roads.
The U.S. Administrative Plan for Infrastructure
The Biden administration released a proposal to spend an estimated $1 trillion on infrastructure projects that both lawmakers and the White House will now work to get through Congress [3]. The proposed bill was negotiated among a group of bipartisan Senators that includes a large piece of the bill targeting innovations for smart infrastructure. The administration’s transportation infrastructure goes hand in hand with the Vision Zero strategy as they are both working to fix the same problem. Through increased funding for roadway projects, the bipartisan proposal puts roadways on the right track to decrease fatalities, and hopefully lead to zero fatalities over the years.
It’s estimated that the infrastructure plan will spend $312 billion on surface transportation projects, with $109 billion invested in roads, bridges and other major projects [4]. From there, $20 billion is estimated to be put toward improving the safety of all roadway users, with a specific emphasis on bicyclist and pedestrian safety [5]. The proposed plan includes increasing funding on existing safety programs, while also creating a new program called “Safe Streets for All.” This new program will provide funding for state and local governments’ Vision Zero plans.
Preventing roadway incidents is key to Vision Zero, and advanced technologies like real-time video analytics play an integral role in smart infrastructure. While legislation is a good first step, it will not work without new technology to enable better road safety. It’s important that local city and community leaders adopt the mentality of Vision Zero to enable smart infrastructure advances. While access to the data collected through sensors being deployed is a crucial first step, the implementation of technology solutions is necessary to move forward to actually improve road safety and reduce traffic fatalities.
About the Author:
Dr. Georges Aoude is the CEO and Co-Founder of Derq, an MIT spinoff powering the future of connected and autonomous roads, making cities smarter and safer for all road users and enabling the deployment of autonomous vehicles at scale. Derq provides cities and fleets with an award-winning and patented smart infrastructure platform powered by AI that leverages existing traffic cameras and sensors to help them tackle the most challenging road safety and traffic management problems.
Sources:
1: https://visionzeronetwork.org/about/what-is-vision-zero/
2: https://www.nbcnews.com/health/health-news/mind-boggling-pedestrian-deaths-surged-2020-despite-fewer-cars-road-n1267910
3: https://www.wsj.com/articles/biden-senators-agree-to-roughly-1-trillion-infrastructure-plan-11624553972
4: https://www.cnbc.com/2021/06/24/infrastructure-deal-talks-biden-invites-bipartisan-senators-to-white-house.html
5: https://www.atssa.com/Advocacy/ArtMID/2254/ArticleID/423/Biden-proposes-2-trillion-infrastructure-plan
The State of Intelligent Transportation in 2021
It’s reported that road collisions kill approximately 1.3 million people and injure tens of millions more every year. It is of the utmost importance to leverage the advancements in various technologies including artificial intelligence (AI) to address this hidden, global epidemic. The use of V2X (vehicle-to-everything) enables communication between vehicles, as well as between vehicles and roadside communication systems for road safety. […] In addition, various sensor technologies including cameras powered by AI are enabling transportation agencies to better understand roadways than ever before with applications ranging from accurate traffic counts to predicting spots on roads that are most prone to incidents.
Posted by Dr. Georges Aoude, CEO and Co-Founder at Derq
It’s reported that road collisions kill approximately 1.3 million people and injure tens of millions more every year [1]. It is of the utmost importance to leverage the advancements in various technologies including artificial intelligence (AI) to address this hidden, global epidemic. The use of V2X (vehicle-to-everything) enables communication between vehicles, as well as between vehicles and roadside communication systems for road safety. In fact, within three to five years, this will be a factory feature on almost every vehicle. In addition, various sensor technologies including cameras powered by AI are enabling transportation agencies to better understand roadways than ever before with applications ranging from accurate traffic counts to predicting spots on roads that are most prone to incidents. It’s even predicted that the global intelligent transportation system (ITS) realm is expected to rise to an estimated value of over $63 billion by 2026 [2].
Autonomous vehicles (AVs) operating within communities as shuttles is one example of this technology currently being used to improve safety. For example, AVs and roadside AI platforms “communicate” with each other to improve traffic flow and help protect both pedestrians and vehicle riders. Pedestrians and hazards are detected by traffic cameras, which are then processed by the AI “brain” in real-time to send alerts back to the vehicle. This means an AV would get notified of a pedestrian or potential road hazard before it even comes into view.
Current And Future Utilization of AI Technology For Safer Roads
Volkswagen announced in 2020 that it intended to install the V2X antennae in the newest generation of its Golf model [3]. Ford also plans to deploy C-V2X technology in all new 2022 vehicles [4]. In Dubai, the goal is to have 25% of its transportation autonomous by 2030 [5]. One issue that remains is that today’s road infrastructure is not ready to support AV deployments. There is a clear technology gap that needs to be filled to open the door for additional opportunities for innovative solutions. However, many road agencies, such as the Departments of Transportation in Michigan (MDOT), Florida (FDOT) and Ohio (ODOT & DriveOhio), are working to make road infrastructure smarter to help save lives by deploying AI enabled traffic systems. This technology also helps these agencies prioritize infrastructure investments and improve their road safety.
Technology companies like Miovision, Waycare, Derq and Lyt are all playing a vital role when it comes to AI being incorporated with road infrastructures. They’re leveraging technology to prevent traffic fatalities and sharing their platforms with various agencies to ensure open communication and information. Collaborating with transit authorities and law enforcement is just the beginning for what’s to come regarding safer roads. Some of these analytics platforms are partnering with automotive companies and cities to provide their AI-powered systems to enable safer and smoother deployments of autonomous vehicles.
AI Technology and Analytics Expanding To Other Industries
With so many industries are already incorporating AI technology into practice, more industries are sure to follow as the technology and reporting continues to advance. The automotive insurance industry can leverage AI with video analytics to streamline claims management. Utilizing AI technology will supply insurance companies with more complete information for processing as well as save them both time and money.
There’s no question that the implementation of AI into various other industries is sure to come as those currently utilizing the technology continue to analyze data that had previously never been available.
Sources:
1: https://www.who.int/news-room/facts-in-pictures/detail/road-safety
2: https://www.mccourier.com/global-intelligent-transportation-system-its-market-recent-industry-trends-and-projected-industry-growth-2020-2027-top-players-thales-group-siemens-garmin-ltd-cubic-corporation-ef/
3: https://www.cnet.com/roadshow/news/volkswagen-new-golf-cars-will-use-v2x-to-talk-to-each-other-wirelessly/
4: https://www.smartcitiesdive.com/news/ford-cv2x-tech-new-vehicles-2022/545412/
5: https://edition.cnn.com/2020/01/21/middleeast/technology-reduce-traffic-accidents-intl/index.html
Can existing cameras and AI help us all #staysafe? Yes, and it won’t take much.
Our award-winning AI predictive algorithms, first used for road and pedestrian safety, are being leveraged to help cities tackle COVID-19, by measuring the distance between people, how long they interact, whether face masks are being used or not, and if establishments are following occupancy requirements.
Derq's AI platform detecting adherence to COVID-19 social distancing guidelines
May 4, 2020 - Our lives have changed because of COVID-19. We must be vigilant and aware of our surroundings more than ever before. Governments and societies are slowly opening up, experimenting with the effect of safeguarding people’s livelihoods while maintaining community health. As vaccines are still under development, our communities are recommended to follow strong guidelines of social distancing to protect ourselves and our loved ones, and trust that others are doing the same. This will be our new normal.
Can technology help? Yes. Most COVID-19 infections happen in community spaces, areas we used to always visit, places that are already covered by security cameras for safety. These same cameras can also be used to ensure our health. Adding a smart analytical layer on top of camera feeds can help implement the new global safety standards and free citizens from their homes. And by leveraging existing infrastructure, we achieve the fastest and most cost effective way to safeguard against the disease spread while maintaining the privacy of our communities. In these videos (link #1, link #2), one can get a glimpse of the technology in action.
When Derq was asked how we could help cities tackle COVID-19, we rose to the opportunity. Citizen safety has been our primary focus since Derq’s inception. Our award-winning AI predictive algorithms, first used for road and pedestrian safety, are being leveraged to measure the distance between people, how long they interact, whether face masks are being used or not, and if establishments are following occupancy requirements. We can produce actionable results that we report to a customized dashboard. We can also automate and push notifications to the community members and leaders. And, we can do this in the places that matter most, such as roads, malls, hospitals, offices, factories, parks and beaches.
Of course, we cannot do it alone. Governments and businesses are our partners to implement innovative society-wide solutions to ensure the safety of their employees, customers and citizens. The timing is critical and we are eager to help. Together, we will come out of this pandemic stronger, safer and healthier.