On the road to autonomous driving, one safety weakness still needs to be overcome
ADASKY’s innovative thermal sensor system brings the solution
Numerous automakers and AV developers have announced plans to deploy fully autonomous vehicles on public roads by the beginning of the next decade. Most have started public testing in certain areas. But, before this deadline can be met and before anything over SAE Level 2 automation can be achieved, AV developers need to eliminate existing vision and perception weaknesses to ensure that vehicles can sense their surroundings 24/7, in any environment, and in any condition. for AVs regardless of the surroundings or conditions.
ADASKY’s innovative thermal sensor makes the vehicle see – even in poor visibility
ADASKY’s innovative thermal sensor makes the vehicle see – even in poor visibility
Far infrared technology has been around for decades in night vision goggles and security cameras, but its automotive applications have been limited to date because they were sold as pricey options for a few luxury cars. While long-wave infrared (LWIR) technology has been used in several markets for decades, ADASKY has optimized the cost structure, safety features, system architecture and performance, image-processing algorithms, and computer-vision solutions to match the unique needs of the automotive market. ADASKY believes that LWIR cameras will be one of the most important sensors in AVs and an essential part of a sensor suite for full autonomy. Because current sensing technologies have inherent perception problems, a human driver still needs to be ready to take control of the car at any moment. This will decrease the use and efficiency of autonomous driving vehicles on the roads in harsh environments and off road. According to disengagement reports from car and truck manufacturers testing AVs, one of the main reasons that human drivers have to take control in autonomous vehicle testing is due to adverse weather conditions.
Due to the rapid development of autonomous vehicle (AV) technology, the automotive industry is experiencing a seismic disruption that is predicted to affect everything from individual car ownership to the design of cities. Most major automakers and tier-one suppliers are currently investing billions into autonomous innovations, while tech companies such as Apple, Google and Uber are pouring massive financial resources into self-driving research and development.
A vital component on the road to full autonomy is the development of sophisticated sensors and software that enable AVs to “see” the world around them and to react independently and better than human drivers.
Sensing modalities, such as radar and cameras have been used for years in advanced driver assistance systems (ADAS). But with the advent of AV technology that depends on sensors to detect every possible object in its environment under all visibility conditions.
ADASKY’s thermal sensor works well while other vehicle sensors have perception issues
Each of these sensors has its strengths and weaknesses. Radar sensors can detect objects far away but cannot identify the object. Cameras, on the other hand, can more effectively determine whether an object is, for instance, another car or a pedestrian, but only at a closer range. For this reason, radar sensors and cameras are used in conjunction for a car’s forward collision warning with automatic emergency braking feature.
Like radar, lidar sends out signals and measures the distance to an object via a reflection of those signals, but uses light waves or lasers rather than radio signals. Moreover, the best lidar sensors provide a wide field-of-view whereas radar and cameras are more directional.
Unfortunately, the effectiveness of these sensors can be hampered by weather and other environmental factors. For instance, while radar can still detect faraway objects in heavy fog or haze, at night, or in other conditions, most cameras have near-field sight limitations that constrain their ability to see in foul weather and darkness.
Fortunately, a new type of sensor that employs Longwave (LWIR) Infrared thermal technology can fill the reliability gaps left by other AV sensors. LWIR has been used for decades in defense, security and construction, making it a mature and proven technology. ADASKY, an Israeli startup, is making LWIR technology available for demanding AV applications.
An LWIR-based camera uses far infrared light waves to detect differences in heat (thermal radiation) naturally emitted by objects and converts this data into an image. Unlike the common optical sensors used on cars that capture images perceptible to the human eye, LWIR cameras scan the infrared spectrum just above visible light and can, thus, detect objects that may not otherwise be perceptible to a camera, radar, or lidar.
Passive, no moving parts: but the sensor sensitivity understands the road
Unlike sensors like radar and lidar that need to transmit and receive signals to be functional, an LWIR thermal camera only collects signals and is, therefore, “passive.” With no moving parts, ADASKY’s initial LWIR product simply senses signals from objects radiating heat. Most importantly, an LWIR thermal camera allows AVs to see and understand the road ahead in almost any lighting or weather condition and can distinguish between living and nonliving objects and between vehicles, road surfaces and roadside vegetation.
Moreover, ADASKY’s LWIR camera can capture not only the thermal radiation or temperature of an object or material, but also its emissivity – how effectively it emits heat. Since each material has a different emissivity, an LWIR camera can sense every object in its path. The camera then uses this information to create a visual painting of a roadway, including, not only other cars, pedestrians, animals, and other objects, but also potholes, debris and more.
The sensitive thermal sensor detects objects more than 230 meters away
With a sensitivity of 0.05 Centigrade that allows for high- contrast imaging, the ADASKY thermal sensor can precisely detect vehicles, pedestrians, cyclists, animals, debris, and other objects more than 230 meters away, allowing more distance for an AV can react. ADASKY’s sensor is ideally suited for use cases in which other vehicle sensors have perception issues and enables awareness of a car’s surroundings as well as detection of living and general objects in harsh conditions, such as rain, snow, fog, haze, and complete darkness.
ADASKY’s thermal sensor uses a chip designed by its engineering team that enables shutterless functionality; this means that the vehicle’s vision isn’t mechanically blinded, even for a millisecond. It is the highest resolution thermal-perception solution for AVs with minimal size and weight and no moving parts.
ADASKY’s thermal sensor is specifically designed for the automotive industry and its strict quality, safety, and environmental standards. At 4.3 cm in length by 2.6 cm in diameter, ADASKY’s sensor is significantly smaller than other automotive cameras and has very low power consumption, making it ideal for the complex autonomous vehicle system in particular in electrified vehicles.
Unlike other LWIR cameras, ADASKY’s application-specific integrated circuit (ASIC), combined with state-of-the-art image- processing algorithms, delivers high-resolution video. Other LWIR cameras are affected by physical artifacts when pointing directly into the sun, creating a shadow or ghosting effect on the image known as a “sunburn” that can take anywhere from minutes to hours to disappear. To compensate, ADASKY’s sensor uses an algorithm called “sunburn correction” that presents a crystal-clear image for processing. ADASKY’s sensor can detect a change of less than 0.05 degrees Celsius and has refresh rates of 30 or 60 frames per second. Resolution is VGA (640×480, 307,200 pixels per frame), with available: 18°/13.5°, 30°/22°, 60°/46°,65°/47.5°, 90°/70° and more can be designed per demand.
The sensor also uses AdaSky’s proprietary algorithms (running on a dedicated ISP) to turn slight differences in the temperature and emissivity of objects into a high contrast graphic output.
Shutterless technology makes ADASKY’s sensor a game changer
According to ADASKY CEO Yakov Shaharabani, conventional infrared sensors have to recalibrate after a few minutes in order to be able to accurately detect the heat-emitting bodies again. But this process takes about half a second, during which no ambient images are available, which is intolerable for safety-related driving functions. “We have developed a patented image perfection algorithm that is the only one in the world to date that does not require a calibration pause and can thus constantly provide valuable information.”
Software and thermal tech – key elements to protect vulnerable road user
The artificial intelligence-based perception evaluation algorithms can detect even the silhouettes of people half-hidden by a vehicle, for example, on the thermal image. Compared to CMOS camera-based image analysis, this should cause far fewer false-positive situations in borderline cases. The resolution of the FIR is so fine that it can detect temperature differences of 0.05 degrees Celsius and the characteristic wavelengths of heat radiation from different objects. According to ADASKY, it should retain these properties up to a distance of 230 meters.
ADASKY’s sensor entering the global automotive market
ADASKY is developing several application modules of its sensing solution, each matched to the desired AV application required by customers, whether it be urban, rural, or highway driving – or a combination of all three or even in harsh environments. For example, on the
highway, it’s crucial to have long-range sensing so that if an object is detected, there is ample time for the vehicle to make the decision to stop – even as it travels at high speeds. In urban areas, having a wider field-of-view is prioritized in order to be able to detect pedestrians and cyclists on the sidewalk and at crosswalks. ADASKY’s solution can be adapted to specifically detect and analyze these and many other AV use cases.
“We see the sensor as the missing link that complements the sensor suite of camera and radar, because even in darkness, rain, fog and other adversities, it distinguishes objects from their surroundings based on their emitted thermal radiation”, Yakov Shaharabani, ADASKY CEO said. In North America, an OEM took a similar view, and Adasky was able to secure an initial supply contract in 2021. The sensor and AI-based software will initially be used in a sensor network for a situational awareness function starting in 2024, and later for an active emergency braking function and still later for automated driving at SAE level 4 for commercial vehicles.
ADASKY, the Israelian startup, has made an important step forward entering the European market beginning this year. In the “StartUp Challenge” Konnect and Volkswagen Commercial Vehicles screened more than 30 Israeli startups. ADASKY eventually won a paid proof-of-concept with VW Commercial Vehicles. ADASKYs technology will be evaluated for VW Commercial Vehicles L4 autonomous vehicles. “It’s an important validation for ADASKY’s lifesaving thermal technology,” said Shaharabani. “ADASKY’s sensor is market-ready and will allow us to deliver to our customers safe and top-quality mobility services, in all weather conditions for all vehicle applications on and off the road for cars and commercial vehicles.”
A groundbreaking new technology is automotive grade and ready for series production
ADSKY is ready for mass production, setting up a production line in Israel already, which started in low volumes but will ramp up significantly in a few years. An important goal of ADASKY is to enable mass market pricing by offering a complete LWIR solution and by designing and choosing components based on scalable technology. Pricing of sensors is significantly affected by volume, and at large volumes, thermal sensor systems will be priced competitively with other sensors on the market.
The application of ADASKY’s thermal camera:
https://www.adasky.com/adasky-thermal-camera/
ADASKY Automotive Camera multi-class object detection and classification:
ADASKY’s thermal vision technology sees where other sensors can’t: