Optimizing ML Operations on Edge: Real-Time Deep Learning Solutions on Edge Devices
Abhishek Chopde
Senior Data Scientist
AI@Scale, Machine Vision, Conversational AI
Deep learning is a technology that provides a machine with the ability to process visual images for the tasks that it is performing. For many years, it has aided in raising product quality, accelerating production, and optimizing manufacturing and logistics. This tried-and-true technology is now combining with artificial intelligence to drive the shift to Industry 4.0.
MLVS (Machine Learning Vision Solution) becomes more challenging when the collected data needs to be processed in real-time and operated in specialized environments. Are global institutions and enterprises equipped to implement MLVS effectively in real time?
Machine Learning operations
The success of any AI operation is the effective deployment and tracking of models in production through machine learning operations or MLOps. It is an operational development framework for ML applications that involve the digital architecture of the entire life cycle of developing, testing, optimizing, deploying, and monitoring ML models.
MLOps for Machine Vision
This is similar to how the human brain’s occipital lobe provides a control center for visual functions. The hippocampus provides one for learning, memory, spatial recognition, and navigation functions. MLOps for machine vision guide global businesses in training their operational machines for visual data recognition, data processing, and storage quickly and efficiently.
Far-reaching applications of Machine Vision
In 2023, MLVS is poised to impact fields such as sensor technology, 3D imaging, event-based vision, and model optimization, with applications ranging from absence/presence detection, barcoding, and surveillance to crowd control, defect detection, and logistics automation.
Part of the cause of the rapid development of MLVS is the increasing amount of image and video data available. With the abundance of data, there is a greater need for faster and more efficient processing times. You need heavy computational resources to train an intelligent MLVS model that automatically and correctly detects people or classifies objects.
However, only some organizations will have the required infrastructure for this computational need. To solve this problem, many organizations rely mainly on cloud computing to remotely process their data with the help of many third-party data centers. While this solves the computing problem, the weaknesses of cloud computing come to the fore with specific applications. There are some privacy concerns with transferring data to servers you have no control over, and any application that requires real-time processing at scale will run into latency problems.
The solution to this problem is edge machine learning (edge ML).
Deploying edge ML for real-time MLVS
Edge ML can process data locally at the point of collection. It addresses security issues by storing sensitive user data in the cloud. It also makes real-time data processing possible, essential for technologies like autonomous vehicles, shipment sorting facilities, and critical patient monitoring systems.
Many edge devices we come across are as small as a flash drive. They are adaptable to integrations and are available as boards and development kits. They can, therefore, easily be plugged in and installed into any system without any intrusion on the functioning of the main hardware.
Challenges to using Edge devices
While edge devices offer many benefits compared to current methods, there are some challenges with using this technology.
Edge intelligence risk
Edge intelligence refers to analyzing data close to where it is collected. While this provides advantages in computing performance, physical data poses a significant challenge, especially where edge technology is continuously mobile. For example, a drone monitoring territory of an enemy state close to its border could malfunction or be shot down, allowing it to be collected by the enemy military, resulting in a major strategic and surveillance failure.
Security
Protecting an edge device from advanced adversarial attacks and hacking by malicious users requires very sophisticated encryption of the edge device, the data, and the model, which can be costly and time-consuming to develop and implement.
Data scarcity
As with all ML applications, high-quality data is necessary for high-quality training. So, another challenge in edge ML is the scarcity of real-time training data for the model. The data is collected and processed in cloud-based services with a vast central database. But ML applications use real-time data for training/updating models on edge devices, which is usually self-collected by the device, thus having limited storage and processing capabilities compared to the large servers used in cloud computing.
Federated learning
Federated learning solves the problem of gaps in gathered data by the edge device. It enables the development of a single model trained on several different data sets from various sources without the parties ever needing to exchange their critical data. However, federated learning is only suitable for group training since there are still some concerns about the privacy and security of the data. Therefore, it is not exceptionally suited for ML operations that are highly clandestine or top-secret.
Data consistency
Data consistency is another challenge, and it occurs primarily due to the inefficiency of the sensors of the edge device – noise in the background or environment gets superimposed on “useful” collected data. To overcome this issue, companies will have to use data augmentation to effectively teach the model how to filter out the noise.
The breakthrough: Fractal’s forward-looking IVA models
Fractal has developed deep learning-based image and video analysis (IVA) models deployed on edge devices, allowing data processing at the source of data capturing. We have already implemented IVA models for some of our clients.
A cut above: IdeaForge drone surveillance
IdeaForge is one of India’s leading manufacturers and was ranked 7th among the top dual-use drone manufacturers in the world by Drone Industry Insight. They are a key supplier of UAV technology to the Indian Armed Forces, focused on surveillance and mapping solutions. Fractal is IdeaForge’s strategic partner in developing drone technology, especially for difficult terrains and border patrol areas, and is helping to realize the Indian Army’s “Year of Transformation” goal for 2023.
Sandalwood plantation intruder/theft detection through Edge ML
Fractal also worked with a client to provide night surveillance for a sandalwood plantation with a history of intruder-related issues due to the resource’s lucrative price point in India. We deployed a thermal-imaging camera mounted on a UAV, which can detect and collect emissivity and thermal sensitivity data and return the exact coordinates of intrusion to the security team in real time due to the rapid processing enabled by an edge ML system.
Figure 1: Fractal’s edge-optimized deep learning image and video analysis model
Fractal has stayed at the forefront of the edge ML technology through advancements in its IVA solutions, focusing on the following characteristics:
- Precision object detection and identification
- Lightweight and faster models for real-time insights through quantization and sparsification
- Patented advance encryption pipeline for data security and privacy
- Ground activity insights through image segmentation
- Improved UX through customer action identification
To enhance Fractal’s capability in platform-agnostic Video Surveillance, we have developed a Surveillance platform, “IVAHWKI” equipped with optimized and calibrated deep learning-based models capable of integrating with any environment on-prem, cloud, or edge/embedded devices.
Mapping a future of innovation in Edge ML in Machine Vision
An MLVS ecosystem traditionally collects data on location but processes it remotely on the cloud. The challenges associated with cloud, such as latency issues, present a problem for applications that need real-time data processing and instant decision-making.
Our competitors in the machine vision space use traditional setups such as cloud networking to deploy their solutions. But Fractal has leveraged edge ML to enhance our customers’ operational insights, so they achieve their critical and strategic goals effectively and efficiently.
MLOps on edge devices are attempting to operationalize the AI or ML life cycle, which includes various activities, from data preparation – through model training and experiments – to testing. Fractal will continue to innovate edge ML for vision systems to empower our customers as the global scope of this technology expands to hundreds of use cases.
- Share this post
