The Role of Machine Vision in Modern Manufacturing
industry trends
As we read this article, our nation celebrated a historic moment in space research on 23rd August 2023 – as ISRO’s mooncraft, the Chandrayaan-3 lander – VIKRAM, soft landed on the surface of the moon, making us the first nation to land on its southern polar region. Launched earlier, the moon orbiter Chandrayaan-2 played a crucial role in identifying a safe landing spot for Chandrayaan 3. It continues to act as an intermediary between the lander and ground stations.
In recent years, significant technological strides have been achieved in earth and space research across the globe, particularly in the field of terrestrial satellite imagery, resulting in notable improvements in spatial and temporal data resolution. A prime example of such progress is ISRO’s launch of the high-resolution earth observation Cartosat-3 satellite 2019, providing an impressive resolution of 0.28 meters in panchromatic imagery of the earth’s surface. ISRO’s geostationary navigation satellite system, NAVIC, continues to play an instrumental role in enhancing numerous positioning applications in surveying, defense, transportation, and resource monitoring, to name a few. These advancements, along with the unprecedented acceleration witnessed in AI, specifically in the Computer Vision (and language) domain, have greatly expanded the potential for technological innovations in satellite image analytics on a global scale.
The availability of multiple streams of geographical data has unlocked the possibility of developing applications at scale that analyze such data and derive meaningful insights from it. Geospatial analysis is a technique that studies and interprets geographic data, extracting spatial, spectral, and temporal relationships that help businesses and governance make essential decisions. To produce these insights, it incorporates diverse information about the earth’s surface, gathered from sensors like navigation receivers, optical and radar imagery, and seismic data. Derived data such as land use, land cover, real-time transportation networks, administrative regions, and biodiversity zones also fall within its scope.
Advanced techniques and tools, including remote sensing and geographic information systems (GIS), are harnessed for the analysis. This is combined with cutting-edge computer vision algorithms to visualize and comprehend spatiotemporal correlations and patterns within the data. Innovations in machine vision models, especially the advent of vision language models, promise to enrich geospatial analysis further, allowing more complex extractions of geographic and business contexts and facilitating informed decision-making for businesses and governments.
IVA Geo: From social experiment to commercial tool
In the scenario of difficulties presented by the COVID-19 pandemic, the idea for IVA Geo (formerly recognized as IVA HWKI) was born. Fractal crafted a machine vision infrastructure for drones to aid communities in containment zones facing resource limitations. Partnering with one of India’s largest drone manufacturers, we aimed to capture and analyze behavior in these communities, particularly where shared sanitation facilities made social distancing challenging. Employing advanced data science techniques, we could support local authorities in providing essential medical resources and establishing guidelines to mitigate the virus spread. The project, a part of Fractal’s social initiative for the common man in his struggles during the pandemic, garnered attention from other government agencies, offering us the opportunity to contribute to more exclusive projects.
Fractal’s focus with IVA Geo has since broadened from image and video analytics with drones to address the challenge of climate change with satellite imagery, its impact on businesses, and the necessity to develop strategies to mitigate these effects. This led to the development of IVA Geo, a collection of AI models and APIs that harness the potential of remote sensing and geospatial analytics.
The current set of services
Since its formal inception as a commercial platform for assessing hurricane damage for a property insurance company by offering detailed analyses from multimodal imagery — IVA Geo has expanded its application. Its advanced modeling capabilities hold tremendous potential for various industries. They are being expanded to include other NAT CAT events like floods and wildfires, highlighting their potential to influence various industries and society.
● Agriculture can leverage it for comprehensive crop monitoring, accurate yield estimation, and precise land management.
● Mining industries can utilize this technology for compelling exploration, comprehensive environmental monitoring, efficient site planning, management, and mine closure.
● It’s also poised to offer its services to manufacturing, supply chains, and healthcare solutions.
Fractal’s goal with IVA Geo is to make Geo AI models less reliant on data and more adaptable to different scenarios simultaneously.
Fractal’s work currently relies on freely available, open-source analysis-ready data (ARD). These data sets consolidate multimodal satellite imagery from various sources, like Sentinel-1 and Sentinel-2, and other satellites equipped with optical, Synthetic Aperture Radar (SAR), and multispectral/hyperspectral sensors. Presently IVA Geo is in the process of training backbone models that cater to multiple applications, primarily focused on natural catastrophes. These models leverage Fractal’s ongoing internal research in vision language models, self-supervision, and model adaptation to reduce the platforms’ dependency on data, enabling adaptability to different application areas across various geographies.
In addition to these openly available data resources, commercial providers offer satellite image data tailored for specific applications. For example, the Geographical Insurance Consortium (GIC) caters primarily to the property insurance sector.
Data privacy
Balancing technological benefits with privacy is challenging, necessitating multiple stakeholders’ collaborative efforts in creating ethical guidelines and regulations. Privacy issues are managed nationally in some geographies, but the global nature of satellite technology calls for an international solution. While the General Data Protection Regulation (GDPR) has been significant in regional privacy protection, it doesn’t fully cover privacy issues with satellite imagery. At Fractal, client data is handled with stringent privacy terms. Internal research towards privacy preservation, like federated learning measures, is being explored to ensure data protection.
Sharp, clear images for optimal analysis
IVA Geo can accommodate scene variations without extensive retraining by leveraging its ongoing research in incremental and self-supervised / semi-supervised learning areas. Progress made in image super-resolution modeling, especially with the state-of-the-art diffusion models, has helped in tasks like damage detection, where data resolution can be challenging, leading us to develop models that enhance poor satellite image quality.
In addition, we are preparing our model architectures to infer with multimodal image streams such that the lack of resolution in one could be compensated by the other. This approach has the potential to provide insights comparable to high-resolution imagery that can reveal detailed aspects in situations where determining physical composition, like roof material detection or understanding the extent of damage spread in case of fires and oil spillage, is crucial.
IVA Geo has progressed in quickly assessing natural catastrophes using multimodal satellite imagery and ancillary data. Its capability to analyze and infer from a diverse range of image modalities, from simple color to multispectral and hyperspectral images, and then validate the results with additional ground truth information allows a thorough examination of Earth’s surface.
A vision of the future
The primary plan is to develop IVA Geo into a comprehensive Geospatial AI platform for analyzing climate change impact by automating the process of interpreting multimodal geospatial data for efficient decision-making. We aim to revolutionize disaster management through early warning systems and optimized response strategies. This platform is poised to become an invaluable instrument for governments and businesses. It will aid in fostering well-informed policymaking, pinpointing high-risk regions, and devising robust strategies for resilience. By improving the monitoring of assets and resources, IVA Geo will also contribute to more efficient management practices, leading to cost savings, enhanced operational efficiency, and improved sustainability performance.
Despite the massive success of foundation models in the language and vision domain, the progress of similar geospatial artificial intelligence models is still a work in progress. Tools like the Segment Anything Model (SAM) from Meta AI have significantly improved geospatial analysis. SAM and its variants are widely used for land use, land cover, and infrastructure element segmentation tasks. Current development efforts focus on enhancing SAM for a multiscale understanding of spatial attributes. Large Language Models (LLMs) are being expanded to automate geospatial semantic tasks such as identifying specific locations from textual and image descriptions and answering geographical questions. Fractal’s internal research efforts are also being extended towards developing intelligent agent frameworks to organize and execute data processing and modeling tasks autonomously, making geospatial analysis more streamlined and readily available.