April 25, 2019
By Eleanor Crane
With record-breaking wildfires, extreme weather, and increasing population around pipelines and power lines, protecting critical infrastructure has become more important and more difficult than ever before. Aerial LiDAR datasets are a fantastic tool to understand the complexities of this infrastructure, which is reflected in the increasing adoption of LiDAR technology. LiDAR can be used to accurately detect vegetation encroaching on power lines and identify excavation and landslides that impact the safe operation of pipelines. But while LiDAR is promising, it is a sensor and not a solution. Unleashing its full potential means addressing three key challenges:
Using traditional LiDAR methods, a large-scale survey takes many months to be analyzed causing the value of the data to expire. The challenges of vegetation management can become intractable when there is a complete growing season between data collection and the ability to action the results. Where does this long delay come from? Heroic levels of manual effort. Using traditional processing methods, the classification/analytics on the point cloud will be done by analysts using desktop-based software. The scalability of this approach is limited, as the number of files that can be processed in parallel is capped by the number of analysts on staff. In addition, the time it takes for each file to be analyzed is directly proportional to the point density, so a high-quality scan will take longer to process. This can create a perverse incentive for a program manager to favor lower-density surveys, even though they provide less rich information about the scene. And while the data products generated by these traditional methods are good, the huge delay limits the effectiveness as a practical tool in infrastructure management.
Advances in technology can solve these challenges. Here are the technologies, available today, that the industry-leading utilities are leveraging to maximize the effectiveness of their LiDAR programs:
In order to fully capture the value of this rich data source, the LiDAR remote sensing community needs to rapidly decrease the turnaround time between the completion of a survey to the delivery of initial results. A great deal of the work required to identify critical infrastructure encroachments is based entirely on geometry, and so has great potential for automation. Software automation, including machine learning, has an amazing capacity for speeding up rote tasks. These algorithms, however, are not capable of displacing skilled operators, who will always have greater insight into the real world. Rather, the future lies in combining the unique skills of specialized analysts with the raw horsepower of AI: trained operators should be spending more of their time on the most interesting and challenging regions, while the AI should be able to rapidly churn through standard calculations and promote those challenging situations up to analysts for review. Similarly, as analysts and industry experts interact with the data outputs, any advanced insights they identify should be incorporated back into the machine learning training dataset for future improvement.
In order to meet the demands of these new high-capacity, automated workflows, an entirely different software paradigm is required. Traditional desktop-based applications have been effective at enabling analysts to interact with data, but as the industry moves towards increasing automation, these applications will struggle to be executed at the necessary scale. Cloud computing platforms, such as Amazon Web Services, Google Cloud, or Microsoft Azure, provide the obvious way forward in enabling large-scale computation without the need for massive capital and human investment. These cloud platforms are most amenable to deploying headless code, that is, software that is intended to run without any graphical user interface. Click-driven applications can be re-architected to support operations in the cloud, but this involves a time-consuming development process before deployment.
The faster path to large-scale deployment is to utilize a cloud-ready solution that has been built from the ground up to operate on massive datasets. Cloud providers have thousands of CPUs and GPUs available to execute processing jobs at a moment’s notice, so long as the software has been designed to utilize them fully. In order to make the most efficient use of the cloud compute resources, significant software infrastructure is required to dynamically scale the number of processors to the minute-to-minute demands of each processing job that runs. That tight control over specific cloud computing resources ensures the number of remote servers that are instantiated can be tuned up or down to meet the project needs in size, point density, and urgency.
This new software model, combined with the unprecedented compute resources available in the cloud, represent a new approach that is actually capable of quickly scaling up to meet the coming demands of LiDAR survey needs.
Justifying costs for a LiDAR program can be challenging when the data cannot be shared across the organization. A complete, high-density LiDAR survey is an incredibly rich data source that can provide valuable insights to many different customers within an organization but traditionally data has been siloed to a single department. By using the same survey to serve multiple departments, organizations can drive cost savings and increased ROI across the company. The same survey can be used to identify possible wildfire ignition points, map building footprints within a potential impact radius, and identify the volume of earth moved in landslide sites. Monitoring and mitigating the risks associated with each of these types of analyses will lie with different groups, but a well-architected software solution will make it possible for each group to maximize the value they derive out of the single survey. By making the cloud-deployable, automated workflow heavily modularized, it is much easier to deploy new analytics capability against the same dataset. Many workflows will contain common steps–the automated classification of the LiDAR point cloud is one of the most common–so a system that can utilize that existing data product in future analyses will enable faster turnaround times.
As the software ecosystem supporting cloud-based LiDAR analytics develops, the end user should also be able to develop and execute their own custom analyses without needing to be an expert in cloud computing. All of these benefits, however, can only be enabled through a forward-thinking architecture that understands and leverages the full capability of cloud-computing platforms along with the business needs that drive LiDAR data collection.
Critical infrastructure is increasingly threatened, and groups responsible for the monitoring of that infrastructure are being asked to do more with less. LiDAR-based programs have made significant strides but remain limited by scale, timeliness, and siloed data. The most progressive utilities are addressing these challenges by looking to new technologies to prevent threats from becoming incidents. By leveraging newly available cloud computing platforms along with a thoughtful approach to machine learning, we are on the cusp of a sea change in the scale of data that can be processed and the depth of insights that can be derived. Enview is bridging the gap between leading-edge cloud computing and machine learning techniques with the needs of energy and other infrastructure groups to set a new standard for timely and insightful data processing. Our technical work is driven by a mission to protect people and communities from the potentially catastrophic consequences of infrastructure failures in a changing environment. To that end, we deliver clear, actionable insights for our customers in their efforts to safeguard that infrastructure.
We’re building the next generation of computing, machine learning, and visualization products that can help map our ever-changing world and provide actionable insights to protect it.
Article originally published on LinkedIn: https://www.linkedin.com/pulse/tech-revolutionizing-lidar-programs-eleanor-crane/