A Pentagon team is working to automate the analysis of millions of hours of video collected by drones and sensors, but the three-star general leading the effort has bigger plans for tactical, more defense-based artificial intelligence.
Project Maven, led by the Defense Department’s Algorithmic Warfare Cross-Functional Team, launched in April to build on what is already done commercially in AI from an operational standpoint, but redirected to the warfighter. It plans to deliver AI-based algorithms to tactical unmanned aerial systems, and to specific wide-area motion imagery processing and exploitation systems by the end of 2018.
-- Sign up for our weekly newsletter to receive the latest analysis and insights on emerging federal technologies and IT modernization.
“The pebble called Maven has been dropped into the [Department of Defense] operational pond," said Gen. John N.T. Shanahan, director of defense intelligence in the Office of the Under Secretary of Defense for Intelligence. "The ripples have started to spread; they get bigger and bigger and get faster and faster as they expand. But this is the starting point.” He spoke at the Nov. 1 NVIDIA GTC event in Washington, D.C., about the project’s future.
In just six months, the team has had six companies on contract developing algorithms.
“The first algorithms will be delivered — I say will, not plan to — will be delivered in December of this year,” Shanahan said, and the team will follow with sprints throughout 2018.
So, What’s the Problem?
Ultimately, the department lags behind the commercial industry in automation. It has the most platforms of data it has ever had from an increasing number of sensors that are only becoming more advanced and high definition, and not enough people to analyze or exploit it all.
But more people is not the answer.
“What that will do is prevent us from making the cultural changes that are required to recognize that the world has changed to a data-driven environment,” Shanahan said.
However, there needs to be enough training data to build the first algorithms to analyze video from UAS. Currently, analysts spend up to 12 hours at a time dissecting full-motion videos.
“What’s most egregious about this approach is these are the finest, most well-trained analysts in the world,” Shanahan said. They are combat hardened and trained, but are frustrated knowing what is commercially available while they’re still cutting and pasting from Excel spreadsheets to PowerPoint slides.
They are also tasked with manual processes related to computer vision, confidence-building, geo-registration and data tracking. Shanahan said wide-area motion imagery sensors can take about 20 analysts working 24 hours a day to exploit just 6 to 12 percent of images. The rest disappear.
“It’s an unacceptable waste of resources, which is one of the best imagery sensors we have available to us,” he said.
The recognition of private sector solutions sparked work on Project Maven.
“More people is not the solution to the problem,” Shanahan said. ”Better tools are the trade craft, and algorithms are the solution to the problem.” Computers can simply do it better.
The Concept
Shanahan calls the effort “prototype warfare.” It’s how he envisions the future of AI in the Defense Department. It means getting things done quickly, fielded and tested with user feedback, user engagement and user refinement throughout the project, because the first prototypes and algorithms won’t be perfect.
He also stressed the urgency of operationalizing AI and machine learning across the entire department, beyond just the intelligence enterprise, full-motion video exploitation and research and development. And to do it rapidly and at scale through partnerships with organizations, companies and academia.
To get there, the department needs to augment, automate and amplify. Augmenting means not replacing analysts, but allowing them to focus on cognition and reasoning while computers and machines automate. Then, humans and machines will need to work together, as they are far more powerful as one than apart.
The Many Moving Pieces
Project Maven relies on data management, compute power, algorithm development and integration, user engagement and optimization. Data is a critical dependency for computer vision. So far, the team has labeled hundreds and thousands of datasets, but the goal is to automate full-motion video pre-processing and labeling on high-computing power so it can be used for algorithm development and rapid prototyping. The team is exploring transfer learning, reinforcement learning and even photo-realistic video simulations to do so.
User engagement is now factored in from the very beginning. Analysts sit down with software engineers and data scientists at the start so all parties understand how the data is processed and the nature of the department’s special missions.
The first algorithms have already been produced. Out of several created, four are ready to be tested and evaluated. Right now, those algorithms are on processing or exploitation workstations, but eventually, Shanahan wants to have these algorithms at the tactical edge, on platforms and on sensors.
“I would be so bold to suggest the Department of Defense should never buy another weapons system for the rest of its natural life without artificial intelligence baked into it,” he said.
Project Maven and Beyond
After Project Maven tackles video, Shanahan wants to expand the AI capability to document exploitation, collection management, model and simulation wargaming, and indication and warning processes.
He’s particularly interested in the futuristic possibilities of model simulation and wargaming, and its ability to generate an outcome by only being told the desired end state. From a tactical standpoint, instead of taking months to go through four or five excursions to figure out what might work and what won’t, a system could play itself millions of times over and allow humans to test various situations.
“Don’t get me wrong, the human element never changes . . . commanders will still be commanders,” Shanahan said. “But the ability to do that and understand what the excursions will show will make a difference to the department forever. I’m confident in that.”
And beyond the intelligence enterprise, AI and machine learning can help with other major areas of the department, like cyber detection.
“Maven is designed to be that pilot project, that pathfinder, that spark that kindles the flame in front of artificial intelligence across the rest of the department,” Shanahan said. It aims to set an operational AI foundation even after those currently working on the project leave.
For example, it’s designed to accelerate investments that set the research agenda. Not all commercially available AI solutions are fit for all defense-based problems, so Project Maven can help guide and shape investments for the commercial world, creating a centralized strategy easily executable.
This strategy can even expand to what Shanahan calls national enterprise enablers; cloud, AI simulators, quantum computing, and encryption and decryption. The department already has significantly invested in cloud, but it’s mostly in the form of storage and search capability.
“We’re looking for cloud that’s optimized for AI and [machine learning] both for training and optimization,” Shanahan said. Algorithmic warfare is not going to be effective without the proper cloud compute.
Further Reaching Potential
Eventually, Shanahan wants this project’s implications to not only lead to an AI defense strategy, but to a national government AI strategy — but it will take people, governance, methodology and technology working together to do so.
The team works with the Defense Innovation Unit Experimental, the Army Research Lab and the Strategic Capabilities Office for Project Maven. They’re partnering with traditional contractors of thousands of people down to startups of 10 to build these algorithms, and the work they’ve done in the past six months has been under the confines of the Defense Acquisition Regulations System.
It’s been challenging, but to create and sustain an AI-ready culture, “we have to accelerate the operational integration of AI and [machine learning] throughout the entire fabric of the Department of Defense and to me, Maven is that pebble,” Shanahan said.
