DARPA prototypes 'breakthrough' cyberattack 'hunting' technology

DARPA and BAE Systems are prototyping a new AI-empowered cybersecurity technology to fight new waves of highly sophisticated cyberattacks specifically engineered to circumvent the best existing defenses.

The program, called Cyber Hunting at Scale (CHASE), uses computer automation, advanced algorithms and a new caliber of processing speed to track large volumes of data in real-time, enabling human cyber hunters to find advanced attacks otherwise hidden or buried within massive amounts of incoming data.

DARPA information explains the technology as “adaptive data collection” able to conduct real-time investigations by sifting through enormous amounts of information not “trackable” by human defenders.

“The CHASE program seeks to develop automated tools to detect and characterize novel attack vectors, collect the right contextual data and disseminate protective measure both within and across enterprises,” DARPA CHASE Program Manager Jennifer Roberts said in a written statement.

Working in tandem with DARPA, a BAE Systems scientist says the potential promise of these advanced techniques is quite significant, because there is often simply not enough storage and memory to monitor nearly 80-percent of trafficking data goes undetected in large enterprise networks.

“Cyber hunt teams are currently massively overburdened and can only look at a small percentage of data collected using filters. Advanced adversaries take advantage of this,” Sam Hamilton, BAE Systems Chief Scientist, told Warrior Maven in an interview. “Sophisticated adversaries understand today’s cyber defense chain very well and are building things to defeat it.”

DARPA’s CHASE Broad Agency Announcement emphasizes this challenge and explains that large enterprise networks generate more data than there is available storage.

“…the fraction of cyber data stored within distributed databases still exceeds analysis capabilities. Proposed research should focus on dynamic approaches to accelerate cyber hunting via extraction of the right data from the right device at the right time,” the DARPA BAA writes.

Hamilton further specified that increasingly sophisticated adversaries are developing methods of hiding attack “footprints,” or weaving them into data streams not likely to be flagged at high-priority by cyber defenders.

CHASE uses “adversary resistant” machine learning, developers explain; the aim of machine-learning is to build automation able to organize and analyze new information by identifying patterns, placing things in context and comparing new data against very large historical databases.

As part of the technology, emerging methods of computer automation will also be used to “disseminate protective measures,” DARPA information explains.

The program is currently in phase one of a three-phase process which aims to bring an operational technology to the US military services in about three years, provided the technology properly matures.

“CHASE aims to prototype components that enable network owners to reconfigure sensors…at machine speed with appropriate levels of human supervision,” Roberts writes.

“We use advanced modeling to detect and defeat cyber threats that currently go undetected in large enterprise networks,” Hamilton explained.

Cyber defenders typically categorize into high-and-low priority areas of data flow, because they are forced to leave large amounts of traffic, deemed lower priority, uninvestigated. CHASE is constructed to track attack trails not typically flagged as high priority, he said.

“Details necessary to confirm these categories of attack or trace back their effects are rarely stored long term for potential forensics due to massive data storage requirements,” Hamilton added.

This early effort seeks to leverage the principle advantage of AI-enabled machine learning, namely that computer automation can process an exponentially greater volume of information while also placing data in its proper context. This is accomplished, at least in part, by using algorithms to analyze data – at times in milliseconds.

“Machine learning dynamically reconfigures sensor deployment, so you are capturing much more detail which you could not otherwise do,” Hamilton said.

The concept is to not only thwart commonly used malware, phishing and denial-of service attacks but also defeat much more elaborate, sophisticated kinds of attacks.

“An advanced piece of malware could be a program designed to hide in computer memory or on a router,” Hamilton explained.

More Weapons and Technology - WARRIOR MAVEN (CLICK HERE)