Hespanha: Co-PI of UCSB ACTION Institute

Cyberattacks have become increasingly common, sophisticated, and costly. Researchers at the new NSF-funded, UCSB-led ACTION Institute intend to team humans and AI to protect mission-critical systems and infrastructure

Excerpt from The COE/CLS Convergence magazine (Fall 2023) – "Taking Bold ACTION to Bolster Cyberdefense" 

The proposal that UC Santa Barbara researchers submitted to the National Science Foundation (NSF) for a grant to develop new ways of combating cyberattacks, with artificial intelligence (AI) as a main component, included a hypothetical attack scenario. In it, a group of individuals aligned with a hostile nation-state launch a sophisticated multiphase attack against key infrastructure elements of a fictional city: New Esperanza. The scenario is a chillingly realistic representation of how sophisticated hackers can gain access to inadequately defended cyberconnected systems.

The proposal succeeded, and last May, UCSB was named the lead institution in a five-year, $20 million NSF grant to pursue new approaches to cybersecurity linking humans to AI agents, and multiple agents to each other. UCSB computer science professor Giovanni Vigna is the institute’s director. He is joined by fellow co-PIs (and UCSB professors) Christopher Kruegel (computer science), who has worked with Vigna on seminal research in the areas of intrusion detection, malware analysis, and threat intelligence; Ambuj Singh (computer science), a renowned expert on machine learning on networks and human-AI teaming; and João Hespanha (electrical and computer engineering), a world expert in control systems, game theory, and optimization. In addition, the NSF Institute for Agent-based Cyber Threat Intelligence and OperatioN (ACTION) brings together 21 other top AI researchers from ten other U.S. universities in a collaborative effort to develop revolutionary new forms of integrated cyberdefense.

Vigna describes the ACTION Institute researchers as “some of the very best people in AI and security, who have been at the forefront of expanding the foundations of AI, machine learning, game theory, and computer security.” They and each of their institute colleagues will work primarily in one of eight highly integrated and interdependent research thrusts — four each in foundational AI and cybersecurity.

Paralyzing a City

The attackers in the New Esperanza scenario aim to create uncertainty and chaos by shutting down the city’s water- and power-distribution infrastructure, which are controlled, respectively, by the Great Aqueduct and the Las Palomas power plant. The control systems for both are integrated with New Esperanza’s smart-city system, which incorporates open-source software to monitor and distribute power, water, and other services.

The nation-state actors gather intelligence about the targets, identify open-source software used in the smart-city system, and then use false identities to contribute a vulnerable software component to the project, which goes undetected. They use credentials obtained from underground forums to connect to the virtual private network (VPN) of the aqueduct system, gain entry to various connected systems, introduce and exploit a vulnerability to obtain administrative access to the main server and upload a wiper malware component, all in ways beyond the ability of the systems to detect. After a few more steps, the attackers cause the power plant to cease operations, such that the smart-city system cannot be controlled. Simultaneously, they activate malware that they installed, shutting down the aqueduct and blocking water flow to New Esperanza. The city is paralyzed, and chaos ensues.

Details of the attack included in the NSF proposal highlight multiple fail points at which suspicious or otherwise anomalous activity went undetected, exactly the kind of vulnerabilities that can bring down the operations of any connected entity that is inadequately protected. ACTION Institute researchers plan to bring forward innovations in AI and its application to cybersecurity that will protect critical infrastructure from sophisticated attacks like this one.

Fighting Back: Challenges of Time and Scale

Currently, the task of defending against cyberattacks depends largely on the skills, intuitions, and experience of human defenders, who must attend to all the elements of a typical cyberdefense life cycle: risk assessment and prevention, detection, attribution, and response and recovery.

As a result of the ever-increasing number, complexity, and sophistication of cyberthreats, however, the effectiveness of humans who staff the thousands of security operations centers (SOCs) at the nation’s hospitals, financial institutions, government agencies, and other large connected entities can no longer respond with adequate speed or at sufficient scale to combat next-generation threats. There are simply not enough people, Vigna says, “to monitor what’s happening in a network of mind-boggling complexity, make sense of it, and identify and resolve problems in a timely fashion.

“Solving that time-and-scale problem will require automation,” he adds, “but it has to be smart automation, and that means AI.” 

The ACTION Institute is part of a $140 million investment by the NSF, in collaboration with other federal agencies and stakeholders, to establish seven new National Artificial Intelligence Research Institutes, itself part of a broader federal effort to advance a cohesive national approach to AI-related opportunities and risks.

Says Vigna, “The ACTION Institute mission is to find new AI concepts and constructs that can be used to create new security applications that will change how mission-critical systems are protected against sophisticated, ever-changing security threats.”

That will occur on two broad fronts: one is fundamental AI research — finding new ways for AI to model and reason about knowledge; the other is creating interaction and integration between and among humans and autonomous AI agents.

Stacking the Defense

ACTION Institute researchers aim to accomplish their mission by building a new AI stack, “a set of integrated tools that work together like a package that allows you to build AI-powered applications,” Vigna explains. The AI stack will provide ways for intelligent agents to learn new facts and reason about them, communicate with humans and with each other, and support the planning of their actions.

These basic AI capabilities become the building blocks for developing security intelligent agents, such as agents that identify vulnerabilities in software before they are exploited, or intelligent agents that are able to suggest an effective remediation procedure after a breach has been detected.

One notable aspect of this AI stack is its focus on logical reasoning: While current AI approaches to cybersecurity mostly focus on machine learning (that is, the learning from large amounts of data), the vision brought forward by the ACTION Institute focuses on being able to apply deductive and inductive reasoning on what is observed in a computer network. This will support novel ways to understand the security posture of critical systems and deploy effective protections. 

“This new AI stack will need to operate in a world where attackers will also use automation and AI to overcome cyberdefenses,” João Hespanha explains. “Designing security systems must therefore involve reasoning about how the actions of one AI agent will affect the behavior of another. This type of reasoning is needed to make sure that whatever protection mechanisms we deploy to protect our systems do not create a completely new vulnerability.”

Read More About the Topics Below: COE/CLS Convergence magazine (Fall 2023) – "Taking Bold ACTION to Bolster Cyberdefense" (full article pgs. 13-17)

• Reasoning & Human-Agent Teams
• Trust, Ethics, and the AI Landscape
• Collaboration and “Polarizing” Interest
• A Stack at Market?
• Education, Workforce Development, Community Engagement