Summary

Cyber (computing and communication) technologies underpin every facet of the U.S. economy, are nearly ubiquitous in daily life, and critical for national security. Cyber and cyber-enabled systems are rapidly growing in both complexity and scale, and—despite significant progress—are outpacing the capacity to keep them safe, secure, and resilient to disruptions.¹ Cyber resilience challenges arise from unintentional technical and operational flaws as well as deliberate misuse. Many resist solutions to these challenges because of their technical difficulty, while others resist them owing to intertwined technical, human, business, and policy factors.

Some cyber problems are well defined, and progress toward their solution would significantly improve the safety and resiliency of cyber and cyber-enabled systems. This set of problems, called “cyber hard problems” in this report, stands in contrast to the many other computing and communication problems whose solutions would be beneficial in some way but would not improve resiliency in a meaningful way (e.g., new encryption algorithms), as well as problems whose solutions could have a transformative impact on cyber resiliency but lie outside of the cyber realm (e.g., the geopolitics driving ransomware attacks).

Cyber hard problems are frequently caused or sustained by human or societal factors and misaligned incentives. These in turn are exacerbated by the continuing tremendous growth in the production and use of cyber technologies and their resulting near ubiquity in societally important systems and institutions.

Another contributor to hard problems is the difficulty of measuring cyber resilience or how a particular capability or solution improves it. The resulting failure to establish

• The vast increase in and pervasive use of cyber-physical systems—such as critical civil infrastructure (power, water, pipelines, etc.), medical systems, and operational technologies for manufacturing, plant operations, civil infrastructure, and national security. This growth of critical cyber-physical systems means that when failures occur, lack of resiliency results in extended outages of business-, safety-, and life-critical systems.
• Widespread adoption of social media and other cyber-mediated influences on opinions and actions.
• Rapid advances and growing adoption of machine learning and other artificial intelligence (AI) technologies.
• Increasing growth of autonomous and semi-autonomous systems.
• Growth in the number of well-resourced, motivated, and sophisticated state and non-state attackers.
• Adoption of AI and other advanced technologies by attackers.

Compounding these challenges are persistent market and policy shortcomings that have failed to incentivize responses that are adequate for meeting society’s needs. Thus, a broader analytical lens is needed that acknowledges both the technical and systemic nature of cyber vulnerabilities and accounts for the institutional barriers that impede robust resilience measures.

Cyber hard problems can be approached from the following two perspectives: (1) the key attributes that, if satisfactorily addressed, would enhance cybersecurity and resiliency of cyber systems and (2) the key considerations for developing cyber systems that, if satisfactory progress is made, would enhance the resilience of the resulting system. Another way to view these two perspectives is to see the first as hard problems from the perspective of “consumers” (adopters and users), and the second as hard problems from the perspective of “producers” (vendors and developers). The consumer list is what this report calls the new cyber hard problems list, while the producer list reflects the perspective of those responsible for building cyber and cyber-enabled systems. The two lists overlap significantly, but the different perspectives are nonetheless valuable in understanding the structure of the problems.

CYBERSECURITY AND CYBER RESILIENCE

“Cybersecurity” refers to the customary security dimensions of confidentiality, integrity, and availability of information in accordance with explicit security policies, along with several other key attributes. A secure system (1) does not interfere with

• CHP2—Secure development. How does one reliably engineer systems that are secure “out of the box” and safely evolve them in response to changing needs? The goal of building in security has long been asserted, but there are challenges in accomplishing this with respect to incentives and tools, techniques, and practices. Incentives are a challenge because it is difficult to measure if certain practices are yielding security improvements and determine if the cost or risk in adopting those practices is justified. Tools, techniques, and practices can be discounted because developers often insist on achieving significant improvements while not impairing productivity or system performance. However, there is evidence that practices are emerging that have negative cost, in that they increase developer productivity and create means to support rapid system evolution with continuous assurance.
• CHP3—Secure composition. What are the technical principles for securely integrating larger-scale systems from diverse components and services? The goal of secure composition is to enable reliance on separately made security judgments regarding individual system elements—both components and services—to support efficient judgments regarding the composite system. This is an issue almost universally faced due to the success of software libraries, frameworks, and reuse generally. For Web applications, for example, component libraries include millions of open-source and vendor components from which developers can draw.
• CHP4—Supply chain. How can one securely develop and manage large software and hardware systems engineering projects when there are diverse sources of components and services? A significant challenge for system designers, implementers, and sustainers is how to confidently assemble such systems when supply chains are complex and, very often, opaque due to trade secrecy and other considerations. There are technical principles, such as architecting for least privilege (which includes zero trust), and there are also policy principles, such as offering some degree of “translucency” regarding vendor components and services.
• CHP5—Policy establishing appropriate economic incentives. How can liability and accountability be allocated in a way that both encourages higher levels of security and also promotes rapid innovation? It is evident that considerations of cost and time-to-market often dominate security-focused engineering practices. Indeed, the difficulty of measuring levels of security can remove incentives to enhance security, since there may be no easy way to measure and thus reward the outcome. Process compliance and other surrogates are helpful but not sufficient in the face of modern attackers. A combination of technical

• and policy interventions could create incentives to address this “measurement conundrum” and enable producers to be rewarded for enhanced levels of security.
• CHP6—Human–system interactions. How can systems be designed in ways that reduce the extent to which attackers are able exploit human behavior to gain access to systems? Phishing and social engineering attacks are currently the dominant means of adversary access to all categories of systems, from mobile devices to governmental systems. Social science has shown that there is not an immutable trade-off of security and usability, and that with good engineering informed by social science, systems can be both secure and usable by operators and end users. How can systems be designed to support effective interactions with humans to support security-related activities ranging from authentication to operational attack response?
• CHP7—Information provenance, social media, and disinformation. How can social media support free speech and exchange of ideas while also protecting the safety and privacy of users and keeping them alert to deep fakes and false information. There is an ongoing war of attrition between developers of deep fake media, including fraudulent images, videos, and text, and those attempting to detect and flag such fakes—with modern AI technology having strong roles on both sides. Nation states may launch campaigns that exploit access to online user profiles to accomplish precision targeting. There are also emerging technical approaches to watermark (or otherwise attribute) media creations to facilitate tracking of information provenance.
• CHP8—Cyber-physical systems and operational technology. How can one better secure the Internet of Things and operational technology devices that are the central nervous system for manufacturing, civil infrastructure, and transportation, as well as for the many systems that operate in homes and offices? Many operational technology devices were designed on the assumption, now mostly false, that they would not be connected to the Internet and thus not exposed to cyber threats. Software and firmware updates are also challenging for many of these systems. Making matters worse is the complexity and modeling difficulty and uptime requirements due to their role in managing real-time operations of physical systems.
• CHP9—AI as an emerging capability. Challenges include the use of AI in offensive cyberoperations, and more broadly, the use of AI in—or as—mainstream software. Indeed, many of the challenges associated with the fact that AI models are increasingly being incorporated into software systems are implicitly addressed in other problem statements. There are, however, challenges

• that are unique to modern AI systems. What are the best techniques to model and analyze modern AI systems to understand the kinds of security-related weaknesses and vulnerabilities that are present? The opaque manner in which models learn from training data and subsequently transform input data into behaviors creates huge challenges for modeling and analysis, not just for functional behaviors but also for aspects ranging from supply chain (e.g., possible effects of poisoned training data) to runtime (e.g., evading guardrails) security. How can AI models be restructured, augmented, or encapsulated to enhance auditability? What drivers will more fully bring secure software design principles to AI systems when the models exhibit unexpected non-smooth and/or non-deterministic behavior? How can run-time controls of generative models be improved when included in scoped applications with specific safety and security requirements? What will be the new security risks associated with offloading more traditionally human-centric tasks to AI-powered autonomous systems? Additionally, AI capabilities are starting to have a significant impact on cyber operations generally, creating new opportunities (and challenges) for system developers, system operators and defenders, and for red teams.
• CHP10—Operational security. How can the resilience of the operational systems—the central nervous systems for larger private and governmental organizations—be enhanced in an environment of active threats and high consequence? All aspects of operational security—prevention, detection, response, and recovery—pose challenges to technical design, test and evaluation, operational security practices, and data management—as well as an understanding of the threat environment, legal requirements, and business considerations. Progress would enable improved organizational capability and productivity through automation, despite the presence of sophisticated threats.

CHALLENGES FROM A “PRODUCER” PERSPECTIVE

As discussed earlier, the committee also considered the challenges preventing the producers of cyber systems from solving the problems that consumers see. These include the technical, policy, and operational principles and procedures that the producer has to consider when building resilient cyber systems.

Key challenges for the producer include the following:

• Functional challenges, which deal with the design of secure interoperable products and infrastructure;
• Operational challenges, which concern the secure operation of an “at scale” infrastructure, including responding to attacks in a resilient manner;
• New technology challenges, which are caused by new and emerging computing paradigms and approaches, and their realization and application; for example, the inclusion of AI models in systems and the use of AI models in cyber operations; and
• Policy challenges, which are missing or misaligned policies that result in unexpected consequences to operations; for example, hiding or requiring isolation of previously relied-on data due to privacy concerns.

* * *

Together, the cyber hard problems list and the list of producer challenges provide a useful view of the range of issues associated with cyber resiliency. All stakeholders in the cyber ecosystem—including research funding agencies, computer technology companies, policy makers, and cybersecurity and other computing researchers—can contribute to making progress by reflecting on the driving factors and attacking the hard problems while considering the producer perspective.