2
Key Considerations for Cyber Resiliency

Difficulties in maintaining a resilient cyber ecosystem are hardly ever caused by single-dimensional issues. There are fundamental considerations that existed at the inception of computing technologies and persisted as they grew into the cyber ecosystem of today. These considerations are impossible to capture accurately because they cannot be bound; yet, it is important to understand them in order to have a comprehensive view of why cyber hard problems are persistently hard. In this section, the committee analyzes the overarching considerations as well as two key standalone drivers—engineering of resilient cyber systems and system complexity.

OVERARCHING CONSIDERATIONS

There are overarching considerations that affect each and every cyber hard problem described in this report. Many of their adverse effects were evident and relevant in 1995 and 2005, but new considerations have emerged or have been exacerbated due to scale, globalization, policy, and new technology. With increased usage, many technology problems are affected by human, cultural, political, or economic problems. These important influences were, perhaps, less apparent in earlier lists but are undeniable.

These considerations include the following:

• Computing technology is affected by global-scale economics. Scale affects personal data whose shared—and often uncontrolled—use is vast, and whose

• impacts on privacy are increasing, sometimes in unexpected ways. Financial incentives for collecting personal information and broadly sharing, selling, and repurposing the information are large and growing.
• The vastly increased complexity and interconnection of cyber systems makes it very difficult (and expensive) to implement resilient designs to prevent cyber failures. The complexity is not only due to the increased size and broad deployment of cyber systems but also to the entanglement and integration of disparate stakeholders, data providers, and technology providers within the scope of a single solution ecosystem. Modern services and apps rely on global multi-party ecosystems with interacting policies (e.g., privacy policies) that operate over many jurisdictions. This scaling and interconnection amplify the need to develop architectures that are resilient to compromises and failures in individual system elements.
• Security metrics sufficient to predict or verify the security properties of a cyber system are non-existent. In the absence of metrics based on measurable external parameters of systems, risk assessment has often fallen to the level of penetration testing because establishing the correct context may be difficult. However, there are far too few skillful “pen testers,” and the quality of a test too often seems a measure of the skill and luck of the tester. Systems are so large that complete coverage can be too time consuming (and therefore expensive) to be practical. Additionally, many security attributes are contextual—so not amenable to traditional testing—but instead require analysis and inspection, including of design models.
• The opacity and diversity of components in a single cyber system makes the challenges in risk assessment even more difficult. Dependencies among subsystems in a large system may create never-been-tested timing dependencies that only become evident when a disaster forces the entire collection of systems to be restarted for the first time in its life.
• The increasingly complex cyber-physical systems (CPS) and their central role underpinning critical functions mean that such systems are harder to defend, and compromises can cause large, immediate real-world damage. Integration of sensors and actuators broaden the important surface area that must be assured for CPS, such as critical infrastructure, as well for other CPS that can cause real-world damage. As an example, sensors often have their values checked only periodically for exceptional conditions. Local attacks can be timed to cause irreversible damage before the next time a value is checked.
• Complicated human–systems interaction and societal entanglement can confuse users, even in basic interactions, but follow-on consequences can be subtle

• and important. A significant challenge for designers is that users—in a hurry to accomplish specific tasks or no longer vigilant against rare anomalies—cannot always be counted on to attend to security concerns.
• The emergence of artificial intelligence (AI) (and statistical models) can increase capability; however, due to the inherent weaknesses and consequent vulnerabilities of neuro-network models, it is extremely difficult (in some cases, impossible) to validate decisions made by AI-enabled systems.
• The emergence of new concentrated targets such as clouds, popular software systems, and mobile devices, whose failure could have extremely broad adverse effects and whose complexity and opacity defy principled risk evaluation and principled risk management.

CONSIDERATIONS FOR ENGINEERING RESILIENT CYBER SYSTEMS

There are fundamentally only two operative methodologies to assure resilience of a cyber system. However, both require a complete understanding of the behavior of the entire engineered cyber system.

1. A guarantee that a delivered system will operate in accordance with a well-tailored specification of security-relevant quality behavior, so that an outside observer can be assured that the operations for which the system is employed, if they meet the specification, are safe. Generally, cyber systems are not warranted to operate in accordance with an enforceable behavior; in fact, cyber systems usually come equipped with extensive liability disclaimers protecting the supplier from claims about any but the most obvious, egregious failures and even then, providing incomplete remedies.
2. The development of an accurate risk assessment of every component of a cyber system as well as their interactions. This requires a comprehensive understanding of the cyber system and the components and services that it encompasses, the environment in which it operates, its users, as well as the principles according to which they are composed into the overall system.

Complete understanding of cyber systems is complicated by the fact that modern cyber systems are among the most complex artifacts created by humans, and current cyber systems are literally millions of times more complex than their predecessors 20 years ago, and too complex to model completely. Engineered systems depend, for predictability, on a profound understanding and specification of component subsystems, extensive

• Isolation and partitioning can offer protection for a relatively small system with well understood and carefully designed components but does not provide complete protection for complex systems with only partially understood or cursorily verified operations. Strict adherence to security-focused architecture designs can avert problems, but the required rigor is seldom employed.
• Systems are often built in an ad hoc way from many existing and widely used subsystems, which are themselves poorly understood. The most visible example is the incorporation of a vast library of open-source components. The web-focused open-source NPM (node package manager) library, for example, has more than 3 million packages available to web developers, along with various package management capabilities. Another common example is that, in the interest of economy, systems are frequently assembled using legacy subsystems that were not even built with then existing security standards.
• The way software contributions enter open-source projects varies widely. Attempts using social engineering to add flawed, exploitable code to a widely used open-source code brought additional attention to the difficulties of assurance in open-source projects.⁴
• Even well-designed systems depend entirely on operational configuration and policy management to operate properly; these systems and procedures are themselves complex and often error prone, and especially difficult to debug.
• Finally, modern cyber systems are heterogeneous, and a complete analysis relies on understanding the design and function of hardware, software, sensors and actuators, communications infrastructure, a bit of mathematics, and any relied-on external services.

Process assurance, including compliance regimes that include third-party verification of identified properties, can help a little but is usually an inadequate substitute for well-informed, engineered resilience based on direct modeling, analysis, and evidence. Process assurance is also often influenced by powerful stakeholders who can help shape rulemaking, placing small providers at a disadvantage in security perception and standards-based acquisition.

As a result, well-designed complex cyber systems often employ active remediation like rapid patching and reliable verifiable recovery as a resilient design crutch to compensate for future discovered flaws.

CONSIDERATIONS FOR COMPLEXITY

The previous section analyzed the resilience of small- or medium-scale cyber systems designed by a single supplier. However, the complexity of modern cyber systems, including having been integrated from multiple producers, introduces even greater challenges, including the following:

1. Verification of the function and qualities of acquired subsystem components, including open source.
2. The global hardware supply chain, which includes widely sourced complex components that can include deliberately inserted vulnerabilities.
3. The technical, economic, political, and cultural dynamics that make it difficult, if not impossible, for individual organizations to secure themselves effectively when they are dependent on diverse, complex integrated systems.
4. Relocated risk associated with deploying systems in one of a few large cloud providers whose infrastructure, operations, and even legal jurisdiction is undisclosed. Although most cloud providers offer safer infrastructure than less well-trained and well-practiced users provide for themselves—and they have strong business incentives to deliver high levels of security—they can still be single points of failure that become focused targets of malign nation states and rogue law enforcement and are well shielded from consequential liability by contract and law.
5. The sheer effect of scale that provides capability built on vast compositions of and interdependencies among complex systems.
6. The lack of clear metrics, monitoring, and operational procedures to accurately reflect residual risk.
7. Resilience and recovery (returning to a known-good state or, more precisely, to a well-understood state) from catastrophes is greatly complicated by the complexity and scale of modern cyber systems. This includes data and system configuration as well as hardware and software components. This is partly a technical problem but also part of the operational challenge discussed above.
8. Automated audit and forensics to find sources and effects of compromise are not widely available or widely used.
9. Development of standards for infrastructure that enable safety and assessment is not prescriptive enough to determine liability for a failure.
10. Data provenance integrity is seldom well implemented.
11. Protection of critical data (like PII) is not reliable against attacks from skillful threats, foreign and domestic, outsiders and insiders. Consider the attacks

11. against the U.S. Office of Personnel Management^5,6 and on unpatched vulnerable infrastructure particularly intended for legal interception of telephone calls at multiple telephone carriers internationally (“Salt Typhoon” or “RedMike”).⁷

Complexity is increasingly a factor in limiting the extent to which it is possible to understand a cyber system or CPS. However, proprietary barriers also prevent examination of code, design, hardware, and operations.

Many products have been admirably assessed by normal “market” incentives and mechanism, not cyber. Cyber is not unique in experiencing market failures that prevent reasonable risk assessment and effective regulatory oversight. Indeed, there is an extensive study of this in the used-car market under the “lemon law doctrine” of economics.⁸ Information asymmetry in the cyber-market means users and regulators cannot effectively identify, reward, or punish better or worse products or services. The result is that “good” cyber products may only sell for the same price as “bad” products because the usual market price discrimination incentives are ineffective. Correspondingly, information asymmetry in the cybersecurity market leaves little competitive pressure for security, and good security products sell for the same price as bad security products.

Absent effective liability protection, which is usually precluded by license disclaimers, information asymmetry effectively bars a principled determination of end-customer risks. Investment in cybersecurity by technology and service providers suffers when security benefits, which are obscured and difficult to measure, are scored against new revenue that can be obtained by equivalent investment in new capabilities. As a result, end-customers have no effective way to value better security, and providers are both unable to compete, pre-sale, on the basis of the security they offer, and they are at risk, post-sale, from unquantifiable losses because of liability protections. This situation is further complicated by the presence of adaptive adversaries whose own research investments in new attacks can serve to devalue past investments in defense.

Attacks on most engineered systems in the past were mounted on targets that were specific, well identified, and were motivated by comprehensible, predictable risk–reward assessments by attackers. They also required proximity, physical access, and specific tooling, so the risk of detection and punishment was significant. Many such attacks