Principles and Practices for a Federal Statistical Agency: Eighth Edition (2025)
Chapter: 4 Practices
4
Practices
Highlights
- Ten practices support the achievement of principles guiding federal statistical agencies:
- A Clearly Defined and Well-Accepted Mission
- Necessary Authority and Procedures to Protect Independence
- Commitment to Quality and Professional Standards of Practice
- Professional Advancement of Staff
- An Active Research Program
- Strong Internal and External Evaluation Processes for an Agency’s Statistical Programs
- Coordination and Collaboration with Other Agencies
- Respect for Data Subjects and Data Holders and Protection of Their Data
- Dissemination of Statistical Products That Meet Users’ Needs
- Openness About Sources and Limitations of the Data Provided
- These practices require policy support, funding, and staff resources to implement.
PRACTICE 1: A CLEARLY DEFINED AND WELL-ACCEPTED MISSION
A statistical agency’s mission should serve as a foundation not only for the work it does but also for how it does its work. Some agency missions are clearly spelled out in legislation; other agencies have only general authority granted them by legislation. Sometimes specific requirements are set by legislation or regulation (see federal-wide requirements for agency strategic plans and evaluation plans, such as in Title I of the Evidence Act, in Appendix A; see agency descriptions in Appendix B). A statistical agency’s mission includes its responsibility to:
- Produce and disseminate relevant and timely statistical information;
- Conduct credible and accurate statistical activities;
- Conduct objective statistical activities; and
- Protect the trust of information providers by ensuring the confidentiality and exclusive statistical use of their responses.1
These responsibilities should be so ingrained into agency staff during their training and through the procedures and practices they follow that they become part of the culture of the agency. To be effective, a statistical agency also should
- Ensure the quality of all aspects of its statistical programs, including measurement methods, data collection and processing, and appropriate methods of data analysis;
- Evaluate, implement, and document new methods and processes that better serve users’ needs (see Practice 5);
- Curate its data to ensure their availability for future use, as well as documenting the methods used and the quality of the estimates (see Practice 9); and
- Train its staff in a culture of responsible statistical practice.
Because nonstatistical activities threaten public trust in the agency, a statistical agency’s mission must focus on information that is to be used for statistical purposes. A statistical agency should defend its mission and resist external attempts to extend its work beyond statistical purposes (see Practice 2). If a statistical agency is charged with collecting information for nonstatistical purposes (e.g., collecting data, not only for statistical purposes, but also for possible use in administrative actions affecting an
___________________
1 See 44 USC § 3563(a)(1); originally issued as Statistical Policy Directive No. 1 (see Appendix A).
individual), the agency should carefully segregate the statistical activities from the nonstatistical ones (e.g., perhaps locating the latter within a clearly demarcated office). If the senior leadership of the agency conclude that it is not possible to develop a satisfactory arrangement responsive to the agency’s statistical mission, they should request that the activity be assigned elsewhere. Departments with federal statistical agencies have the responsibility to support and facilitate those statistical agencies in carrying out their mission and should not impose nonstatistical activities on them.2
Title III of the Foundations for Evidence-Based Policymaking Act of 2018 (2019; Evidence Act), also known as CIPSEA 2018, effectively expanded the mission of the federal statistical agencies with new authorities, roles, and responsibilities for evidence-based decision making. Statistical agencies should expand their administrative and alternative data holdings to develop evidence, as well as facilitate and expand secure, privacy-protected data access for evidence-building purposes. Statistical agencies should strategically implement these new authorities and responsibilities in order to maximize their mission impact (National Academies of Sciences, Engineering, and Medicine [NASEM], 2022a). Also, under the Evidence Act, statistical agency heads have the responsibility of serving as their cabinet department’s chief Statistical Official3 to lead statistical policy and activities, including setting standards for data quality and confidentiality. The Statistical Official should work closely with other senior officials, such as the Chief Data Officer and Chief Evaluation Officer, and other bureaus to advance the development and use of scientifically rigorous evidence, as well as promulgating good statistical principles and practices throughout the department (see Practice 7 and Appendix A).
A statistical agency should publicly communicate its mission and disseminate its statistical information and associated documentation on its website and other appropriate venues. The website should also provide information about enabling legislation, the scope of the agency’s statistical programs, confidentiality provisions, data quality guidelines, and data access procedures. Consequently, agencies should carefully design their websites to maximize their utility to their users, stakeholders, and the public.
A statistical agency should periodically review its mission. As part of strategic planning to carry out its mission within its budget, it should review priorities among different programs, the infrastructure (e.g., computing capabilities, staff with appropriate expertise) needed to support them, and the relative urgency of needed improvements, say, in timeliness versus
___________________
2 See 44 USC § 3563(b).
3 In the case of the Departments of Agriculture, Commerce, and Health and Human Services, which each host more than one recognized statistical agency or unit, the Statistical Official role is determined by the Chief Financial Officer (CFO) Act agency.
accuracy. Statistical agencies should regularly evaluate their programs to determine whether they are fulfilling the agency’s mission (see Practice 6), and an agency may need to eliminate or cut back an existing program in favor of a new initiative to better meet its mission (see Practices 5 and 9).
PRACTICE 2: NECESSARY AUTHORITY AND PROCEDURES TO PROTECT INDEPENDENCE
To maintain its credibility and reputation for providing objective, relevant, and accurate information, a federal statistical agency must have authority to maintain its independence from political and other undue external influences. Within an agency’s ecosystem—which includes its own department, the Office of Management and Budget (OMB), and Congress—there are often important safeguards for its independence. In some cases, these are enshrined in law, such as the requirement that data collected for exclusively statistical purposes may not be used for law enforcement.4 Other safeguards exist as longstanding governmentwide directives promulgated by the Office of the Chief Statistician in OMB that, for example, specify strict procedures for the release of statistical information that moves financial markets.5 Others may exist as departmental policies or agency policies, widely accepted norms, or longstanding practices.
Some statistical agencies have more safeguards for their independence built into their originating statutes than others do,6 while others rely on a history of having certain authorities without formal acknowledgement by their department. The proper functioning of individual agencies and the entire federal statistical system requires that there be strong and uniform recognition that these agencies have the authority to do the following:
- Make decisions over the scope, content, and frequency of data compiled, analyzed, and disseminated within the agency’s authorizing statutes based on sound scientific and professional considerations;
- Select and promote professional, technical, and operational staff based on their professional skills and knowledge (see Practice 4);
- Release statistical information, including accompanying press releases and documentation, without prior clearance regarding the statistical content of the release;7
___________________
4 See Confidential Information Protection and Statistical Efficiency Act of 2018 (2018), Appendix A.
5 See OMB Statistical Policy Directive No. 3 (OMB, 1985), Appendix A.
6 For example, the statute creating the Energy Information Administration specifically gives the Administrator the right to release statistical information without review by the Department of Energy.
7 See Statistical Policy Directives No. 3 (OMB, 1985) and 4 (OMB, 2008) in Appendix A.
- Be able to make pledges to data subjects and other data holders that their data will be kept confidential and used only for statistical purposes (see Practice 8); and
- Be able to meet with members of Congress, congressional staff, and the public to discuss the agency’s statistics, resources, and staffing levels.
In order to provide objective statistical information, a statistical agency must have highly qualified staff (see Practice 3), who can make decisions on the scope, content, and frequency of data compiled, analyzed, and disseminated without political or other undue external influence. Their decisions should be based solely on scientific and professional considerations. These decisions should be well informed by consultations with users and stakeholders, including policy officials in their department, on their need for information (see Practices 5 and 9), and they must also meet statutory requirements for content and OMB clearance of information collections.
The selection of qualified professional staff, including senior executive career staff, should be determined by the statistical agency. While departments may need to approve some appointments, they should allow great discretion to the statistical agency in selecting staff with appropriate expertise. Agency staff who report directly to the agency head should have formal education and deep experience in the substantive, methodological, operational, and management issues facing the agency, as appropriate for their positions. For the head of a statistical agency, professional qualifications are of the utmost importance, whether the profession is that of statistician or is in a relevant subject-matter field (National Research Council [NRC], 1997a). Relevant professional associations can provide valuable input about suitable candidates.
Statistical agencies must protect the confidentiality of the data they acquire throughout the lifecycle of those data and their use.8 Thus, statistical agencies must be able to exercise appropriate control over their data and the information technology (IT) systems on which they reside to securely maintain the integrity and confidentiality of individual records, ensure that the data can only be used for statistical purposes, and reliably support timely and accurate production of key statistics. A statistical agency must demonstrate the integrity, confidentiality, and impartiality of the data collected and sustained under its authority to maintain the trust of its data subjects, data holders, and data users (see Practices 8 and 9). Such trust is fostered when a statistical agency has control over its IT resources and there is no opportunity or perception that policy, program, or regulatory agencies could gain access to records of individual respondents. When departments
___________________
8 See CIPSEA guidance, Appendix A.
seek to centralize IT functions, they must support statistical agencies’ ability to control access to and use of their confidential data to ensure that the data are kept confidential and used only for statistical purposes.9
Although statistical agencies must be able to make pledges of confidentiality, it is not required that they do so for all of their collections. Statistical agencies may collect aggregated data from state and local governments that are already publicly available, and it would not serve the public good for the agency to then keep such data confidential.10 Because it is expected that statistical agencies will collect data solely for statistical purposes with pledges of confidentiality, they must be very clear when any data they are collecting will have nonstatistical uses.11
The authority of a statistical agency to release statistical information (including press releases) without prior clearance for the statistical content by department policy officials is essential, so that there is no opportunity for or perception of political manipulation of any of the reports.12 Statistical agencies are required to adhere to predetermined schedules for the public release of key economic indicators and to take steps to ensure that no person outside the agency has prior access except under carefully specified conditions.13 Agencies are also required to develop and publish schedules for the release of other important social and economic indicators and to announce and explain any changes in schedules as far in advance as possible.14
Statistical agencies are encouraged to use press releases to expand the dissemination of data to the public. However, such press releases must “be produced and issued by the statistical agency and must provide a policy-neutral description of the data.”15 Any policy pronouncements must be issued separately by executive branch policy officials and not by the statistical agency, and “policy officials of the issuing department may review
___________________
9 See 44 USC § 3563(b).
10 The Census Bureau collects data from state and local governments in the Census of Governments without a pledge of confidentiality, but only uses the information for statistical purposes. The National Center for Education Statistics (NCES) collects some data on public schools that it makes publicly available and does not promise to keep the data confidential.
11 See Confidential Information Protection and Statistical Efficiency Act (CIPSEA) guidance (Confidential Information and Statistical Efficiency Act of 2018, 2019), Appendix A.
12 The Energy Information Administration had its independence authorized in this regard in Section 205 of the Department of Energy Organization Act of 1977; 42 USC § 7135(d): “The Administrator [of EIA] shall not be required to obtain the approval of any other officer or employee of the Department in connection with the collection or analysis of any information; nor shall the Administrator be required, prior to publication, to obtain the approval of any other officer or employee of the United States with respect to the substance of any statistical or forecasting technical reports which he has prepared in accordance with law.”
13 See Statistical Policy Directive No. 3 (OMB, 1985) in Appendix A.
14 See Statistical Policy Directive No. 4 (OMB, 2008) in Appendix A.
15 See Statistical Policy Directive No. 4 (OMB, 2008) in Appendix A.
the draft statistical press release [solely] to ensure that it does not include policy pronouncements.”16
Statistical agencies should also have dissemination policies that foster regular, frequent release of major findings from the agency’s programs to the public through the traditional media, the Internet, and other means. They should also provide access to the underlying data, using appropriate safeguards to protect confidentiality (see Practice 9) to permit their results to be replicated. In these ways, an agency can guard against the perception of political and other undue external influence that might bias its operations.
The head of the statistical agency or unit should be able to meet with congressional staff and members to explain the agency’s statistics and programs. Although department representatives may also attend these meetings, the department should fully support the statistical agency in this regard. The head of the statistical agency or unit should also be able to prepare a budget request specific to their agency and meet with OMB during the annual budget development process.17 Similarly, it is essential that statistical agency leadership and staff be able to interact directly with their users and stakeholders. While the department may benefit from hearing the needs and concerns of these groups and individuals, the statistical agency should have the autonomy to arrange these meetings.
Finally, statistical agencies should be vigilant to threats to their independence, but they should also seek to educate officials in their ecosystem proactively about the appropriate roles and responsibilities of a statistical agency. Statistical agencies, not their parent agencies, should be given clear credit or recognition for their data/reports. Senior leaders of an agency should cite relevant laws, regulations, and these widely accepted principles and practices for federal statistical agencies as precedent and as necessary for the mission of the agency. Undermining the authorities described in this practice undermines the mission of the agency itself, so if serious threats are made to a statistical agency’s independence and references to the relevant laws, regulations, principles, and practices are not heeded, senior leaders should turn to the secretary of the department, the Chief Statistician of the U.S. at OMB, Congressional oversight committees, stakeholders, professional associations, and users to come to the agency’s defense. Such outreach should not be undertaken lightly but should not be avoided if the fundamental mission of the agency is at stake.
___________________
16 See Statistical Policy Directive No. 4 (OMB, 2008) in Appendix A.
17 The Office of the Chief Statistician of the U.S. is responsible for assessing federal statistical agency budgets under the Paperwork Reduction Act (Paperwork Reduction Act, 1995). Language at § 1321.4(g)(2) in the final Trust Regulation (OMB, 2024b) requires federal statistical agencies and units to be given the opportunity to participate in program and staffing budget preparation and their engagement during the annual budget review process.
PRACTICE 3: COMMITMENT TO QUALITY AND PROFESSIONAL STANDARDS OF PRACTICE
A federal statistical agency’s commitment to quality and professional standards is the foundation of its credibility; such a commitment provides a strong defense against baseless attacks on the quality of statistical products and other forms of intentional disinformation. Such commitment should be deeply embedded in the agency’s culture and reflected through:
- Adhering to and implementing OMB standards and guidelines;
- Publishing and implementing agency quality standards;
- Maintaining quality assurance programs and innovating to improve data quality and the processes of compiling, editing, documenting, analyzing, and disseminating data;
- Evaluating the quality of the agency’s data (see Practice 6);
- Communicating clearly what is known about the validity and accuracy of the agency’s data and the resulting measures of quality (both uncertainty and bias; see Practice 10);
- Documenting and updating concepts, definitions, and data collection methods and possible sources of error in data releases to the public (see Practice 10); and
- Developing and maintaining relationships with appropriate professional organizations in statistics and relevant subject-matter areas (see Practice 5).
An effective statistical agency devotes resources to developing, implementing, and updating standards for data quality and professional practice. Although a long-standing culture of data quality contributes to professional practice, an agency should document standards through an explicit process. Having explicit standards that are regularly reviewed and updated facilitates the training of new in-house staff and contractors’ staffs. The reviews should include a careful consideration of quality frameworks used by other national statistical organizations as well as international organizations (see Appendix C).
To ensure the quality of its data collection programs and data releases, an effective statistical agency combines formal quality assurance programs with mechanisms and processes for obtaining both inside and outside reviews (see Practice 6). Formal quality assurance programs include well-developed methods for detecting outliers and other errors in raw data, methods for identifying errors from editing and other data processing steps, and, increasingly, reviews of processes followed by holders of administrative and private-sector input data. Reviews help ensure data quality by addressing various aspects of an agency’s operations, including the soundness of the data collection
and estimation methods and the completeness of the documentation of the methods used, metadata, and the error properties of the data. For individual reports, formal processes are needed that incorporate review by agency technical experts and, as appropriate, by technical experts in other agencies and outside the government.18
An effective statistical agency keeps up to date on developments that may be relevant to its program—for example, modern methods for combining data from more than one source, use of artificial intelligence methods, estimating error in the resulting statistics, and new technologies for data collection, processing, and dissemination.
Statistical agencies should be alert to social and economic changes that may call for innovations in the concepts or methods they use (NASEM, 2017e, 2019a, 2020b). The need for change often conflicts with the need for comparability with past data series. Agencies have the responsibility to manage this conflict by initiating more relevant series or revising existing series to improve quality, while providing information to compare old and new series.
The best resource for ensuring high-quality data is a strong professional staff, which includes experts in the subject-matter fields covered by the agency’s programs, experts in statistical methods and techniques, and experts in data collection, computing and information science, and other operations (see Practice 4). A major function of an agency’s leadership is to strike a balance among these staff and to promote collaboration, with each group of experts contributing to the work of the others. An effective statistical agency encourages its professional staff’s membership and participation in relevant professional associations to refresh their skills and knowledge and to develop networks of experts from other statistical agencies, academia, and the private sector (see Practice 4).
An effective statistical agency also has policies and practices to instill the highest possible commitment to professional ethics among its staff. Because knowledge of codes of ethics from professional associations can reinforce this commitment in the agency culture (Hogan & Steffey, 2014), an effective agency ensures that its staff members are aware of and have access to such statements of professional ethical practice as those of the American Association for Public Opinion Research (2021),19 the American Statistical Association (2018 and 2024),20 the American Economic Association (2018),21 and the International Statistical Institute (2023),22 as well as the agency’s own policies and practices regarding such matters as the
___________________
19 https://aapor.org/standards-and-ethics/
20 https://www.amstat.org/asa/files/pdfs/EthicalGuidelines.pdf and https://www.amstat.org/policy-and-advocacy/asa-board-statements
protection of confidentiality, respect for privacy, and standards for data quality. This reinforcement of professional and agency ethical cultures is recognized by the federal Data Ethics Tenets (General Services Administration, 2020). An effective agency endeavors in other ways as well to ensure that its staff are fully cognizant of the ethics that must guide their actions in order for the agency to maintain its credibility as a source of objective, reliable information for use by all (Hartman et al., 2014).
PRACTICE 4: PROFESSIONAL ADVANCEMENT OF STAFF
The long-term credibility of a statistical agency depends on the agency’s staff and the culture it builds and maintains for quality and professionalism. Thus, a statistical agency should recruit and support highly qualified and dedicated staff for all aspects of its operations, including subject-matter experts in fields relevant to its mission (e.g., demographers, economists), statistical methodologists who specialize in data collection and analysis, computer and data scientists, and other skilled staff such as budget analysts, procurement specialists, and human resource specialists. Statistical agency staff should be recruited and promoted based solely on their professional qualifications and performance, and these personnel decisions should be made solely by agency career staff without external interference (see Practice 2). Statistical agencies also should consider the pipeline of future professionals to ensure the long-term viability of statistical programs and products. This includes finding ways to facilitate the training and development of future generations of professionals needed to design and manage statistical systems of the future.
To manage its staff effectively, an agency should provide them with opportunities for work on challenging projects in addition to more routine, production-oriented assignments. An agency’s personnel policies, supported with sufficient resources, should enable staff to extend their technical capabilities through appropriate professional and developmental activities (see below). These activities enhance the knowledge and skills of the staff members and pay dividends to the agency, helping it to stay on top of new developments.
The personnel policies of an effective federal statistical agency should encourage the development and retention of a strong professional staff who are committed to the highest standards of quality work for their agency and in collaboration with other agencies. Key elements of such policies include the following:
- Providing staff with continuing technical education and training, appropriate to the needs of their positions. Technical education may come from in-house training programs and opportunities for
- external education and training at universities or through professional societies. Supervisory and leadership training from the U.S. Office of Personnel Management or other institutions should also be encouraged for managers and emerging leaders;
- Structuring position responsibilities to ensure that staff have the opportunity to participate, in ways appropriate to their experience and expertise, in research and development activities to improve the quality of data and cost-effectiveness of agency operations;
- Encouraging and recognizing professional activities, such as publishing in refereed journals and presenting at conferences. The latter should include technical work in progress, with appropriate disclaimers;
- Supporting participation in relevant statistical and other scientific associations and committees, including leadership positions, to promote interactions with researchers and methodologists in other organizations that can advance the state of the art. Such participation is also a mechanism for disseminating information about an agency’s programs and helps ensure a culture of scientific integrity within the agency;23
- Fostering interaction with other professionals inside and outside the agency through a variety of mechanisms, for example participation in technical advisory committee meetings, interaction with contract researchers and research consultants on substantive matters, interaction with visiting fellows and staff detailed from other agencies, developmental assignments with other relevant statistical, policy, or research organizations, and rotational assignments within the agency;
- Exploring opportunities to engage experts for short duration projects at federal statistical agencies to share and apply the latest statistical and data science techniques through existing authorities, such as through the Intergovernmental Personnel Act (Cui et al., 2024, forthcoming; Ho & O’Connell, 2024; Intergovernmental Personnel Act of 1970, 1970; Temporary Assignments Under the Intergovernmental Personnel Act, 2024) and Excepted Service (Excepted Service, 1978);
- Supporting participation in cross-agency collaboration efforts, such as the Federal Committee on Statistical Methodology and its subcommittees. Such participation not only benefits the professional staff of an agency, but also contributes to improving the work of the statistical system as a whole (see Practice 7);
___________________
23 See Office of Science and Technology Policy Memorandum on Scientific Integrity (Office of Science and Technology Policy, 2023b) in Appendix A.
- Rewarding accomplishment by appropriate recognition and by affording opportunities for further professional development. The prestige and credibility of a statistical agency is enhanced by the professional visibility of its staff, which may include establishing high-level nonmanagement positions for highly qualified technical experts; and
- Seeking opportunities to reinforce the commitment of its staff to ethical standards of practice.
Implementing these policies requires sufficient funding, time off, and institutional respect for professional education and development.
An effective statistical agency carefully considers the costs and benefits—both monetary and nonmonetary—of using contractor organizations, not only to collect data but also to supplement in-house staff in other areas, such as carrying out methodological research. Outsourcing can have benefits, such as providing expertise in areas in which the agency is unlikely to be able to attract highly qualified in-house staff (e.g., some information technology functions), enabling an agency to handle an increase in its workload that is expected to be temporary or that requires specialized skills, and allowing an agency to learn from best industry practices. However, over time excessive outsourcing can also have unintended costs, including a transformation of agency staff from being primarily technical experts in their fields to serving primarily as contract managers, with an associated loss of in-house knowledge. (See, in particular, recommendations 5.1 and 5.2 in NASEM, 2022a.)
An effective statistical agency maintains and develops a sufficient corps of in-house staff, including mathematical statisticians, survey researchers, subject-matter specialists, data and computational scientists, and information technology experts, who are qualified to analyze the agency’s data and to plan, design, carry out, and evaluate its core operations, so that the agency maintains the integrity of its data and its credibility in planning and fulfilling its mission. Agencies also need staff with specialized skills to create visualizations, metadata, and application programming interfaces for data dissemination (see Practice 9). At the same time, statistical agencies should maintain and develop staff with the expertise necessary for effective technical and administrative oversight of contractors. Given the increasing use of alternative data sources, agencies should not only encourage training in programming and software engineering to build up their staff’s skills in data science, but also encourage their subject-matter experts to become fully knowledgeable about the origin, content, and quality of various relevant data sources.
Having sufficient in-house staff with the required types of expertise is as critical as having adequate budget resources for enabling a statistical
agency to carry out its mission. Statistical agencies are constrained by federal personnel policies that can affect whom they are permitted to hire (e.g., U.S. citizens) and by federal pay scales. However, some statistical agencies have been needlessly constrained in the number of agency staff they can employ regardless of their budgetary resources, resulting in too few staff to adequately handle the work needed to maintain existing programs and oversee contractors (see American Statistical Association, 2024). As part of their fundamental responsibilities to support statistical agencies, departments housing statistical agencies should work with and support them in being able to hire a sufficient number of staff with the right expertise to carry out their missions.
PRACTICE 5: AN ACTIVE RESEARCH PROGRAM
Statistical agencies need active research programs that are closely tied to their mission of producing relevant and high-quality statistics. Research is not an “optional” or “extra” activity that can be deferred whenever resources are tight. It produces the innovation that refreshes relevance. The underfunding of statistical agencies’ research has threatened the data infrastructure that provides vital information needed by governments, businesses, organizations, and individuals.24
To maintain relevance for public and policy purposes, federal statistical agencies must identify emerging needs and look for ways to develop new information sources. To improve the quality and timeliness of their data products, they must keep abreast of methodological and technological advances and be prepared to implement new procedures in a timely manner (see Practice 3). They must also continually seek ways to make their operations more efficient (see Practice 6).
An effective statistical agency’s research program includes research on the substantive issues for which the agency’s data are compiled as well as methodological research to improve statistical methods and operational procedures. Questions related to the use of administrative records and alternative data sources to enhance or potentially replace some of the information currently obtained through surveys have been a focus of research for statistical agencies for decades, which is only growing in importance. These questions include how closely statistics from these administrative and alternative data sources correspond to existing measured concepts, what additional information they may offer, and the appropriate methodologies for evaluating quality and integrating data sources.
___________________
24 https://www.linkedin.com/pulse/federal-statistical-agencies-struggle-maintain-vitalrole-citro/?trackingId=hWmaUxpC4ao5VxtMmWioyg%3D%3D
Substantive Research and Analysis
A statistical agency should include staff with responsibility for conducting objective substantive analyses of the data that the agency compiles, such as analyses that assess trends over time or compare population groups. Substantive analyses provided by an agency should be kept relevant to policy by addressing topics of public interest and concern; however, such analyses should not espouse policy positions or be designed to reflect any particular policy agenda (Martin, 1981; Norwood, 1975; Triplett, 1991). The existence and output of an analytical staff can contribute not only to the knowledge base in the applicable subject areas, but also to the credibility, relevance, accuracy, timeliness, and cost-effectiveness of the agency’s data collection programs. Benefits that a strong subject-matter staff bring to a statistical agency include the following:
- Agency analysts understand the need for the data from a statistical program and how the data will be used, and they can communicate more effectively with data users (see Practice 9);
- Agency analysts typically have access to the complete microdata and so are better able than outside analysts to understand and describe the limitations of the data for analytic purposes and to identify errors or shortcomings in the data that can lead to subsequent improvements (see Practice 10); and
- Substantive research maintains the relevance of an agency’s data program, suggesting changes in priorities, concepts, and needs for new data or discontinuance of outmoded or little-used series.
An agency’s subject-matter analysts should be encouraged and have ample opportunity to build networks with analysts in other agencies, academia, the private sector, other countries, and relevant international organizations and to present their work at relevant conferences and in working papers and refereed journal articles (see Practice 4).
Research on Methodology and Operations
Statistical agencies should be innovative in the methods they use for data collection, processing, estimation, analysis, and dissemination, with the goals of improving data accuracy, timeliness, and operational efficiency and of reducing respondent burden. Careful evaluation of new methods is required to assess their benefits and costs in comparison with current methods and to determine effective implementation strategies, including the development of methods for bridging time series before and after a change in procedures.
Research on methodology and operations must be ongoing and geared to both current and future needs. Some current research topics are listed below.
- Developing methods for producing rapid statistics to respond to high-priority situations or emergencies, such as the COVID-19 pandemic;25
- Evaluating administrative records for use to replace or enhance existing surveys;
- Investigating the use of artificial intelligence and related methods to improve estimation or processing;
- Assessing uncertainty when combining data from a variety of data sources;
- Examining administrative records and other data sources as a means to provide provisional subnational estimates;
- Improving the accuracy of survey estimates in the presence of nonresponse;
- Using adaptive designs for maintaining and improving the quality and the cost-effectiveness of surveys;
- Understanding and minimizing mode effects on data quality; and
- Developing and evaluating new methods of confidentiality protection.26
Surveys will likely remain an important component of federal statistical agencies’ portfolios because (a) some information is best (or only) obtained by asking questions; and (b) surveys can collect information on many characteristics at the same time, thereby permitting rich multivariate analysis. But declining survey response rates are making it increasingly difficult to maintain high data quality while controlling data collection costs (NASEM, 2017a; NRC, 2013b). Many of the large federal surveys are designed to produce annual nationwide estimates and do not produce the rapid and granular estimates needed by some data users. It is thus essential to consider how administrative records and alternative data sources can bolster the completeness, quality, and utility of statistical estimates while containing costs and reducing respondent burden (OMB, 2016f).
___________________
25 For example, see National Center for Health Statistics (NCHS) guidance in data collection and methodology for COVID-19 data at https://www.cdc.gov/nchs/covid19/index.htm; the NCES School Pulse Panel data collection and methodology described at https://nces.ed.gov/surveys/spp/ and Statistical Agency Changes in Response to the COVID-19 Pandemic at https://www.statspolicy.gov/assets/docs/ICSP-COVID-19-Report_011521.pdf
26 The National Secure Data Service Demonstration Project at the National Center for Science and Engineering Statistics (NCSES) will be informed by several studies sponsored through the America’s DataHub Consortium, to evaluate privacy enhancing technologies and their application to statistical products. See Practice 9.
Expanding the Statistical Use of Administrative Records
Administrative records include records of federal, state, and local government agencies that are used to administer a government program. Examples include U.S. Social Security Administration records of payroll taxes collected from workers and benefits paid out to beneficiaries; state agency records provided by applicants for assistance programs and payments to applicants deemed eligible; and local government property tax records. Administrative records have been used for statistical purposes for many years to generate up-to-date population estimates by age, gender, race, and ethnicity. In turn, these estimates are used to adjust population survey weights for coverage errors and for many other purposes (NRC, 2004d, 2007a).
Some of the many examples of statistical agencies’ use of administrative data include the Census Bureau using tax records for the economic censuses for small and non-employer businesses,27 the NCHS National Vital Statistics System drawing upon birth and death records from the states,28 and the NCES’ National Postsecondary Student Aid Study, drawing upon federal and institutional administrative data to analyze student financial aid.29 Research is being conducted to assess whether tax records can replace income items in the American Community Survey (NASEM, 2019b). Administrative records are also frequently used with survey data to produce model-based estimates with improved accuracy for small geographic areas or population groups (NASEM, 2019b, 2023c; NRC, 2000a,b; Young, 2019; Young & Chen, 2022).
There are many other potential statistical uses for administrative records from program agencies, and expanding the use of these records could improve the cost-effectiveness and quality of some statistical programs. Potential uses include substituting administrative records for specific survey questions and adding richness to a combined dataset by appending administrative records variables to matched survey records (Commission on Evidence-Based Policymaking, 2017; NASEM, 2018a, 2019c, 2023b,c; NRC, 1997b, 2009a, 2012a). Administrative records from multiple federal agencies are also being used in the decennial census to verify vacant units and, when good information exists, to fill in data if an initial nonresponse follow-up visit is not successful in locating a respondent.30
___________________
27 Non-employer businesses include just the sole proprietor with no other employees.
28 See https://www.cdc.gov/nchs/nvss/index.htm.
29 See https://nces.ed.gov/surveys/npsas/index.asp.
30 See https://www2.census.gov/programs-surveys/decennial/2020/program-management/planning-docs/administrative-data-use-2020-census.pdf.
Evaluating and Using Alternative Data Sources
This data-rich age has a multitude of data sources beyond administrative records, including data gleaned or “scraped” from internet websites (e.g., price quotes, social media postings), data extracted from sensors (e.g., from traffic cameras), and data obtained from the private sector (e.g., credit card transactions, scanner data on retail purchases). Often, these sources generate large volumes of data that require computationally intensive techniques for extracting useful information for statistics (NASEM, 2017a,c, 2023c; NRC, 2008b). However, to make use of most nontraditional data sources, it is necessary for statistical agencies to first evaluate the accuracy and error properties of the data, and then to compare error magnitude and impact between alternative and traditional data collection methods.
In an era when data users expect timeliness and when budgets are constrained, researchers in statistical agencies should explore how nontraditional data sources can contribute to their programs (NASEM, 2017a,c, 2023b,c). Procedures could include (a) augmenting information obtained from traditional sources; (b) replacing information elements previously obtained from traditional sources; (c) providing preliminary estimates that are later benchmarked with traditional sources; and (d) analyzing information streams to identify needed changes (e.g., in types of jobs, education majors) in statistical classifications and survey questions (NASEM, 2017a,c, 2023b,c). A major challenge for statistical agencies has been the difficulty of identifying, locating, and accessing alternative data sources that could be useful for their programs. As the Evidence Act (Foundations for Evidence-Based Policymaking Act of 2018, 2019) is implemented, the data inventories and practices of the program agencies should continue to make these resources more transparent and make processes for obtaining these datasets for statistical purposes more streamlined (also see Practices 8 and 9).
In considering their strategies, statistical agencies should adopt broad quality frameworks that capture user needs, including aspects such as relevance, accuracy, timeliness, comparability (over time and with other data sources), transparency, accessibility, privacy, protection from outside manipulation, and interpretability. They should examine the tradeoffs between different quality aspects, such as trading precision for timeliness and granularity (see NASEM, 2017c,e, 2023c, 2024c; Appendix C). An agency’s own research staff can assist in examining these tradeoffs, and the Federal Committee on Statistical Methodology (Federal Committee on Statistical Methodology, 2020; Prell et al., 2019) also has been pursuing work in this area to assist agencies.
Value of an Active Research Program
Supporting federal agencies’ in-house research staffs is critical given the challenges and opportunities posed by the increasing availability of alternative data sources. Many current practices in statistical agencies were developed through research they conducted or obtained from other agencies. Federal statistical agencies, frequently in partnership with academic researchers, pioneered the use of statistical probability sampling, the national economic accounts, input-output models, and other analytic methods. The Census Bureau pioneered the use of computers for processing the census. Several statistical agencies use academic principles of cognitive psychology—a research strand dating back to the early 1980s (see NRC, 1984)—to improve the design of questionnaires, the clarity of data presentation, and the ease of use of electronic data collection and dissemination tools. History has repeatedly shown that research conducted within federal statistical agencies on subject areas, methods, and operations can lead to large productivity gains in statistical activities at relatively low cost (American Statistical Association, 2024; Citro, 2016; NRC, 2010b).
An effective statistical agency also actively partners with the academic community and private sector for methodological research. It seeks out academic and industry expertise for improving data collection, processing, and dissemination operations. For example, a statistical agency can learn techniques and best practices for improving software development processes from computer scientists (NRC, 2003a, 2004c). An effective agency also learns from and contributes to methodological research of statistical agencies in other countries and relevant international organizations (see Practice 7). Thus, it is important for agency staff to seek to publish their work in the leading peer-reviewed journals, and to post white papers and reports on agency websites, both of which enable broader dissemination as well as adding credibility to the changes the agency makes.
Preparing for the future requires that agencies periodically assess the scope of existing data series, alter data series as required, and innovate to improve their programs. Because of the decentralized nature of the federal statistical system, innovation often requires and benefits from cross-agency collaboration (see Practice 7) and a willingness to implement different kinds of data collection efforts to answer different needs, while being mindful of the need for historical trend data and comparability across different levels of geography. As described under Practice 7, a significant role of the Office of the Chief Statistician of the United States is to support cross-agency collaboration.
PRACTICE 6: STRONG INTERNAL AND EXTERNAL EVALUATION PROCESSES FOR AN AGENCY’S STATISTICAL PROGRAMS
Statistical agencies should have processes in place to support regular evaluations of their major statistical programs and their overall portfolio of programs. Reviews of major data collection programs and their components should consider how to produce relevant, accurate, and timely data in the most cost-effective manner possible. Reviews of an agency’s portfolio should consider ways to reduce duplication, fill gaps, and adjust priorities so that the overall portfolio is as relevant as possible to the information needs of policymakers and the public (NASEM, 2018c, 2020b; NRC, 2009b). Such evaluations should include internal reviews by staff and external reviews by independent groups.
Agencies should seek administrative and outside reviews not only of specific statistical programs but also of program priorities and quality practices across their entire portfolio. They should also consider ways to improve program cost-effectiveness by combining data from multiple sources, particularly because fewer people and organizations are responding to surveys than in the past. It is increasingly urgent to determine whether there are alternative data sources to surveys that offer similar or better quality. (See NASEM, 2017a,b,c, 2018b, 2022a, 2023c; and Practice 5).
Statistical agencies that fully follow practices related to an active research program (Practice 5), openness (Practice 10), dissemination of statistical data products (Practice 9), and commitment to quality and professional standards (Practice 3) will likely be in a good position to make continuous assessments of and improvements in the relevance, quality, and efficiency of their data collection systems. Yet even the best-functioning agencies will benefit from an explicit program of internal and independent external evaluations to formalize success criteria, evaluate performance, and obtain fresh perspectives.
Evaluating Quality, Relevance, Efficiency
Evaluation of data quality for any kind of data collection program begins with regular monitoring of quality indicators that are readily available to users. Agencies should use broad quality frameworks (see Practice 3 and Appendix C) and assess the costs and benefits of using alternative data sources (see Practice 5 and NASEM, 2017a,c). These evaluations should be undertaken periodically and the results made public (see Practice 10 and NRC, 2007a).
When it is disruptive to implement improvements on a continuing basis, a common practice is to bundle changes to implement several at the same time. For example, classifications such as the North American Industry
Classification System (NAICS) are updated every 5 years, and agencies may implement other changes at the same time as this. Agencies should ensure that the intervals between innovations do not become so long that data collection programs deteriorate in quality, relevance, and efficiency. Regular, well-designed program evaluations, with adequate budget support, are key to ensuring that data collection programs do not deteriorate. Having a set schedule for research and development efforts will enable data collection managers to ensure that the quality and usefulness of their data are maintained and help prevent locking in less optimal procedures.
As part of ongoing evaluation, the relevance of an agency’s data collection programs and products needs to be continually assessed. The question of relevance is whether the agency is “doing the right thing,” in contrast to whether the agency is “doing things right.” Relevance should be assessed not only for particular programs or closely related sets of programs, but also for an agency’s complete portfolio in order to make the best choices among program priorities given the available resources (see Practice 1).
Engaging and consulting with stakeholders—through such means as regular meetings, workshops, conferences, and other activities—is important to ensuring relevance (see Practice 9). Including other federal statistical colleagues in this communication, both as users and as collaborators, can be valuable (see Practice 7).
Finally, statistical agencies should review their statistical programs for efficiency and cost-effectiveness.31 Federal statistics as a public good represent a legitimate draw on public resources, and statistical agencies in turn are properly called on to analyze the costs of their programs on a continuing basis to ensure the best return possible on tax dollars. For this purpose, statistical agencies should develop complete, informative models for evaluating the costs of current procedures and of possible alternatives and follow best practice in the design of statistical production processes.32
Types of Reviews
Regular statistical program reviews should include a mixture of internal and external evaluation. Agency staff should set goals and timetables for internal evaluations that involve informed staff outside the program under
___________________
31 “Efficiency” is generally defined as an ability to avoid waste (of materials, energy, money, time) in producing a specified output. “Cost-effectiveness” connotes a broader, comparative look at inputs and outputs to assess the most advantageous combination. (“Cost-benefit” analysis attempts to add monetary values to outputs.) In the context of federal statistical programs, cost-effectiveness analysis would assess the costs of conducting a program for different combinations of desired characteristics, such as improved accuracy or timeliness and reduced burden on respondents.
32 See Generic Statistical Business Process Model in Appendix C.
review. Independent external evaluations should also be conducted on a regular basis. The frequency of these external evaluations should depend on the importance of the data, how quickly the phenomena being measured change, and how quickly respondent behavior and data collection technology may adversely affect a program change.
External reviews can take many forms. They may include recommendations from advisory committees that meet at regular intervals (typically, every 6 months). However, advisory committees should never be the sole source of outside review because the members of such committees rarely have the opportunity to become deeply familiar with agency programs. External reviews can also take the form of a special committee or panel established by a relevant professional association, such as the American Statistical Association, or by some other recognized group, such as the National Institute of Statistical Sciences or the Committee on National Statistics (also see NRC, 2009b).
Sunsetting Statistical Products or Programs
Although it can be difficult to stop producing something, all statistical agencies must be able to do so. While most reviews serve to improve ongoing programs, others may inform the decision to discontinue a particular product or program. The situations that call for sunsetting include an innovation or experiment that fails by some criterion, a budget reduction, a decline in relevance to users, a higher priority need elsewhere, or replacement by an alternative source (NASEM, 2022d). By informing such decisions, effective evaluations within agencies support continuous learning and program improvement in the federal statistical system and promote trust in the agencies.
An agency that never ends programs or products will eventually cease innovating and need to reduce the quality of all its programs. For example, in the face of a budget cut, discontinuing a program or product is preferable to across-the-board cuts in all programs, which would reduce the accuracy and usefulness of both the more relevant and less relevant data series (NASEM, 2022a). More generally, prudent use of taxpayer dollars requires that agencies be ready and able to sunset programs whenever resources could be more productively used elsewhere.
On considering a decision to terminate, a product or program communication is critical. The agency should be transparent to stakeholders, as far in advance as possible, about the nature of the change under consideration, the reasons for it, and alternative sources of information.
PRACTICE 7: COORDINATION AND COLLABORATION WITH OTHER AGENCIES
Statistical agencies should not be islands unto themselves. They need to engage and collaborate not only with their stakeholders but also with other statistical agencies and units in the federal government, in state governments, and internationally. The U.S. federal statistical system consists of many agencies in different departments, each with its own mission and subject-matter focus (see Appendix B). Yet these agencies have a common interest in serving the public need for credible, relevant, accurate, and timely information gathered as efficiently and fairly as possible. Moreover, needed information may often span the mission areas of more than one statistical agency: for example, both the Bureau of Labor Statistics (BLS) and NCES have programs that relate to education and employment outcomes. Consequently, statistical agencies should not and do not conduct their activities in isolation.
An effective statistical agency actively seeks opportunities to conduct research and carry out other activities in collaboration with other agencies to enhance the value of its own information and that of the system as a whole. Such collaboration is essential not only for smaller statistical agencies with limited staff and resources but, equally, for larger agencies so that they do not overlook useful innovations outside their own agency. When possible and appropriate, federal statistical agencies should collaborate not only with each other but also with policy, research, and program agencies in their departments, with state and local statistical agencies, and with foreign and international statistical agencies.
Such collaborations can serve many purposes, including standardization of concepts, measures, and classifications (see, e.g., NRC, 2004a,e; Appendix A); augmentation of available information for cross-national and subnational comparisons (NRC, 2000a,b); identification of useful new data sources and data products; and improvements in many aspects of statistical programs.
In their new roles as the chief Statistical Officials for their departments, heads of the recognized statistical agencies and units should proactively collaborate with their departments’ Chief Data Officers, Chief Information Officers, Chief Artificial Intelligence Officers, and Evaluation Officers in overseeing departmentwide data governance and use of data for evidence building. Departments should consult the Statistical Officials during decision making, as statistical agencies add value by contributing expertise in areas such as data standards, privacy protection, rigorous methods for developing credible data that are fit for purpose, and appropriate interpretation of evidence and statistics (NASEM, 2022a). In addition, statistical agency heads should help improve statistical practices not only within
their departments, but also across government by providing advice and mentoring, as appropriate, in particular to the Statistical Officials who are not heads of a recognized statistical agency or unit. This type of collaboration can help foster the understanding and integration of principles and practices for statistical activities that will be more widely adopted across the federal government as agencies continue to implement the Evidence Act (Foundations for Evidence-Based Policymaking Act of 2018, 2019; NASEM, 2022a), the Trust Regulation (OMB, 2024b), and the Information Quality Act of 2000 (Information Quality Act, 2000).
Coordinating Role of the Office of Management and Budget
The responsibility for coordinating statistical work in the federal government is specifically assigned to the U.S. Chief Statistician, who leads the Statistical and Science Policy (SSP)33 Office in the Office of Information and Regulatory Affairs in OMB (see Appendix A). The U.S. Chief Statistician chairs the Interagency Council on Statistical Policy (ICSP), which consists of the heads of the recognized statistical agencies and units, and other agency Statistical Officials, to coordinate federal statistical programs and activities across the federal government (see Appendix B).
A primary responsibility of the Office of the U.S. Chief Statistician (also known as the SSP of OMB) is to identify issues of common concern and create interagency committees for collaborative work, such as concepts of interest to more than one agency (e.g., classifications of sexual orientation, gender identity, and race/ethnicity), the development and periodic revision of standard classification systems (e.g., of industries, products, occupations, and metropolitan areas), and best practices for domains such as survey methods, statistical use of administrative records, and confidentiality protection. SSP may then take the recommendations of these committees and issue more formal guidance or directives for all agencies to follow. To facilitate this coordination, some experts have called for a systemwide strategic plan that could assist the Office of the U.S. Chief Statistician in communicating its priorities and assessing the capacity of the federal statistical system (American Statistical Association, 2024).
Forms of Interagency Collaboration
Interagency collaboration and coordination take many forms, some multilateral, some bilateral. Some collaborations are formally chartered by OMB or the ICSP to perform a specific task, while others result from
___________________
33 Officially titled the Statistical and Science Policy Office; also known as the Office of the Chief Statistician of the U.S.
common interests and may continue for years as a means of sharing information. Some interagency collaborations have been active for decades. Since 1975, the Federal Committee on Statistical Methodology (Federal Committee on Statistical Methodology, 2024), chaired by SSP, has convened technical experts across the federal government to advise OMB and the ICSP on methodological and statistical issues that affect the quality of federal data. This committee also provides a forum for statisticians in different federal agencies to discuss issues affecting federal statistical programs and promotes collaborative research.
Other ongoing collaborations, such as the Federal Interagency Forum on Aging-Related Statistics (Federal Interagency Forum on Aging-Related Statistics, 2024) and the Federal Interagency Forum on Child and Family Statistics (Federal Interagency Forum on Child and Family Statistics, 2024), provide statistical information to the public in a broad area of interest. These forums produce regular products that draw data from a wide range of agencies to provide a broad description of their population of interest in publications and materials that are easily understood and used by a broad audience.
A common bilateral arrangement is an agreement of a program agency to provide administrative data to a statistical agency to use as a sampling frame, a source of classification information, a summary compilation to check (and possibly revise) preliminary sample results, and a source with which to improve imputations for survey nonresponse, reduce variability in estimates for small geographic areas, or substitute for survey questions. The Census Bureau, for example, uses Schedule C tax information from the Internal Revenue Service in place of surveys for millions of nonemployer businesses. Such practices improve statistical estimates, reduce costs, and eliminate duplicate requests for information from the same respondents.
In other arrangements, federal statistical agencies engage in cooperative data collection with state statistical agencies to let one collection system satisfy the needs of both. A number of such joint systems have been developed, notably by BLS, the National Agricultural Statistics Service, NCES, and NCHS.
Another example of a joint arrangement is one in which one statistical agency contracts with another to conduct a survey, compile special tabulations, or develop models. Such arrangements make use of the special skills of the supplying agency and facilitate the use of common concepts and methods. The Census Bureau conducts many surveys for other agencies; both NCHS and the National Agricultural Statistics Service receive funding from other agencies in their departments to support their survey work; and NCSES receives funding from agencies in other departments to support several of its surveys.
International Collaborations
In order to be most relevant and useful, many federal statistics must be internationally comparable. The U.S. Chief Statistician at OMB is responsible for coordinating U.S. participation in international statistical activities. Many other agencies’ staffs participate in a wide variety of activities in collaboration with other national statistical offices, such as working groups sponsored by the United Nations Statistical Commission and the Organization for Economic Co-operation and Development. These activities include participating in the development of international standard classifications and systems; supporting educational activities that promote improved statistics in developing countries; and learning from and contributing to the work of established statistical agencies in other countries in such areas as survey methodology, record linkage, confidentiality protection techniques, and data quality standards.
There are a growing number of international frameworks and tools that describe the common activities of statistical organizations and facilitate the documentation and sharing of data and metadata. The Generic Statistical Business Process Model describes and defines the set of business processes needed to produce official statistics. It provides a standard framework and harmonized terminology to help statistical organizations modernize their statistical production processes, as well as to share methods and components (see Appendix C). There is also ongoing international work on using administrative and big data sources for federal statistics and on the quality frameworks for these data sources.
Challenges and Rewards for Collaboration
Collaborative activities, such as sharing and integrating data compiled by different statistical and program agencies, standardizing concepts and measures, reducing unneeded duplication, and working together on methodological challenges, invariably require effort to overcome differences in agency missions and operations. There are also potentially greater threats to confidentiality because the linking of data provides more information that can lead to indirect identification. Yet with constrained budgets and increasing demands for more relevant, accurate, and timely statistical information, the importance of proactive collaboration and coordination among statistical agencies cannot be overstated. To achieve the most effective integration of their work for the public good, agencies must be willing to take a long view, to strive to accommodate each other, and to act as partners in the development of statistical information for public use. The rewards of effective collaboration can be not only data that are more efficiently obtained, of higher quality, and more relevant to policy concerns, but also a stronger, more effective statistical system.
Statistical agencies that collect similar information should consider integrating their microdata records for specified statistical uses as another way to improve data quality, develop new kinds of information, and increase cost-effectiveness. One cost-effective approach is for a large survey to provide the sampling frame and additional content for a smaller, more specialized survey. The National Health Interview Survey run by NCHS of the Centers for Disease Control and Prevention serves this function for the Medical Expenditure Panel Survey of the Agency for Healthcare Research and Quality. Similarly, the American Community Survey serves this function for the National Survey of College Graduates, which the Census Bureau conducts for NCSES (NRC, 2008c). The Office of the Chief Statistician of the U.S., in its established role of reviewing federal agency information collection requests to evaluate public benefit given respondent burden, identifies opportunities for cross-agency frame and content sharing. The Trust Regulation calls for further engagement and consultation across federal statistical agencies to better identify and address sources of duplication, and thereby reduce respondent burden (OMB, 2024b). As the data sharing provisions of the Evidence Act are implemented through the regulatory process, the Office of the U.S. Chief Statistician could further identify and make progress on opportunities for sharing through convening an interagency workgroup.
Another key collaboration is with states. Many federal statistical agencies have relationships with states for data collection, but they would also like greater access to state administrative records. Obtaining such access often depends on having good relationships with states (Goerge, 2018; NASEM, 2017a, 2024c).
Unfortunately, there are sometimes legal or administrative barriers that prevent statistical agencies from collaborating on common activities. Both BLS and the Census Bureau maintain business establishment lists, but each of the lists derives from different sources (state employment security records for BLS and a variety of sources, including federal income tax records, for the Census Bureau). Research has demonstrated that synchronizing the lists would improve the accuracy of the information and the coverage of business establishments in the United States (NRC, 2006a, 2007b). However, business establishment lists cannot currently be synchronized between BLS and the Census Bureau, partly because the latter is prohibited by law (Internal Revenue Code of 1986, 1986) from sharing with BLS (or Bureau of Economic Analysis) any tax information on businesses or individuals that it may acquire from the Internal Revenue Service, even for statistical purposes. Previous National Academies reports have recommended that these barriers be removed, as they negatively impact data quality and efficiency (NASEM, 2017a; NRC, 2007b). The future publication of regulations implementing the Evidence Act and identification of these barriers should
lead to legal and administrative changes that will encourage joint statistical activities.
PRACTICE 8: RESPECT FOR DATA SUBJECTS AND DATA HOLDERS AND PROTECTION OF THEIR DATA
Federal statistical agencies are able to produce useful statistical information because they can collect and acquire data from data subjects and data holders, including survey respondents, organizations that provide data files, government agencies that provide administrative records, third-party data aggregators, and others. A statistical agency’s ability to fulfill its mission thus depends upon the relationships that the agency is able to build and maintain with data subjects and data holders. Effective statistical agencies demonstrate respect for their data subjects and data holders and protect their data to ensure that agencies can fulfill their missions.
To maintain a relationship of respect and trust with data subjects and data holders, a statistical agency should respect their privacy, minimize the reporting burden imposed on them, and respect their autonomy when they are asked to participate in a voluntary program to collect data. The statistical agency must also comply with all legal requirements to ensure that the data are used only for statistical purposes. To do this, a statistical agency must communicate its privacy and confidentiality protection procedures and policies,34 as well as the societal benefits from collecting the data.
Respecting Privacy in Surveys
To promote trust and encourage accurate responses from survey respondents, it is important that statistical agencies respect their privacy by reducing, to the extent possible, the intrusiveness of questions they ask, and the time and effort required to respond. Agencies must also give respondents adequate information with which to decide if a survey is worthy of response—that is, so respondents can give their informed consent (see below). Thus, when individuals or organizations are asked to participate in a survey, they should be told whether it is mandatory or voluntary, how the data will be used, and what confidentiality protections apply to the data. They should also be informed of the likely duration of a survey response
___________________
34 Informational privacy is “an individual’s freedom from excessive intrusion in the quest for information, and an individual’s ability to choose the extent or circumstances under which his or her beliefs, behaviors, opinions, and attitudes will be shared with or withheld from others,” while confidentiality refers broadly to an obligation not to transmit information to an unauthorized party (NRC, 1993a, p. 22).
task, whether they will be asked to consult records, and whether the survey involves repeated responses over time (OMB, 2016d).
To reduce the burden of replying to surveys (NRC, 2013c, Ch. 4; OMB, 2016f), statistical agencies should write clear questions that fit respondents’ common understanding, minimize the intrusiveness of questions, and explain why intrusive-seeming questions serve important purposes. Statistical agencies should also allow alternative modes of response when appropriate (e.g., Internet, smartphone) and use administrative records or other data sources, if sufficiently complete and accurate, to provide some or all of the needed information. In surveys of businesses or other organizations, agencies should seek to obtain information directly from the organization’s records and so minimize the need for duplicate responses to multiple requests. As described under Practice 7, the Office of the U.S. Chief Statistician reviews federal agency information collection requests to determine public value given respondent burden. Through this established process, opportunities for sharing sampling frames and content across agencies are identified, with an eye on reducing duplication. To further identify and make progress on these opportunities, the Office of the U.S. Chief Statistician could usefully convene an interagency working group.
Protecting and Respecting the Autonomy of Human Research Participants
Collecting data from individuals for research purposes using federal funds falls under a series of regulations, principles, and best practices that the federal government has developed over a period of more than 50 years (NRC, 2003b, 2014). The pertinent regulations, which have been adopted by 20 departments and agencies,35 are known as the “Common Rule” (Federal Policy for Protection of Human Subjects, 2018). The Common Rule regulations, most recently revised effective January 2019, require that researchers adequately protect the privacy of human participants and maintain the confidentiality of data collected from them, minimize the risks to participants from the data collection and analysis, select participants equitably with regard to the benefits and risks of the research, and seek the informed consent of individuals to participate (or not) in the research. Under the regulations, most federally funded research involving human participants must be reviewed by an independent institutional review board (IRB) to determine whether the design meets ethical requirements for protection.36
___________________
35 https://www.hhs.gov/ohrp/sites/default/files/revised-common-rule-reg-text-unofficial-2018-requirements.pdf
36 For information about the Common Rule and certification of IRBs by the Office for Human Research Protections in the U.S. Department of Health and Human Services, see http://www.hhs.gov/ohrp and Appendix A.
Not all federal statistical agencies’ data collections are subject to IRB review. Nonetheless, agencies should strive to incorporate the spirit of the Common Rule in the design and operation of all activities that involve data collection from individual respondents. Statistical agencies should seek ways to inform potential respondents that will help them decide whether to participate, such as sending respondents an advance letter. Such information should include the planned uses of the data and their benefits to the public.
Even for mandatory data collections, such as the decennial Census of Population and Housing and the quinquennial Economic Census, a statistical agency should respect its respondents by giving them as much information as possible about the reasons for the collection and making it as easy as possible for them to respond (OMB, 2016d). The principles and practices of respect apply not only to individuals asked to participate in a survey, but also to representatives of organizations (e.g., businesses, state, and local governments) asked to participate in a survey and to custodians of existing data, such as administrative records, who are asked to share their data for statistical purposes.
Respecting the Holders and Subjects of Administrative and Other Data
Moving to a new paradigm of using multiple data sources for federal statistics, an agency must develop procedures that respect the constraints of data holder organizations. In working with federal agencies that hold useful administrative records, statistical agencies should plan to cooperate, communicate, and coordinate with them on a continuing basis, as urged in Hendriks (2012). A continuing relationship of mutual respect and trust enables a statistical agency to better understand the strengths and limitations of data held by a custodial agency. Mutual respect can help identify improvements in the data that are useful to both agencies.
An important consideration in using administrative records is whether informed consent of the data subjects (whether individuals or organizations) that provided their information to the data holder agency is required. In many cases the statistical use of administrative records may qualify under the “routine use” exception of the Privacy Act to provide evidence for the effective operation of the program. In some instances, it may be necessary to obtain new consent from the original data subjects.
Protecting the Confidentiality of Data Subjects’ Information
When individuals and organizations provide information to statistical agencies, they advance the public good. These data subjects must be able to rely on the statistical agency’s promise to protect their information, to use it only for statistical purposes, and to protect it from other uses.
A credible pledge of confidentiality for data subjects is considered essential to encourage high response rates and accuracy of responses from survey participants.37 Moreover, if data subjects have been assured of confidentiality, disclosure of identifiable information would violate the principle of respect even if the information is not sensitive and would not result in any social, economic, legal, or other harm. For sensitive administrative data obtained from another government agency data holder, there must be a credible pledge of confidentiality in a properly formulated memorandum of understanding or other authorizing document.
CIPSEA (Confidential Information Protection and Statistical Efficiency Act, 2002) was originally enacted in 2002 and recodified as part of the Evidence Act (Foundations for Evidence-Based Policymaking Act of 2018, 2019; see Appendix A). This law protects the confidentiality of all federal data collected for statistical purposes under a confidentiality pledge, including but not limited to data collected by statistical agencies.38 CIPSEA thus provides a common basis for the protection of all statistical data across agencies, which enables some data sharing and provides statistical agencies the ability to designate external researchers as their agents in order to allow them access to data for statistical purposes. The law contains penalties for employees and agents who knowingly disclose identifiable statistical information (up to 5 years in prison, up to $250,000 in fines, or both).
Both the perception and the reality of agencies’ confidentiality protection may be affected by departmental initiatives to consolidate data processing and storage to bolster computer and network security in the federal government, improve the cost-effectiveness of information technology development and maintenance, and protect against cyberattacks. An effective statistical agency will work with its department on approaches to computer security. As part of their responsibilities to support federal statistical agencies, departments should ensure that statistical agencies are able to control their data and information systems so that the data are only used for statistical purposes and are kept confidential (see Practice 2).
___________________
37 See (Hillygus et al., 2006; NRC, 1979, 2004d, 2013b).
38 Section 508 of the USA PATRIOT Act of 2001 (P.L. 107-56) amended the NCES Act of 1994 to allow the U.S. attorney general (or an assistant attorney general) to apply to a court to obtain any “reports, records, and information (including individually identifiable information) in the possession” of NCES that are considered relevant to an authorized investigation or prosecution of domestic or international terrorism. Section 508 also removed penalties for NCES employees who furnish individual records under this section. This exclusion for NCES has not been invoked.
PRACTICE 9: DISSEMINATION OF STATISTICAL PRODUCTS THAT MEET USERS’ NEEDS
An effective statistical agency produces and disseminates statistical products that meet the needs of its users. Users’ needs for information evolve as new tools for using data become available. Once statistical information is made public, it will be used in numerous ways, including ways not originally envisaged, and by numerous types of users, ranging from government officials (whether federal, state, or local) to media, activists, data wholesalers, academic scholars, students, and data subjects. A statistical agency should continually strive to obtain input from data users on its programs, products, and dissemination tools and methods. Understanding data users’ needs and how they use data products is critical for making an agency’s data services as relevant, accurate, timely, and accessible as possible.
To return full value to the populace, statistical agencies also have an important role to play in helping users understand the strengths and limitations of the agencies’ data, and, when asked, to contrast the usefulness of those data to user needs as compared to data from other sources being considered in the public domain. Citizens who might be misled by untested or biased alternative sources of data will benefit from understanding that they can count on the statistical agencies to produce high quality, objective, and trustworthy information. Furthermore, as the statistical agencies model good practices, they provide a natural contrast to less reliable statistical practices and products.
There is also increasing demand for statistics at greater levels of granularity, whether for disaggregated geographies or subgroups of the population relevant for policymaking. Statistical agencies must find ways to meet these increasing data demands while balancing the needs of data subjects and data holders (see Principle 3; Bowen & Snoke, 2023; NASEM, 2023b,c, 2024c).
Keeping abreast of the interests of current and potential new users requires continual attention to changes in the relevant policy issues and social and economic conditions in a statistical agency’s domain, as well as changes in technologies for data access. Statistical agencies should work with professional associations, institutes, universities, and scholars to determine the current and emerging needs of research communities. They should also work with relevant professional associations and other organizations to determine the needs of business and industry as well as state and local governments. Statistical agencies also should proactively explore the needs of users through advisory committees.
Individual persons, households, businesses, institutions, organizations, and government entities have provided the underlying source data for an
agency’s statistics. Furthermore, the public has paid for the data collection, compilation, and processing. In return, the information created with such data should be accessible in ways that make it as useful as possible to the largest number of users—for decision making, program evaluation, scientific research, and public understanding (see also the Federal Data Strategy (OMB, 2021a) in Appendix A).
A statistical agency should strive for the widest possible dissemination of the statistical products it produces, consistent with its obligations to protect confidentiality. The products should be clearly identified, easy to find and use, and well documented. They should adhere to the FAIR (findable, accessible, interoperable, and reusable) principles,39 which are becoming an international standard with adherence often required by research funding organizations and increasing adoption by national statistical institutes (Cabrera et al., 2020; Group of Eight, 2013). Dissemination should be timely, and information should be made readily available on an equal basis to all users. Agencies should have data curation policies and procedures in place so that data are preserved, fully documented, and accessible for statistical use in future years.40
Statistical agencies disseminate two broad classes of products: products that are publicly available, such as statistical releases, analytical reports, infographics, and public use microdata, and restricted access products, such as datasets containing confidential information, which are available in Federal Statistical Research Data Centers (FSRDCs) or through other restricted arrangements.41 Both of these broad dissemination classes of data products actually come from the same underlying survey, administrative, and private data sources. This interdependency is important to understanding the risks when balancing public data products and expanded access to restricted data, and the effort thus required to manage access to both dissemination classes appropriately. Nonetheless, the process of gaining access to restricted data is often very time-consuming and costly for those who request access (see Conclusion 2–3 in NASEM (2023a). This undermines the principle of equitable access to data by a variety of users.
Increasingly, statistical agencies are moving away from this binary system of access models.42 “Tiered access is an application of data minimization, a key privacy safeguard for evidence building […] Data minimization means giving access to the least amount of data needed to complete an
___________________
39 See https://www.go-fair.org/fair-principles/
40 Data curation involves the management of data from collection and initial storage to archiving (or deletion should the data be deemed of no further use—e.g., a data file that represents an initial stage of processing). The purpose of data curation is to ensure that information can be reliably retrieved and understood by future users.
approved project. Tiered access uses a variety of controls to minimize data needed for a given project and thereby reduce risk of disclosure of confidential data” (Commission on Evidence-Based Policymaking, 2017, p. 38). The Standard Application Process (SAP; OMB, 2022a) implemented as part of the Evidence Act is designed to simplify and speed up the application process for access to restricted data, but more needs to be done to facilitate such access (NASEM, 2023c).
Public Statistical Data Products
Dissemination of aggregate statistics may take the form of regularly updated time series, cross-tabulations of aggregated characteristics of respondents, analytical reports, interactive maps and charts, infographics, profiles, fact sheets, and press releases providing key findings. Such products should be readily accessible through an agency’s website, supplemented by more detailed tabulations and data tools.43
Statistical agencies are using a variety of tools that make it easier for users to discover, query, retrieve, and use statistics on their own. These include redesigned websites, new platforms, interactive tools and applications, customized tables, new mapping capabilities, and application program interfaces.
For publicly available data products, a statistical agency’s dissemination program should include the following elements:
- An established publications policy that describes, for each statistical program, the types of reports and other data releases to be made available, the formats to be used, the audience to be served, and the frequency and timing of release;44
- A variety of avenues for disseminating information about data availability and upcoming releases;
- Multiple data products (suitably processed to protect confidentiality), so that information can be accessed by users with varying skills and needs. Useful data products may include not only understandable maps, graphs, indicators, tables, and interactive data tools and applications on statistical agency websites that meet the needs of different user groups, but also public-use microdata samples when practicable;
___________________
43 Note the Trust Regulation requires each recognized statistical agency or unit to maintain a distinctive, outward-facing website with its own domain name and with adequate control over the website content and management to uphold the fundamental responsibilities. See § 1321.4(e).
44 See Statistical Policy Directive No. 3 (OMB, 1985) and 4 (OMB, 2008) in Appendix A.
- Statistical press releases when data products are made available, produced, and issued by the statistical agency to provide a policy-neutral description of key findings and links to the data. Such releases must not include any policy commentary;45
- Explanatory materials for all statistical product releases that assist users in understanding the product and convey the strengths and limitations of the data (see Practice 10); and
- Archiving policies that guide decisions on which underlying data assets are to be retained, where they are to be archived (with the National Archives and Records Administration, or with an established archive maintained by an academic or other nonprofit institution or both), and how they are to be made accessible for future secondary analysis while protecting confidentiality.
Individual-level microdata files make it possible for users to conduct in-depth research and analyses that are not possible with aggregate data. Such files contain data for samples of individual respondents that have been processed to protect confidentiality by deleting, aggregating, or modifying any information that might permit individual identification (Federal Committee on Statistical Methodology, 2022). Statistical agencies should keep abreast of new developments in confidentiality protection so that they can continue to provide as much useful aggregate data and microdata as possible at a time of increasing threats to privacy and confidentiality. The increasing availability of the amount and type of data increases risk to confidentiality, even of previously released public-use data (NASEM, 2017a,b,c, 2024c). Statistical agencies must remain mindful that once released in public-use form, these microdata cannot be unreleased. Preparation of public-use data files must take these factors into account.
Restricted-Access Statistical Products
Some statistical agency data are deemed too difficult to protect in public releases and are made available to bona fide researchers only through some form of restricted access. To provide researchers with the ability to run their own analyses on restricted microdata, some statistical agencies provide access to an analysis engine on their websites that performs the selected statistical operations on the confidential data. Safeguards are built in so that the researcher cannot see the individual records and cannot obtain any output, such as too-detailed tabulations, that could identify individual respondents.46
___________________
45 See Statistical Policy Directive No. 4, Section 6a, (OMB, 2008) in Appendix A.
46 For example, https://nces.ed.gov/datalab/index.aspx
A second method, pioneered by NCES, is to grant licenses to individual researchers to analyze restricted microdata at their own sites for statistical purposes. Such licenses require that the researchers agree to follow strict procedures for protecting confidentiality and accept liability for penalties if confidentiality is breached.47
A third method is to allow approved researchers to analyze restricted microdata for statistical purposes at a secure physical site, such as one of the FSRDCs currently located at 33 universities and research organizations around the country.48 The FSRDC network began as a Census Bureau initiative. It has now expanded as a shared service across the federal statistical system, directed through the ICSP through the FSRDC EXCOM subcommittee, with the Census Bureau serving as the program management office. The FSRDCs offer access to data from other participating agencies.49
A fourth method allows researchers to analyze restricted microdata for statistical purposes via a secure virtual site, such as virtual data enclaves and secure virtual desktop platforms. Some agencies, such as the Centers for Medicare & Medicaid Services, are moving away from data licenses toward secure virtual environments to support data access while protecting confidentiality.50
Recent Innovations in Facilitating Access to Confidential Data for Statistical Purposes
The Evidence Act is expected to facilitate greater use of data for statistical purposes. The Act requires OMB to issue guidance on tiers of access for data depending on their sensitivity and legal protections. The Act also required OMB to implement a single-application process for access to data from recognized statistical agencies and units.
The Standard Application Process
In December 2022, OMB issued M-23-04 Establishment of Standard Application Process Requirements on Recognized Statistical Agencies and Units. The goal of the SAP is to provide a standardized and transparent process for applying for access to confidential data.51 The memorandum
___________________
47 For example, https://nces.ed.gov/statprog/instruct.asp
48 https://www.census.gov/about/adrm/fsrdc/locations.html#:~:text=There%20are%20currently%2033%20open,research%20institutions%2C%20and%20government%20agencies
49 https://www.census.gov/fsrdc
50 For example, https://coleridgeinitiative.org/adrf/ and https://www.norc.org/Research/Capabilities/Pages/data-enclave.aspX; https://www.icpsr.umich.edu/web/pages/appfed/index.html
51 See also https://ncses.nsf.gov/initiatives/standard-application-process/annual-reports
describes the application process, and the roles assigned to responsible entities. (See also Appendix A.)
In brief, the SAP Governance Board, operating as a subcommittee of the ICSP, functions as an executive steering committee. The SAP Governance Board is responsible for setting criteria and approving requests from non-statistical agencies to access confidential statistical data (OMB, 2022a, p. 7). It is responsible for evaluating progress in meeting the objectives of the SAP and identifying opportunities for improvement (OMB, 2022a, p. 7).
Additionally, an SAP Program Management Office (PMO) “[…] is responsible for the development, operation, and maintenance of the SAP portal and any additional technical services required to facilitate the SAP” (OMB, 2022a, p. 7). The PMO is also responsible for managing the SAP data catalogue, documenting and managing implementation guidance, and supporting stakeholder engagement activities with the SAP Governance Board (OMB, 2022a, p. 7).
The access requirements described in M-23-04 describe tiered authorization requirements for access, although these are not tied to specific modes of access (OMB, 2022a, p. 19).
The National Secure Data Service Demonstration Project (NSDS-D)
There have been additional federal efforts to increase the means by which confidential data can be accessed for research purposes. The NSDS-D project was established by the 2022 CHIPS and Science Act.52 The “goal of the NSDS-D project is to inform efforts for developing a shared services model that would streamline and innovate data sharing and linking to enable decision making at all levels of government and in all sectors.”53 The demonstration project will inform whether an NSDS will be established in the future. To that end, the NSDS-D will pilot potential services, technologies, techniques, and shared services that could be used within a potential NSDS. The focus is on novel research collaborations, data linkage methodologies, and privacy-preserving technologies and techniques. As of September 2024, 15 demonstration projects are active.
___________________
52 A National Secure Data Service, or NSDS, was first introduced by the bipartisan Commission on Evidence-Based Policymaking in 2016, and later further envisioned through specific recommendations provided by the Advisory Committee on Data for Evidence Building in 2022.
53 https://ncses.nsf.gov/about/national-secure-data-service-demo#card1893
America’s DataHub Consortium
The National Science Foundation (NSF) and NCSES established America’s DataHub Consortium (ADC)54 in August 2021 through an NSF contract with Advanced Technology International. The ADC is a public-private partnership through which federal, state, and local government agencies can submit proposals and sponsor testing of data service structures and functions that, in turn, could inform the vision for the NSDS. Members of the ADC include small and large businesses, nonprofit organizations, and academic institutions.
The ADC and NSDS-D are distinct, but their efforts are complementary. The NSDS-D will create and test the infrastructure and resources that support this shared-service operational model. The ADC facilitates this evidence-building by streamlining collaboration between researchers, innovators, and subject matter experts.
PRACTICE 10: OPENNESS ABOUT SOURCES AND LIMITATIONS OF THE DATA PROVIDED
A statistical agency must be transparent about how it acquires data and produces statistics and must be open about the strengths and limitations of its data. No matter how high-quality the statistical data are, they contain some uncertainty and error. This does not mean the data are wrong but does mean that they need to be used with an understanding of their limitations. Statistical agencies need to communicate clearly to a wide range of potential users what the uncertainty in the data means for using the statistical information appropriately.
To be most effective, openness should be tailored to different user groups. For press releases disseminated to the public, the agency must make every effort to note both the meaning of the statistics and their limitations for various uses. For more technically trained users, detailed descriptions of methods and measures of quality should be made available (Federal Committee on Statistical Methodology, 2001, 2020; Mirel et al., 2023).
Openness requires that statistical releases from an agency include a full description of the purpose of the program; the methods and assumptions used for data collection, processing, and estimation; information about the quality and relevance of the data; analysis methods used; and the results of research on the methods and data (NASEM, 2019a, 2022d). Such transparency is essential for credibility with data users and stakeholders and for public trust. Thus, openness about statistical limitations requires much more than providing estimates of sampling error. In addition to a discussion
___________________
54 https://www.americasdatahub.org/frequently-asked-questions/
of nonsampling errors (e.g., coverage errors, nonresponse errors, measurement errors, and processing errors), it is valuable to have a description of the concepts measured and how they relate to the major uses of the data (NASEM, 2022d). Descriptions of the shortcomings of the data should be provided in sufficient detail to permit a user to take them into account in analysis and interpretation. Descriptions of how the data relate to similar data collected by other agencies or generated in the private sector also should be provided, particularly when the estimates from two or more surveys or other data sources exhibit large differences that may have policy implications (NASEM, 2022d).
There is often a tension between timeliness and accuracy. When concerns for timeliness prompt the release of preliminary estimates (as is done for some economic indicators and has been done in response to COVID-19), consideration should be given to the frequency of revisions and the mode of presentation from the point of view of the users as well as the issuers of the data. Agencies that release preliminary estimates must educate the public about differences among preliminary, revised, and final estimates.
An important aspect of openness concerns the treatment of errors that are discovered subsequent to data release. Openness means that an agency has an obligation to issue corrections publicly and in a timely manner. The agency should use not only the same dissemination avenues to announce corrections that it used to release the original statistics, but also additional vehicles, as appropriate, to alert the widest possible audience of current and future users of the corrections in the information. Agencies should be proactive in seeking ways to alert potential users of the data about the nature of a problem and the corrective actions that it is taking or that users should take.
Federal statistical agencies should implement quality frameworks for their programs and use these to describe the strengths and limitations of the statistical information produced by the data (see Practices 3 and 6). Some statistical agencies have developed “quality profiles” for major surveys, which document what is and is not known about errors in estimates to help users (NASEM, 2022d).
In cases where agencies are using data from administrative and other alternative sources, they need to provide information not only on what is known about those sources but also on how the data were linked or blended with other data sources and the potential errors introduced through linkage. This is a challenging and evolving area (NASEM, 2017a), and efforts are ongoing55 to develop best practices and quality frameworks for these topics (Czajka & Strange, 2018; Federal Committee on Statistical Methodology, 2020; NASEM, 2022d, 2023b,c, 2024c; Prell et al., 2019).
___________________
Statistical agencies should treat the effort to provide information on the quality, limitations, and appropriate use of their data as an essential part of their mission. Such information and metadata should be readily accessible to all known and potential users (NASEM, 2022d; NRC, 1993b, 1997a, 2007a). By being open about the sources and limitations of their data and by providing as much data as possible in ways that are as easy as possible for users to access and apply, federal statistical agencies fulfill their vital mission to inform the public, contribute to evidence-based policymaking, and support the development of societal knowledge for the public good.
