The information, examples, and challenges discussed here were contributed by individuals who represent a range of decision-makers and actors within the higher education community, including Title IX coordinators, academic administrators, university leadership, sexual assault prevention and education program leaders, faculty members, and evaluation experts. Further, the authors of this paper have experience and connections to the development of previous evaluation-related resources. Two of the authors of this paper (Nicole Merhill and Mariam Lam) were also authors on the 2021 Guidance for Measuring Sexual Harassment paper. Nicole Merhill was on the Planning Committee on Developing Evaluation Metrics for Sexual Harassment Prevention Efforts for the 2021 Evaluating the Effectiveness of Interventions Workshop, as well as a workshop participant herself. Although only 3 individuals are listed as authors on this paper, 10 others contributed to the conceptualization and development of this paper at various times as representatives of their respective institutions to the Evaluation Working Group.

Furthermore, the methods discussed here were informed by an extensive literature review focusing on evaluation broadly, sexual harassment evaluation specifically, and case studies examining implementation. We believe that building knowledge of the tools needed to gather information on sexual harassment interventions, starting with the institutional members of the Action Collaborative and then expanding beyond, will not only be of service to individual faculty, academic administrators, and higher education broadly, but will also assist in creating the changes in organizational culture and climate essential to preventing both sexual harassment and experiences of institutional betrayal (Cantalupo and Kidder, 2019; Flaherty, 2019; Smith and Freyd, 2014).

Understanding Sexual Harassment Evaluation

Evaluation is an important part of the research process because it provides a way to understand the effectiveness of an intervention and identify areas for improvement or inform choices around de-implementation. Whether or not an intervention is having the desired effect can only be assessed through that process, using any number of approaches. Ultimately, these efforts are intended to support decision-making by answering questions such as the following:

• Does the intervention achieve its intended goals?
• How does the intervention compare to other interventions or approaches? How does it compare with doing nothing?
• What are the costs and benefits of the intervention?
• Who is most likely to benefit from the intervention, and under what circumstances?

Many of the interventions highlighted in this paper are focused on preventing sexual harassment because it has a clear metric for success—the reduction of incidents of harassment—which makes evaluation efforts

more straightforward. However, evaluation is important to all interventions, including those aimed at responding to harassment, as well as policies and procedures that target organizational change. Interventions may look like the University of California, Santa Cruz (UCSC)’s “Guide on Best Practices for Graduate Students Impacted by SVSH,” which is a set of procedures intended to remediate harm to those who have experienced sexual violence or harassment (UCSC, 2020). Interventions may also look like Boston University’s “Beyond Bystanders” program, which is “a longitudinal bystander intervention training series designed to provide male graduate students in STEM programs with the skills to identify and interrupt sexual misconduct, with a focus on gender harassment” (Boston University, 2021). Even Michigan State University’s changes to its disciplinary and sanctions process could be considered an intervention that aims to increase transparency and improve understanding of its policies and procedures (MSU, 2022).

Similarly, the evaluations for each of those interventions may look very different from one another. Evaluations—and the methods selected to support them—are dependent on what the intervention aims to accomplish, and therefore what each intervention defines as a successful outcome. Outcomes of interest can include simple goals related to prevalence rates or awareness and knowledge. Outcomes of interest can also include more complex goals related to climate and culture, trust, or transparency.

For the purposes of this paper, evaluation and intervention are defined as follows:

• Evaluation – “A systematic method for collecting, analyzing, and using data to examine the effectiveness and efficiency of programs and, as importantly, to contribute to continuous program improvement” (CDC, 2023).
• Intervention – Any organized action intended to prevent or respond to sexual harassment, broadly inclusive of policies, procedures, practices, or “any set of related activities undertaken to achieve an intended outcome” (CDC, 2023).²

The goal of this paper is to provide interested parties in higher education with an introduction to methods and approaches to evaluating interventions designed to prevent and respond to sexual harassment in their institutions. To achieve this goal, we (1) describe approaches to the evaluation of interventions broadly, (2) detail a subset of potential methods and resources needed to rigorously evaluate different types of interventions and the application of these methods, (3) provide concrete examples of interventions and their assessment, (4) discuss legal and ethical challenges related to evaluation of sexual harassment intervention efforts, and (5) explore approaches for minimizing challenges associated with evaluating interventions.

___________________

² For examples of the diversity of organized actions that may be considered interventions, see the Action Collaborative’s Repository: https://www.nationalacademies.org/our-work/action-collaborative-on-preventing-sexual-harassment-in-higher-education/repository.