Objectives, Components, and Measures of Effective Traffic Safety Public Awareness and Education Efforts (2025)
Chapter: 3 Strengths and Weaknesses of Current Evaluation Methods
Chapter 3. Strengths and Weaknesses of Current Evaluation Methods
An understanding of the strengths and weaknesses of each method used to evaluate the effectiveness of traffic safety campaigns is crucial to the decision-making process. Each method has pros and cons, including financial resources and expertise needed to accomplish the evaluation, and each evaluation method is linked to different measurement metrics and methods of measurement. It is not uncommon for multiple evaluation methods to be combined to provide a multifaceted approach to the evaluation of traffic safety campaigns. For example, a survey may be conducted to assess awareness of a campaign combined with an analysis of epidemiological data from before and after the campaign to determine the impact of the traffic safety messaging.
Methods
In addition to the literature identified in Chapter 2, data sources containing pertinent information on current evaluation methods and approaches developed both within the United States and around the world were identified. Information was compiled and synthesized to highlight the linkages among the measurement metrics, objectives, and resource requirements associated with the various evaluation methods and best practices in campaign evaluation. Information related to the specific and measurable goal or purpose (objective), data collection methods used, the ability to accurately and comprehensively measure the intended aspect of the campaign (strengths), and the limitations in assessing the effectiveness of the campaign (weakness) were summarized for each metric or data collection method. Additionally, specific resource considerations were also noted, such as tools, data, and staff required to collect, process, and analyze information. To ensure inclusion of all current evaluation methods, the literature search scope was expanded to include campaign evaluations outside of traffic safety.
Results
Results are grouped into eight categories. The first seven categories of Knowledge, Attitudes, Beliefs, Norms, and Intentions; Campaign Awareness and Recognition; Media Metrics; Campaign Cost-Effectiveness and Efficiency; Secondary Quantitative Metrics; Behavioral Metrics; and Other Evaluation Methods focus on metrics used and measured by campaigns. The Data Collection Methods category identified avenues in which information is gathered. The information below is also presented as a series of easy-to-use reference tables to be included in the toolkit for SHSO’s and other stakeholders (see Appendix A. Strengths and Weaknesses of Current Evaluation Methods).
Knowledge, Attitudes and Perceptions, Beliefs, Norms, and Intentions
Knowledge
The purpose of measuring knowledge in road safety campaigns is to evaluate how well the target audience understands traffic safety campaigns and initiatives, such as seat belt usage and enforcement laws. Knowledge assessment aims to quantify how much information the audience retains about these campaigns, serving as a foundation for subsequent behavioral shifts. Data collection methods include both quantitative and qualitative approaches. Quantitative methods
include surveys and questionnaires distributed via phone, internet, or in-person intercepts, which collect measurable data on the public’s awareness. Qualitative methods, including interviews, focus groups, and open-ended survey questions, provide deeper insights into the audience’s thoughts on safety initiatives. Necessary resources include designing comprehensive survey and interview guides, assembling a trained data collection team, and implementing effective database management tools for storing and analyzing responses.
A key strength of measuring knowledge is its objectivity, as knowledge can be quantified through standardized surveys and questionnaires. This quantifiable data can be tracked over time to measure changes, allowing for comparative analyses as the campaign progresses. Knowledge assessments also provide a baseline, enabling evaluators to understand the audience’s prior awareness before the campaign. However, knowledge measurement has several limitations. Since it relies on self-reported data, it can be influenced by response bias, with respondents potentially providing answers they believe are socially desirable rather than reflecting their true knowledge. Moreover, while knowledge can indicate awareness levels, it does not necessarily correlate with behavior change; just because someone knows about a campaign does not mean they will alter their behavior accordingly. Finally, the quality of assessment tools can significantly impact data reliability, underscoring the importance of well-designed surveys and guides to gather accurate insights.
References
1. Delhomme P, De Dobbeleer W, Forward S, and Simões A. (2009). Manual for designing, implementing, and evaluating road safety communication campaigns: Part I. Institut Belge pour la Sécurité Routière. Available from: https://www.vias.be/storage/main/cast-wp3-deliverable3.2a.pdf
2. Kaiser S, and Aigner-Breuss E. (2017). Effectiveness of Road Safety Campaigns. European Road Safety Decision Support System Safety Cube. Available from: https://www.roadsafetydss.eu/assets/data/pdf/synopses/Effectiveness_of_Road_Safety_Campaigns_26072017.pdf
3. Brayne A, Siegfried A, La Rose C, and Price J. (2020). Evidence-Based Behavior Change Campaigns to Improve Traffic Safety. AAA Foundation for Traffic Safety. Available from: https://aaafoundation.org/wp-content/uploads/2020/03/Toolkit-Evidence-Based-BehaviorChange-Campaigns-to-Improve-Traffic-Safety.pdf
Attitudes and Perceptions
Measuring attitudes and perceptions provides insights into how the target audience thinks and feels about road safety topics, such as campaign messages and safe driving practices. By evaluating these subjective views, campaigns can tailor better their messaging and interventions to resonate with audience sentiments and concerns. Data collection methods for this assessment are identical to those used for knowledge, employing both quantitative surveys and qualitative techniques like focus groups and interviews. These methods allow for a comprehensive understanding of attitudes that may be challenging to quantify. Resources required for this assessment are similar as well, including the development of survey and interview guides,
assembling a skilled data collection team, and creating a database management system for processing responses.
One strength of measuring attitudes and perceptions is that it provides valuable insights for crafting targeted interventions that address specific concerns or misconceptions held by the audience. This data allows for the customization of campaign messages to ensure they effectively engage the audience. However, measuring attitudes has inherent weaknesses. The audience’s pre-existing characteristics, such as a natural inclination toward safety, may introduce response bias, skewing results and limiting the evaluative power of these assessments. This limitation can make it challenging to accurately measure changes in attitude since participants who are already conscious of safety may not show significant shifts. Another challenge arises when participants are asked to complete surveys multiple times, as with pre-and post-campaign surveys. Repeated participation can lead to response bias, as individuals more inclined to participate may already exhibit a heightened interest in road safety, potentially skewing the results toward those already engaged in the campaign topic.
References
4. Cuenen A, Brijs K, Brijs T, Van Vlierden K, Daniels S, and Wets G. (2016). Effect evaluation of a road safety education program based on victim testimonials in high schools in Belgium. Accident Analysis and Prevention, 94, 18–27.
5. Sack R, Foreman C, Forni S, Glynn R, Lehrer A, and Linthicum A, et al. (2019). Social Media Practices in Traffic Safety (Report No. DOT HS 812 673). Washington, D.C.: National Highway Traffic Safety Administration. Available from: https://www.ghsa.org/sites/default/files/201906/NCREP_SocialMedia19.pdf
Beliefs
Assessing the audience’s beliefs provides an understanding of the deeper motivations and perceptions that influence attitudes and behaviors related to road safety. These beliefs include behavioral beliefs (about the outcomes of actions), control beliefs (about personal influence over actions), and normative beliefs (about social expectations or pressures). The data collection methods for beliefs are the same as those used for knowledge and attitudes, combining quantitative surveys and questionnaires with qualitative interviews and focus groups. This mixed-method approach captures the complexity of beliefs, allowing evaluators to delve into both measurable data and nuanced perceptions. The resources needed include detailed survey and interview guides, a skilled data collection team, and a reliable data management system.
The primary strength of measuring beliefs is that it reveals valuable information about the audience’s underlying motivations, which is critical for designing interventions that target specific behavioral drivers. By understanding these beliefs, campaign developers can tailor messaging to address misconceptions, fears, or motivations that may impact road safety behavior. However, one of the main weaknesses of self-reported beliefs is their susceptibility to social desirability bias. Participants may respond based on perceived social norms rather than
their actual beliefs, which can distort data accuracy and limit the campaign’s ability to address genuine concerns.
References
6. Warner, H.W. and Forward, S. (2016). The effectiveness of road safety interventions using three different messages: Emotional, factual or a combination of both messages. Transportation Research Part F: Traffic Psychology and Behaviour, 36, 25–34.
7. Chaurand, N., Bossart, F., and Delhomme, P. (2015). A naturalistic study of the impact of message framing on highway speeding. Transportation Research Part F: Traffic Psychology and Behaviour. 35, 37–44.
8. Gauld, C.S., Lewis, I., White, K.M., Fleiter, J.J., and Watson, B. (2017). Evaluating public education messages aimed at monitoring and responding to social interactive technology on smartphones among young drivers. Accident Analysis and Prevention, 104, 24–35.
Norms
The objective of assessing norms is to measure the target audience’s perceptions of social norms related to certain behaviors, such as wearing seat belts or avoiding impaired driving. Understanding these social norms provides insight into the social pressures that may influence individuals’ behavior choices. The data collection methods for this assessment are like those for knowledge, attitudes, and beliefs, using both quantitative surveys and qualitative interviews to gather a range of perspectives. Necessary resources include the same survey/interview guides, data collection staff, and database management.
A key strength of measuring norms is that aligning campaign messages with perceived social norms can encourage behavior change more effectively. When individuals see a behavior as a widely accepted norm, they are more likely to adopt it themselves. This alignment fosters targeted and relatable campaigns that resonate with the audience. However, norms are inherently subjective and can vary widely across different cultural and demographic groups, making them difficult to measure. Another limitation is that norms can be influenced by external factors, such as media representation or societal changes, which are beyond the campaign’s control. This variability complicates efforts to ensure that the campaign is accurately targeting the most relevant social norms.
References
Intentions
The purpose of measuring intentions is to gauge the target audience’s desire or intent to adopt or change a specific behavior, such as complying with seatbelt regulations or avoiding distracted driving. Intentions serve as an indicator of the likelihood of future behavior, although they are not always directly predictive. Data collection methods for measuring intentions are consistent with those used in other sections, incorporating both quantitative surveys and qualitative
interviews to capture a range of responses. Required resources include detailed survey and interview guides, a trained data collection team, and an effective data management system.
A strength of measuring intentions is that it allows for tailored messaging that directly appeals to the audience’s readiness to change behavior. Understanding intentions helps campaign developers identify the best approaches for encouraging desired actions. Measuring intentions is also relatively cost-effective, as it does not require expensive observation methods. Likert-type questions provide an additional advantage by quantifying intentions, making it easy to analyze and interpret data. However, measuring intentions has limitations. Self-reported intentions may not correlate directly with actual behavior change, as people’s plans or desires do not always translate into actions. Additionally, response bias may influence reported intentions, as individuals may provide responses they think are socially acceptable. This method is stronger when combined with other data sources, such as naturalistic observations, to better assess actual behavior.
References
9. Glendon, A.I., and Walker, B.L. (2013). Can anti-speeding messages based on protection motivation theory influence reported speeding intentions? Accident Analysis and Prevention, 57, 67–79.
Campaign Awareness and Recognition
Campaign Awareness
The goal of assessing campaign awareness is to determine whether the campaign has reached its intended audience and raised awareness about its key messages and objectives. Evaluating awareness helps identify if the audience remembers and recognizes the campaign. Data collection methods include surveys, focus groups, and interviews that gauge unaided recall (i.e., without campaign prompts) to understand the level of awareness without any external cues. Required resources include survey and interview guide design, trained data collection staff, and database management for handling the responses.
The primary strength of this approach is that campaign awareness can serve as a moderating variable in campaign evaluations, showing how the campaign’s presence impacts specific outcomes, such as changes in attitudes or behaviors. Additionally, awareness measurements provide valuable insights into whether the campaign has successfully reached the target audience. However, relying solely on memory to gauge awareness can present challenges, as recall may not be a true measure of the campaign’s effect on attitudes or behaviors. Memory is limited in its predictive power, and factors such as time lapse or the audience’s natural memory limitations may affect results. Furthermore, an individual’s ability to recall the campaign does not guarantee that it will influence their long-term attitudes or behaviors.
References
10. Alonso, F., Faus, M., Fernández, C., and Useche, S.A. (2021). “Where Have I Heard It?” Assessing the Recall of Traffic Safety Campaigns in the Dominican Republic. Energies, 14(18). Available from: https://www.mdpi.com/1996-1073/14/18/5792
11. Solomon, M.G., Mark, G., Hedlund, J.H., Haire, E.R., Chaffe, R.H.B., and Cosgrove, L.A. (2008). The 2006 National Labor Day Impaired Driving Enforcement Crackdown: Drunk Driving. Over the Limit. Under Arrest (Report No. DOT-HS-811-039). Washington, D.C.: National Highway Traffic Safety Administration. Available from: https://rosap.ntl.bts.gov/view/dot/1857
12. Agent, K.R., Green, E.R., and Langley, R.E. (2002). Evaluation of Kentucky’s ‘You Drink and Drive. You Lose” Campaign (Report No.: KTC-02-28/KSPI-02-2F). Lexington, Kentucky: Kentucky Transportation Center. Available from: https://uknowledge.uky.edu/cgi/viewcontent.cgi?article=1248andcontext=ktc_researchreports
Campaign Recognition
The purpose of measuring campaign recognition is to evaluate whether the campaign has left a lasting impression on the target audience, which can indicate deeper engagement with the messages. Data collection methods are like those for campaign awareness, with the distinction that respondents are shown campaign stimuli or materials to prompt recognition (aided recall). The resources required include the same survey and interview guides, data collection staff, and database management for data analysis.
A strength of measuring campaign recognition is that it provides an opportunity to compare recognition rates across various elements of the campaign, such as specific images, themes, or messages. This data can help campaign developers assess which components resonate most with the audience and which may need adjustment. However, a limitation is that recognition alone may not fully reflect the campaign’s impact on audience attitudes or behaviors. Recognizing a campaign message does not guarantee an emotional connection or the desire to change behavior, as it merely reflects familiarity with the content rather than commitment to the ideas it presents.
References
10. Alonso et al. (2021).
12. Agent et al. (2002).
Media Metrics
Units of Messaging/Frequency
The objective of tracking the units of messaging and frequency is to measure the volume and regularity with which campaign content is published across various platforms. This tracking provides insight into the visibility of the campaign’s messages, helping evaluators understand the breadth of content distribution. Data collection methods include tracking social media posts on platforms like Facebook, Twitter, or Instagram and monitoring ad buys across traditional media outlets such as radio, TV, and newspapers. Social media management tools, like Hootsuite or CrowdTangle, enable comprehensive tracking by monitoring posts and interactions under
campaign-specific hashtags. In traditional media, tracking may rely on reports from media outlets or third-party tracking services. Necessary resources include these social media and traditional media tracking tools and, if applicable, contracts or agreements with media agencies for ad tracking.
A significant strength of measuring messaging frequency is that it allows campaign managers to verify that content is being disseminated as planned and to adjust based on this data. Tracking frequency is also relatively simple and can be established early on, providing a foundation for ongoing monitoring. However, this metric only captures content distribution and does not provide insight into the number of people who saw or engaged with the campaign messages. Furthermore, the accuracy of these metrics depends heavily on the tracking method and may vary by platform, meaning the data might not accurately reflect audience reach or engagement.
References
13. Atkin, C.K., Freimuth, V., Valente, T.W., and Patchareeya, P.K. (2013). Public Communication Campaigns (Chapter 4 and 6). In: Public Communication Campaigns (4th ed.), SAGE Publications, p. 53–68; 83–98. Available from: https://books.google.com/books?hl=enandlr=andid=B7J1AwAAQBAJandoi=fndandpg=PA53anddq=(Traffic+safety+campaigns+evaluation+OR+Road+safety+campaign+evaluation+AND+Effectiveness+AND+Messaging+AND+Education)andots=CxHe3LY3QDandsig=yGR08yAqBuRBzUhRN0SZ4CWNhUk#v=onepageandqandf=false
14. Akbari, M., Lankarani, K.B., Tabrizi, R., Heydari, S.T., Vali, M., Motevalian, S.A., et al. (2021). The effectiveness of mass media campaigns in increasing the use of seat belts: A systematic review. Traffic Injury Prevention, 22(7), 495–500.
15. Violano, P., Roney, L., and Bechtel, K. (2015). The incidence of pedestrian distraction at urban intersections after implementation of a Streets Smarts campaign. Injury Epidemiology, 2(1), 18.
16. Vision Zero Network. (2016). Communications Strategies to Advance Vision Zero. Available from: https://visionzeronetwork.org/wp-content/uploads/2016/07/vzn-case-studycommunications-strategies.pdf
17. Montana Department of Transportation. (2022). Montana Annual Report for Federal Fiscal Year 2022. State Highway Traffic Safety Section. Available from: https://www.mdt.mt.gov/publications/docs/brochures/safety/hsp-report.pdf
Earned Media
The objective of tracking earned media is to measure the amount of unpaid publicity or media coverage generated by the campaign. Earned media reflects the appeal of the campaign messages, showing which aspects resonate enough to encourage media outlets or influencers to share the content without compensation. Data collection methods for earned media are like those used in tracking units of messaging and frequency. Social media tools can track campaign-related posts, but manual tracking may be necessary for traditional media or for collecting posts that do not explicitly use campaign hashtags. Resources include digital and traditional media
tracking tools, and in some cases, campaign managers may need to contact media sources directly to gather data on unpaid coverage.
One of the key strengths of tracking earned media is that it highlights the campaign’s reach beyond paid efforts, potentially expanding its visibility without additional costs. Earned media coverage also indicates the relevance and appeal of campaign messaging, as media sources and influencers choose to share the content based on its perceived value to their audiences. However, earned media is difficult to track consistently, as mentions can vary widely in nature and coverage can be unpredictable. Counting earned mentions without contextual analysis may lead to incorrect assumptions about campaign effectiveness, as mentions may reflect both positive and negative coverage.
References
Website Metrics
The purpose of website metrics is to assess how many people visit the campaign’s website, how they interact with the content, and whether campaign messaging drives traffic to the site. Website metrics provide quantitative data that offers insights into user engagement and interaction with campaign materials. Data collection methods include tools like Google Analytics, which track visitor counts, page views, session duration, and other website interaction metrics. To determine the origin of website traffic, unique URLs can be generated with UTM codes, and link shorteners like Bitly may be used to track specific campaigns. Resources required include access to Google Analytics, Bitly, and UTM codes to enable precise tracking of visitor sources and behaviors.
A major strength of website metrics is that they provide a quantifiable measurement of user engagement, allowing evaluators to see how effectively the website attracts and maintains the audience’s attention. Tracking is relatively easy to set up at the start of the campaign, making it a manageable and consistent data source. However, website metrics provide limited context, as they do not reveal why visitors interacted in specific ways or what their perceptions were, information that might require additional qualitative research, such as surveys or focus groups. This limitation makes it challenging to fully understand user motivations or perceptions based solely on web interactions.
References
17. Montana Department of Transportation. (2022).
Social Media Engagement
Social media engagement measures user interactions with campaign content, such as likes, shares, comments, and clicks, to evaluate the campaign’s reach, awareness, and resonance with its target audience. Tracking engagement metrics offers real-time insights into the effectiveness
of campaign messages. Data collection methods include using built-in analytics tools on social media platforms or third-party analytics services, which gather data across multiple platforms. Necessary resources include access to analytics tools like Hootsuite or Buffer and, optionally, Google Analytics for a more holistic view of engagement.
A primary strength of social media engagement is that it reflects how well campaign content resonates with the audience, as high engagement rates often indicate positive reception and emotional impact. Real-time tracking enables campaign managers to adjust messaging based on audience reactions, creating opportunities for responsiveness and campaign optimization. However, social media engagement rates may vary due to external factors unrelated to the campaign, such as trending events or platform algorithm changes. These fluctuations make it difficult to attribute engagement rates solely to the campaign’s content. Additionally, engagement levels may depend on the platform’s unique characteristics, meaning high engagement on one platform might not indicate similar success across others.
References
18. Florida Department of Transportation. (2022). FY 2021 Highway Safety Plan Annual Report. Available from: https://fdotwww.blob.core.windows.net/sitefinity/docs/default-source/safety/3grants/hsp-annual-reports/fy2021-hsp-annual-report---final-12-29-21.pdf?sfvrsn=c908fb15_2
19. Triemstra, J.D. (2018). Correlations between Hospitals’ Social Media Presence and Reputation Score and Ranking: Cross-Sectional Analysis. Journal of Medical Internet Research. 20(11), e289.
20. Voorveld, H., van Noort, G., Muntinga, D., and Bronner, F. (2018). Engagement with Social Media and Social Media Advertising: The Differentiating Role of Platform Type. Journal of Advertising, 47(1), 38–54. https://doi.org/10.1080/00913367.2017.1405754.
Social Media Followers
The objective of tracking social media followers is to gauge the growth rate and size of the campaign’s audience on various platforms. This metric provides an indication of the campaign’s potential reach and capacity to expand its influence. Data collection methods include using built-in analytics tools that track follower counts in real-time, as well as third-party tools that allow data gathering across multiple platforms. Required resources are like those needed for tracking social media engagement, including access to platform analytics and social media management tools.
A strength of follower metrics is that they provide a clear, quantifiable measure of audience size, which can serve as a benchmark for evaluating reach over time or comparing with previous campaigns. Additionally, some tools offer follower demographics, which helps campaign managers tailor messages to resonate with specific segments of the audience. However, follower counts do not directly measure engagement or influence at an individual level, and they offer limited information on the audience’s attitudes or intentions. Follower numbers can also fluctuate due to platform algorithms, user activity, or trends unrelated to the campaign, complicating the attribution of follower growth to campaign effectiveness.
References
18. Florida Department of Transportation. (2022).
21. Klipfolio. (2023). Social Media Followers. Available from: https://www.klipfolio.com/resources/kpi-examples/social-media/social-followers
Reach
Reach is used to assess the estimated number of unique individuals who have seen or heard a campaign message or piece of content. By understanding reach, campaign managers can evaluate how far the message has spread and if it is accessible to the intended audience. Data collection methods are the same as those used for social media engagement, relying on built-in platform analytics that estimate the number of unique viewers or listeners. Resources needed are like those for engagement tracking, including access to social media and digital analytics tools.
A strength of reach metrics is that they offer a quantifiable measurement of campaign exposure, helping campaign managers understand the overall visibility of campaign messages. Reach data also allows tracking progress over time and comparing current performance with past campaigns. However, estimates of reach may vary in accuracy, depending on platform-specific algorithms. A high reach metric does not necessarily indicate message retention or impact, as simply seeing a message does not equate to engagement or behavioral change. Additionally, reach alone may not convey the audience’s depth of interaction with the campaign, which could require supplementary data on engagement or impressions.
References
23. Katz, H. (2022). The Media Handbook: A Complete Guide to Advertising Media Selection, Planning, Research, and Buying (8th ed.).
24. What Is Social Media Reach? Buffer: All-you-need social media toolkit for small businesses. (2023). Available from: https://buffer.com/social-media-terms/reach
25. How Do Radio Stations Know How Many Listeners They Have? (2023). Available from: https://radiofidelity.com/how-do-radio-stations-know-how-many-listeners-they-have/
Impressions
Impressions measure the number of times a campaign message has been presented to the audience, capturing the frequency of exposure across all viewers. This metric helps assess the visibility and saturation of campaign content. Data collection methods for impressions are the same as those for social media engagement, using built-in analytics tools to record content exposure metrics. Resources required are like those needed for tracking reach and engagement, including platform-specific analytics tools and third-party tracking services.
A significant strength of impressions is that they provide quantifiable data on how frequently content is displayed, offering insight into the extent of campaign visibility. Impressions data is useful for benchmarking campaign performance against previous efforts, allowing campaign managers to maintain consistency in their evaluations. However, impressions do not differentiate between unique viewers and repeated exposure, which can lead to inflated visibility metrics.
High impression counts do not necessarily correlate with meaningful engagement, as repeated exposures to the same individual do not guarantee an emotional impact or behavioral influence. This limitation makes it challenging to assess the campaign’s real influence based solely on impression data.
References
26. Farris, P., Bendle, N., Pfeifer, P., and Reibstein, D. (2010). Marketing Metrics: The Definitive Guide to Measuring Marketing Performance (2nd ed.).
Campaign Cost-Effectiveness and Efficiency
Cost-Effectiveness Analysis (CEA)
The objective of a CEA is to evaluate the financial cost of the campaign relative to the outcomes achieved, helping stakeholders understand the value of resources invested. CEA involves comparing the costs of the campaign’s components with quantifiable results, offering insight into the efficiency of campaign expenditures. Data collection methods for CEA include reviewing campaign budgets and completing detailed cost worksheets, which outline each expense associated with the campaign. Required resources include comprehensive expenditure data, outcome data (e.g., traffic enforcement records), and data analysis tools, along with trained data analysts who can interpret the results.
A strength of CEA is that it allows for meaningful comparisons across different programs or campaigns, even those implemented in various contexts or years. CEA enables stakeholders to understand what can be achieved within a given budget, offering a clear picture of the campaign’s return on investment and identifying potential areas for improvement in resource allocation. However, CEA has limitations, particularly when used as a standalone measure without incorporating a cost-benefit analysis. Excluding benefits from the analysis may lead to biased results, as it focuses solely on costs without considering broader societal benefits, such as improved safety or quality of life. This exclusion makes it challenging to judge the social desirability of one project over another, as it may not accurately capture the full impact of the campaign.
References
27. Directorate-General for Employment, Social Affairs and Inclusion (European Commission), and TARKI Social Research Institute. (2014). Study on Conditional Cash Transfers and Their Impact on Children: Final Report Volume 1. Available from: https://data.europa.eu/doi/10.2767/40642.
Cost-Benefit Analysis (CBA)
CBA aims to assess the financial costs of the campaign relative to anticipated improvements in safety outcomes, providing a more comprehensive understanding of the campaign’s value. CBA measures both the costs and benefits of the campaign, assigning monetary values to projected safety improvements, such as reduced accidents or fatalities. Data collection methods for CBA are like those for CEA, involving campaign budget reviews, cost worksheets, and collection of
outcome data. Required resources include expenditure and outcome data, data analysis tools, and a skilled analyst to evaluate and interpret results.
A key strength of CBA is its flexibility, allowing stakeholders to evaluate programs with multiple outcomes and compare their projected benefits against associated costs. CBA provides a decision-making framework for stakeholders, helping determine if a campaign’s benefits justify its investment and guiding resource allocation for maximum impact. However, CBA requires several assumptions about the monetary value of benefits, which can introduce bias and affect the accuracy of the analysis. Moreover, CBA may exclude certain benefits that are difficult to monetize, such as improved community well-being, even if these benefits hold significant social value. This exclusion can result in an incomplete assessment of campaign effectiveness and limits the ability to capture intangible benefits that might still be relevant for the campaign’s overall success.
References
27. Directorate-General for Employment, Social Affairs and Inclusion (European Commission), and TARKI Social Research Institute. (2014).
Secondary Quantitative Metrics
Speed Metrics
The objective of tracking speed metrics is to assess the average speed of drivers and the frequency of speeding over the posted limits. This data helps measure the impact of anti-speeding messages on driver behavior, serving as a direct indicator of the campaign’s effectiveness in influencing safer driving speeds. Data collection methods include the use of speed detection devices placed along targeted roadways in areas where the campaign is active. Additional data sources include law enforcement records on speeding citations, as well as surveys using Likert-style questions to capture self-reported speed behaviors. The resources needed for this analysis include technology for speed detection, such as speed radars, along with access to law enforcement citation records and trained personnel to collect survey data.
A strength of using speed metrics is that they provide specific, quantifiable data on speeding behaviors in defined locations, allowing campaign evaluators to track actual changes in driving speeds. This data can be monitored over time to assess trends and identify whether campaign messaging is having a sustained impact on driver behavior. However, one limitation is that it can be difficult to determine if drivers who alter their speeds are influenced directly by the campaign, as opposed to other factors like weather, traffic, or time of day. Establishing a control group can also be challenging, as various environmental or social factors can impact speed independently of campaign efforts. Additionally, self-reported data from surveys is prone to response bias, as individuals may answer based on what they think is expected rather than their actual behavior.
References
28. Van Schagen, I., Commandeur, J.J.F., Goldenbeld, C., and Stipdonk, H. (2016). Monitoring speed before and during a speed publicity campaign. Accident Analysis and Prevention, 97, 326–334.
Incidences of Alcohol-Impaired Driving
The goal of assessing incidences of alcohol-impaired driving is to measure the prevalence of this dangerous behavior in a specific population, helping to evaluate the impact of campaigns designed to reduce impaired driving. This metric offers insight into whether campaign messages about the dangers of drinking and driving resonate with the audience and influence behavior. Data collection methods for this metric include reviewing crash reports and law enforcement records for DUI citations, as well as collecting data from breathalyzers and blood alcohol concentration (BAC) tests conducted during sobriety checkpoints. Surveys with self-reported data on drinking and driving behaviors may also be used. The resources needed may include BAC testing kits, video surveillance technology, trained staff such as law enforcement and medical personnel, and statistical analysis software.
A strength of this approach is that it provides objective, quantifiable data, offering a reliable measure of how the campaign influences alcohol-impaired driving behaviors. This metric also provides a baseline of incident rates, making it possible to track changes in impaired driving behaviors over time. However, alcohol-impaired driving can be affected by numerous external factors that are unrelated to the campaign, such as social events, economic conditions, or shifts in law enforcement priorities, which may influence DUI rates. Moreover, collecting this data can be resource- and time-intensive, requiring extensive involvement from law enforcement and healthcare professionals. Self-reported data on impaired driving is also susceptible to response bias, as participants may underreport their drinking and driving behavior due to social desirability concerns. Additionally, some incidents may go unreported, potentially leading to an underestimation of the true incidence rate, and law enforcement personnel must be properly trained to ensure the data’s reliability.
References
12. Agent et al. (2002).
29. Piontkowski, S.R., Peabody, J.S. Reede, C., Velascosoltero, J., Tsatoke Jr., G., Shelhamer, T., and Hicks, K.R. (2015). Reducing Motor Vehicle-Related Injuries at an Arizona Indian Reservation: Ten Years of Application of Evidence-Based Strategies. Global Health: Science and Practice, 3(4), 619–629. https://doi.org/10.9745/GHSP-D-15-00249.
30. DeJong, W., and Hingson, R. (1998). Strategies to Reduce Driving Under the Influence of Alcohol. Annual Review of Public Health, 19(1), 359–378. https://doi.org/10.1146/annurev.publhealth.19.1.359.
31. Elder, R.W., Shults, R.A., Sleet, D.A., Nichols, J.L., Thompson, R.S., and Rajab, W. (2004). Effectiveness of mass media campaigns for reducing drinking and driving and alcohol-involved crashes: A systematic review. American Journal of Preventive Medicine, 27(1), 57–65.
Number of Crashes
The objective of measuring the number of traffic crashes is to evaluate the frequency of road incidents within a specific area, providing insights into overall traffic safety and the impact of road safety campaigns. Tracking crash data allows evaluators to determine if campaign messages related to safe driving practices are contributing to a decrease in crash incidents. Data collection methods for crash statistics include public safety agency records and crash reports, like those used for tracking traffic fatalities. Required resources for crash data collection include access to crash databases (e.g., National Highway Traffic Safety Administration’s Model Minimum Uniform Crash Criteria), statistical software, and trained analysts to interpret the data.
One strength of using crash data is that it provides an objective, quantifiable metric, which offers a reliable baseline for evaluating changes over time. Crash reports typically include essential details such as date, location, injury severity, vehicle types, and characteristics of individuals involved, enabling a comprehensive analysis of factors contributing to crash frequency. However, crashes are relatively rare events, which makes it challenging to detect significant changes within short time frames or smaller sample sizes. Crashes are also influenced by a variety of factors beyond the scope of campaign efforts, such as changes in laws, societal trends, or environmental conditions, complicating the process of isolating the campaign’s impact. Additionally, crash reporting criteria vary by state and jurisdiction, which can lead to inconsistencies in data collection and impact the overall reliability of the analysis.
References
18. Florida Department of Transportation. (2022).
29. Piontkowski et al. (2015).
31. Elder et al. (2004).
Traffic Fatalities
The purpose of measuring traffic fatalities is to assess the number of lives lost due to traffic incidents, providing one of the most critical and serious metrics for evaluating road safety. This data highlights the potential life-saving impact of road safety campaigns aimed at reducing dangerous behaviors. Data collection methods for traffic fatality data include accessing secondary data from public records, insurance records, and hospital statistics, which provide comprehensive fatality counts across various demographics. Resources required for this analysis include statistical software and data analysts trained in interpreting fatality statistics.
One of the main strengths of using traffic fatality data is its reliability and objectivity, as this data is collected through official channels and is less prone to bias. Additionally, fatality data provides a strong baseline for understanding the severity of road safety issues, making it a valuable metric for long-term evaluations. However, official fatality statistics may lack important contextual details, as they are often not collected with the specific purpose of evaluating campaign effectiveness. For example, fatality records may not include behavioral data, accident
causes, or other factors relevant to the campaign’s goals, limiting the ability to fully interpret the campaign’s impact.
References
29. Piontkowski et al. (2015).
Nonfatal Road Traffic Injuries
Tracking nonfatal road traffic injuries provides additional insights into the safety of road users by capturing data on injuries sustained in traffic incidents. This metric helps assess whether campaigns are effectively promoting safer behaviors that prevent injuries. Data collection methods include gathering data from hospital discharge records and hospital admission records, where trained operators extract specific details about injuries related to road traffic incidents. Necessary resources include trained personnel to review hospital data, hospital access permissions, and statistical analysis tools.
A major strength of using nonfatal injury data is that it provides an objective, quantifiable measure of traffic safety, adding depth to the overall evaluation of the campaign’s impact. Injury data offers a baseline for comparison, which allows evaluators to track changes in injury rates over time. However, not all injuries may be captured if the incident does not require hospitalization or if the data does not accurately reflect the full scope of nonfatal injuries. Injury data that is reported only by law enforcement may lack verification from healthcare sources, reducing its accuracy, and some accident reports may lack sufficient detail for effective analysis, limiting the ability to attribute changes directly to the campaign.
References
32. Zampetti, R., Messina, G., Quercioli, C., Vencia, F., Genco, L., Di Bartolomeo, L., and Nante, N. (2013). Nonfatal Road Traffic Injuries: Can Road Safety Campaigns Prevent Hazardous Behavior? An Italian Experience. Traffic Injury Prevention, 14(3), 261–266. https://doi.org/10.1080/15389588.2012.705189.
33. Staton, C., Vissoci, J., Gong, E., Toomey, N., Wafula, R., Abdelgadir, J., et al. (2016). Road Traffic Injury Prevention Initiatives: A Systematic Review and Meta-Summary of Effectiveness in Low- and Middle-Income Countries. PLOS ONE, 11(1), e0144971.
Behavioral Metrics
Risky Behaviors
The objective of measuring risky behaviors (e.g., distracted driving, risky cycling, speeding, drinking and driving, fatigued driving) is to gather data on behaviors that are associated with increased risks of injury or death, serving as a substitute for crash data in cases where crash data may be unavailable or insufficient. Tracking risky behaviors offers insight into whether the campaign’s messaging is successfully influencing safer practices among road users. Data collection methods for this metric include field observations, intercept surveys conducted before and after the campaign, and data on citations for risky behaviors provided by state or local
agencies. The resources required include field observers, intercept interviewers, and proper training for data collection staff.
A key strength of measuring risky behaviors is that intercept surveys tend to have higher response rates than mail surveys and may be more cost-effective than on-site observations, providing a rich data source at a relatively manageable cost. Additionally, state-level reporting from police officers offers verified data, ensuring objectivity in evaluating risky behaviors. Large-scale surveys that gather numerous observations help improve the representativeness and accuracy of the findings, giving evaluators a more accurate picture of the general road user population. However, intercept surveys and field observations can be time-consuming and resource-intensive, especially for campaigns targeting large geographic areas. Without a preintervention observational dataset, it may be challenging to make meaningful comparisons. Furthermore, intercept surveys tend to be more expensive and time-consuming to conduct than other options, such as mail or phone surveys. Self-reported data is also subject to response bias, as participants may respond in ways they believe are socially acceptable. Pre- and post-surveys introduce additional risks, such as selection bias or testing threats, which can limit the generalizability of the results.
References
18. Florida Department of Transportation. (2022).
34. Nathanail, E., and Adamos, G. (2013). Road safety communication campaigns: Research designs and behavioral modeling. Transportation Research Part F: Traffic Psychology and Behaviour, 18, 107–22.
Protective Behaviors
The goal of measuring protective behaviors (e.g., seat belt and car seat use) is to evaluate whether the campaign effectively promotes safety-enhancing practices, such as the use of seat belts and car seats. This data offers an understanding of how well the campaign’s messages resonate with individuals in adopting behaviors that reduce the risk of injury. Data collection methods include intercept questionnaires administered before and after the campaign and observational assessments at checkpoints where seat belt and car seat use can be directly observed. Resources required include data collection staff, training in observational techniques, and statistical analysis software for data processing.
A strength of intercept surveys in this context is that they yield higher response rates than mail surveys and can be more cost-effective than conducting on-site observations, allowing for flexible timing in data collection. Additionally, the checkpoint observation method provides a direct, objective measure of behavior, reducing reliance on self-reported data. However, like other intercept surveys, this method can be time-consuming and is susceptible to response bias, as participants may answer based on perceived expectations. Pre- and post-intervention surveys
also present risks of selection bias and testing threats, and law enforcement staff responsible for checkpoint observations need proper training to ensure data accuracy.
References
14. Akbari et al. (2021).
29. Piontkowski et al. (2015).
Other Evaluation Methods
Unconscious Feelings
The objective of measuring unconscious feelings is to gauge the target audience’s emotional responses to a road safety campaign at a subconscious level. This approach provides a unique layer of insight, as it captures spontaneous emotional reactions that may not be accessible through traditional self-reports. By analyzing unconscious feelings, campaigns can develop messaging that resonates more deeply with audiences, potentially evoking stronger responses and increasing message retention. Data collection methods for this approach include using facial recognition software, such as the GfK-EMO scan, that captures expressions associated with various emotions. Additionally, surveys or questionnaires may be used to gather supplementary data, helping to cross-reference emotional responses with self-reported attitudes. Required resources include facial recognition technology, skilled staff to operate the software, and experts who can accurately interpret the data, as understanding unconscious reactions often requires specialized training.
A key strength of this approach is its ability to capture emotional responses that individuals may not consciously report. This data can offer valuable insights into audience engagement with the campaign, complementing more traditional measures of attitudes and behavior. Unconscious feelings provide a fuller picture of how well the campaign resonates at an emotional level, which is especially valuable for campaigns aiming to create strong, memorable impressions. However, measuring unconscious feelings has several significant limitations. Techniques to capture these responses are complex, resource-intensive, and require specialized equipment, making them costly and less feasible for smaller campaigns. Furthermore, while facial recognition software can capture immediate responses, interpreting these expressions accurately remains challenging. Emotional cues often require validation through self-reports, and the data can be influenced by various factors, such as the individual’s context or surroundings during the measurement. Consequently, it may be challenging to attribute changes in unconscious feelings solely to the campaign, as opposed to external influences.
References
36. Cable, R. (2013). Key Focus Areas in Road Safety Communication Campaigns. Civil Engineering. Available from: https://www.idconline.com/technical_references/pdfs/civil_engineering/Key_focus.pdf
Data Collection Methods
Interviews
Individual interviews aim to engage participants in one-on-one discussions, allowing for an in-depth exploration of their experiences and perceptions of the campaign’s effectiveness. This approach is particularly useful for understanding nuanced responses, as interviewers can adapt questions in real-time to probe deeper into specific topics. The flexibility of interviews allows for the exploration of complex or sensitive issues that may not be addressed in surveys or group settings. Resources required for individual interviews include trained interviewers who can conduct semi-structured, structured, or unstructured interviews, data inputting tools for recording responses, and data analysis software to interpret findings.
A strength of interviews is the level of detail and personal insight they offer, as participants often feel more comfortable sharing sensitive information in a confidential, one-on-one setting. Interviews can be conducted virtually, which reduces logistical costs and allows for a broader geographic reach. The format also permits interviewers to ask follow-up questions and seek clarification, capturing richer, more nuanced data than is typically achievable through standardized surveys. However, individual interviews also have notable weaknesses. The data analysis process can be complex and time-consuming, requiring skilled analysts to organize and interpret the large volume of qualitative data. Additionally, interviewer bias is a potential risk, as the interviewer’s tone, wording, or reactions may inadvertently influence the participant’s responses. High-quality interviewers are essential for maintaining data integrity, and the process itself is labor-intensive, requiring significant commitment of time and resources.
References
37. Boulanger, A., Daniels, S., Divjak, M., Goncalves, I., Meng, A., Moan, I., … and Zabukovec, V. (2009). Evaluation tool for road safety campaigns.
39. Asian Pacific Partners for Empowerment and Leadership (APPEAL). (2001). Integrating Evaluation into Tobacco Programs for Asian American and Pacific Islander Communities. www.appealforcommunities.org
Surveys
Surveys are a versatile evaluation tool, designed to gather quantitative data on participants’ knowledge, attitudes, and behaviors following a campaign. Surveys are often used in both pre- and post-campaign assessments to measure changes over time, making them ideal for evaluating campaign outcomes. Surveys can be distributed widely and in various formats—online, by mail, or in person—enabling data collection from large samples that represent diverse populations. Resources required for administering surveys include a database for managing responses, data analysis tools, and potentially participant incentives to encourage completion. Larger sample sizes are also needed to ensure the data’s reliability, especially for detecting small changes in behavior or attitudes.
A key strength of surveys is their scalability and efficiency, as they can collect data from many people relatively quickly and at a lower cost than in-depth methods like interviews or focus
groups. Surveys are especially valuable for pre- and post-testing, which helps track changes in key metrics, such as knowledge or behavior, across different campaign phases. Anonymity can encourage honest responses, especially when sensitive topics are involved, and closed-ended questions allow for easier data analysis and interpretation. However, surveys also have limitations. They are often impersonal, as questions are typically standardized, which restricts the opportunity for follow-up questions and limits the depth of response. This structure may make it challenging to capture participants’ full perspectives, especially on complex topics. Additionally, surveys are prone to response bias, as participants may provide answers they believe are socially desirable rather than their true opinions. Moreover, completing pre- and post-surveys can be challenging, as participants may lose interest or fail to respond to follow-up surveys, which limits the completeness of longitudinal data.
References
39. APPEAL. (2001).
Focus Groups
Focus groups are group discussions conducted with a small set of participants to gather in-depth perceptions of the campaign’s effects. This method allows for dynamic exchanges of opinions and ideas, with participants sharing insights and responding to each other’s comments, often generating more detailed feedback than individual surveys or interviews. Focus groups are particularly effective for understanding the social context of behaviors and attitudes, as they reveal how group dynamics may influence individual responses. Resources for focus groups include incentives for participants, experienced facilitators to guide discussions, and facilities or virtual tools for hosting sessions.
The main strength of focus groups is their ability to provide richer, more nuanced data than surveys, as participants are often encouraged to elaborate on their views in response to group interactions. Facilitators can observe verbal and non-verbal cues and ask probing questions to explore ideas more deeply, capturing valuable insights that reflect the social nature of campaign reception. Focus groups are also effective for outcome evaluations, as they allow participants to share personal anecdotes and direct quotes that can illustrate the campaign’s impact. However, focus groups have notable limitations. They require skilled facilitators to manage discussions and ensure balanced participation, as some individuals may dominate the conversation, potentially limiting input from quieter participants. Analyzing qualitative data from focus groups is time-consuming and requires experienced analysts to interpret themes accurately. Additionally, participants may feel social pressure to conform to dominant views expressed within the group, which can introduce response bias and reduce the diversity of perspectives represented.
References
39. APPEAL. (2001).
Observation
Observation involves directly monitoring program activities, participant behaviors, or visual changes related to the campaign, providing immediate insights into the campaign’s operations and effects. Observational data can reveal how the intervention is implemented in practice and whether there are unexpected consequences or patterns in behavior that might not emerge through self-reports. Resources required for observation include trained staff who are skilled in observational techniques and supplies for recording and documenting data, whether through notes, photos, or video recordings.
A strength of observation is that it provides real-time, unfiltered data, capturing details that might be overlooked in other forms of data collection. Observational data can reveal unintended effects or behaviors and show behavioral trends over time, providing context that may enhance the interpretation of other evaluation metrics. Observations are particularly valuable for understanding the immediate impact of interventions in specific settings, as they can highlight environmental or situational factors that influence behavior. However, observation has notable limitations. It can be difficult to interpret observed behaviors accurately without additional context, as actions may be influenced by factors unrelated to the campaign. Observational data is also subject to observer bias, as researchers may influence or interpret actions based on their own expectations. Observations are time-intensive and may require repeat sessions to capture accurate trends, and physical limitations (e.g., restricted viewing angles or demographics) can impact the accuracy of findings. Additionally, observation alone may lack the depth needed to understand participants’ motivations, which may require supplementary qualitative data.
References
39. APPEAL. (2001).
Existing Statistics
The objective of using existing statistics is to leverage pre-collected data to determine campaign effectiveness based on recorded results. Existing statistics can be drawn from public records, government data, or other sources that track relevant metrics over time, offering an accessible source of data that requires minimal additional resources. Existing statistics are particularly useful for evaluating trends before, during, and after the campaign, as they provide a baseline for comparison. Resources required for using existing statistics are relatively low, typically involving data access permissions, basic software for data management, and minimal personnel.
A strength of using existing statistics is that it is an unobtrusive data collection method, as it relies on data that has already been gathered. This approach is flexible in timing, allowing evaluators to access and analyze data as needed without requiring participant recruitment or new data collection. Existing statistics also provide a readily available baseline for comparisons, making them useful for longitudinal evaluations. However, using existing statistics has limitations, as large datasets are typically required for reliable analysis, which may not always be available or feasible to obtain. The lack of control over original data collection methodologies
can also pose challenges, as the accuracy, reliability, and consistency of data from various sources may vary. This inconsistency may make it difficult to verify the data’s quality or ensure that it aligns with the evaluation’s specific needs, limiting the precision of conclusions drawn from the analysis.
References
Meta-Analysis
A meta-analysis is a systematic review of existing studies to assess the campaign’s effectiveness within the broader research context. This method aggregates data from multiple studies, offering a comprehensive perspective on the effectiveness of similar campaigns or interventions. Meta-analyses are particularly useful for identifying consistent trends, patterns, or effects across diverse studies, which can help validate the findings of individual campaign evaluations. Resources needed for meta-analysis include data management tools, access to academic databases, data analysis software, and a team of researchers who can systematically collect, analyze, and interpret results from various studies.
One of the main strengths of meta-analysis is its ability to generalize findings across different contexts, increasing the external validity of the results. By using a structured and consistent approach, meta-analyses ensure thoroughness in data collection and analysis, minimizing bias and maximizing the depth of insights gained from previous research. Meta-analyses can define clear inclusion and exclusion criteria, ensuring that only high-quality studies are included in the analysis. However, there are limitations to this approach. Meta-analyses are time-consuming and require significant expertise, as they involve extensive literature searches and data extraction. Publication bias is another concern, as studies with positive or significant results are more likely to be published, potentially skewing the findings. Additionally, heterogeneity in the included studies, such as variations in methodology, sample sizes, or measures, can make it challenging to draw consistent conclusions. The exclusion criteria may also inadvertently omit relevant studies, limiting the comprehensiveness of the analysis.
References
40. Hoekstra, T., and Wegman, F. (2011). Improving the effectiveness of road safety campaigns: Current and new practices. IATSS Research, 34(2), 80-86.
42. Rice, R.E., and Atkin, C.K. (2012). Theory and principles of public communication campaigns. Public communication campaigns, 3-19.
Literature Reviews
Literature reviews, including systematic reviews, Cochrane reviews, and narrative reviews, provide a structured assessment of previous research to evaluate campaign effectiveness. These reviews offer a comprehensive synthesis of the existing literature on similar campaigns, helping evaluators identify key insights, patterns, and gaps in knowledge. Literature reviews are adaptable, allowing for adjustments in scope depending on the evaluation’s objectives and
resources available. Resources for literature reviews include limited software for managing articles, access to academic databases, and staff time for conducting a thorough and systematic search of the literature.
The strength of literature reviews is their structure and consistency, as they provide a broad overview of relevant research and ensure a thorough examination of existing evidence. This process allows evaluators to define specific search criteria, maintaining consistency across sources, which increases the generalizability of individual studies. Literature reviews are also flexible in scope, allowing evaluators to adjust based on available resources or specific research objectives. However, reviews may be limited by the scope of their inclusion criteria, as overly narrow criteria may exclude relevant studies that could provide valuable insights. Publication bias is another challenge, as some studies may be underrepresented due to restricted access or publication biases. Additionally, the quality of available studies can vary widely, making it difficult to control methodological differences and ensure uniform data quality. Literature reviews may also be time-consuming, as they require extensive searching and data extraction, particularly when covering a broad range of sources.
References
38. Newnam, S., and Muir, C. (2015). Effectiveness of prevention-focused, workplace health and safety campaigns.
39. APPEAL. (2001).
41. Davis, S. M. (2011). How do you engage a community in a randomized clinical trial or a drug trial? In: Clinical and Translational Science Awards Consortium Community Engagement Key Function Committee Task Force on the Principles of Community Engagement in Principles of Community Engagement.
43. Robertson, R. D., and Pashley, C. (2015). Road safety campaigns. DesLibris.
44. Faus, M., Alonso, F., Fernández, C., and Useche, S.A. (2021). Are traffic announcements really effective? A systematic review of evaluations of crash-prevention communication campaigns. Safety, 7(4), 66.
Case Studies
Case studies are used to describe programs or experiences in depth, often combining multiple evaluation methods to provide a holistic view of the campaign. By examining one or more “cases” in detail, this approach can capture unique insights into the processes, outcomes, and contextual factors that shape the campaign’s impact. Case studies are especially useful for exploring complex or multifaceted interventions where traditional metrics may not fully capture the nuances involved. Resources needed for case studies include access to extensive software for data organization and analysis, long-term commitment from evaluation staff, and time to compile data from various sources.
A major strength of case studies is that they allow evaluators to examine both the process and outcome of a program in depth, offering a comprehensive and nuanced perspective. Case studies can capture personal experiences, direct quotes, and unique processes, highlighting factors that may contribute to the campaign’s success or challenges. This method is particularly powerful for
descriptive purposes, as it provides rich detail on how the program operates in practice. However, case studies are resource-intensive and time-consuming, making them challenging to implement for larger-scale or time-sensitive evaluations. Additionally, they may be difficult to organize and analyze, as the data from multiple sources can be complex and voluminous. The personal nature of case studies may also introduce bias, as the findings may not be generalizable beyond the specific cases examined, limiting the broader applicability of results.
References
3. APPEAL. (2001).
