PAVE Training Module 2

Section outline

• Evaluation is a critical but often neglected component of communication efforts. It provides valuable information about what is working and what is not so you can adjust your strategy for future engagements. Refer to the goals of the campaign that you formulated at the planning stage. The more clearly your goals are set, the easier it will be for you to evaluate your success. 

  Checklist for evaluating your campaign based on the GAMMA+ Model:

  
  

  

  
  

  

  
  

  

  Photo source:  RAN Network. (February 2017). Ex Post Paper RAN C&N meeting: How to measure the impact of your online counter or alternative narrative campaign. https://home-affairs.ec.europa.eu/system/files/2017-07/201702_ran_how_to_measure_impact_of_online_campaign_en.pdf

  The monitoring and evaluation of your online counter narrative campaign will also be strongly based on the analytics identified on the platforms used during your campaign. Analytics can be used to determine how your message is (or is not) reaching and engaging with audiences. This includes analytics from social media, newsletters, websites or any other platforms used, as well as interviews with targeted audiences in a digital form (Survey Monkey, Google Forms) or in person (either formal, with a specific set of predetermined questions, or informal—seeking general feedback).

  
  

   

  
  

  Social media metrics can be drawn into two broad categories: awareness and engagement.

  Awareness is the total number of people who view your campaign content (whether videos, ads, posts, websites or other digital content), and who those people are. It is important to consider awareness metrics which help you understand the characteristics of your audience: how, when and where they were exposed to your campaign, and which parts of the campaign reached them. There are two types of awareness metrics. The first relates to the scale of the audience: how many people saw or interacted with your campaign in any way. The second describes the nature of the audience in demographic or characteristic terms, such as gender, location, age, device type and so on. 

  
  

  

  Photo source: The Difference Between Social Media Reach and Impressions. Reputation. https://reputation.com/resources/all-resources/reach-vs-impressions/?fbclid=IwAR2zb0gGY8Y_Z6mBs_66t7bq1MWWmjSvAlKJ_RcK6P4GZPG1vgy10ouKqR4.

  Impressions are the total number of times social media browsers have shown your content. Impressions are different from reach because it doesn't count people who click or engage with your content, just those who are exposed to it. Therefore, it is entirely possible that an impression can take place without an individual even noticing your content. For this reason, it’s important not to focus too much on impressions. Reach is the total number of people who were exposed to your ad or content. If 50 people saw your content, then that content’s reach was 50. However, unfortunately, if you have 50 followers, that does not mean all 50 will engage with your content. That’s because popular social media sites use complex algorithms that determine who to show content to and when.

  How are reach and impressions different? 

  To understand the difference between reach and impressions, consider impressions as the number of times your content is displayed to your users and reach the total number of users who have viewed your content.

  
  

  

  Photo Source: Reynolds, L. and Tuck, H. (2016). The Counter Narrative Monitoring and Evaluation Handbook. Institute for Strategic Dialogue. https://www.isdglobal.org/wp-content/uploads/2017/06/CN-Monitoring-and-Evaluation-Handbook.pdf.

  
  

  Engagement on the other hand, is defined as the volume and types of interaction between audience members, campaigners or campaign materials. Engagements can include everything from likes and shares to email responses and can be positive or negative. The number and nature of engagements can help campaigners understand their audience’s reactions to the campaign or its content. These metrics can be combined and analyzed to build a comprehensive picture of a counter or alternative narrative campaign's performance, and help campaigners understand the impact they are having. They provide an indication of the quality and frequency of interactions between audiences and a campaign. Combined with awareness metrics they contribute to an evaluator’s understanding of the impact of a campaign. Several types of engagement are conducive to qualitative evaluation, which can provide insight into how a campaign is received. Most obviously, comments, or other text-based reactions are such data. Other types of engagement, including likes, favorites or other sorts of responses (such as the emoji list available on Facebook reaction tabs) can also be analyzed to reveal how content is received. 

  Clicks are the number of times people have clicked on your ad or a link in your posts. This is a useful indicator of the number of individuals who have made a proactive decision to engage with your counter narrative content. Sustained engagements are interactions between campaigners and users that go on for an extended period of time. Frequently, sustained engagements take the form of conversations, for example, they may be conducted through comment feeds, direct messages or email. While sustained engagements consist of an exchange or exchanges of multiple messages, there is no formal definition of what constitutes a sustained engagement. While engagement may be viewed as more ‘substantive’ data, it does take a user longer to type a comment than like a post. Not all of your active viewers will want to take the time, or find the need, to live or comment on your post.

  Impact is a measurable change in the audience’s knowledge, attitudes or behavior that can be attributed to exposure to or engagement with counter or alternative narrative content. Awareness and engagement metrics, when properly analyzed, can be brought together to help evaluators understand the impact of their campaign. Additional indicators, such as evidence of offline action, or the qualitative evaluation of online comments, can contribute to the overall impact picture. The measurement of impact and ultimately the answer to the question of whether a campaign succeeded or failed will be defined by the goals and objectives set at the very start of the project. 

  • Select activity Match the terms.

    

    Match the terms. Quiz

    

Your Counter or Alternative Narrative Campaign is Launched - Now What?

Jump to... Main learning site Session 1: Objectives and Expected Results How Does the Internet Support and Create Opportunities for Extremist Narratives and Radicalization? Digital Recruitment in Today's Society What Tactics Do Violent Extremists Use to Radicalize Individuals? Who Do Violent Extremists Target in Radicalization and Recruitment Online? How and Why Do Youth Become Key Targets for Radicalization and Recruitment? Gendered Lens: How and Why Do Women Become Key Targets for Radicalization and Recruitment? How Does Religion Influence Online Recruitment and Radicalization? What About Policymakers? How do They Support Radicalization and Resilience Factors? Additional Resources for Session 1 Session 2: Objectives and Expected Results What Could An Extremist Narrative Look Like? How Can We Address Online Radicalization and Recruitment? What is Counter or Alternative Messaging and How Can it Be Used to Prevent Radicalization? Building a Strategic Campaign Guidelines for Creating an Effective Campaign Mapping Your Target Audience How is Our Target Audience Currently Interacting with Extremist Content? How to Craft Your Counter or Alternative Message Tactics to Disseminate Your Campaign Messages Your Counter or Alternative Narrative Campaign is Launched - Now What? Now It’s Your Turn to Make a Counter Narrative Campaign! Additional Resources for Session 2 Session 3: Objectives and Expected Results Being Safe When Using Online Spaces Safety Through An Inclusive Lens What if Someone I know is Talking to an Unsafe Person? Key Considerations for Building Multi-Stakeholder Collaboration and Trust to Prevent Online Radicalization and Recruitment Additional Resources for Session 3 Print Your Course Certificate

Now It’s Your Turn to Make a Counter Narrative Campaign!