eXtension’s Critical Success Factors/Metrics #5

In this, and subsequent, eXtension UPDATE articles, we be focusing on each of the nine Critical Success Factors/Metrics developed in late 2011. While there are some metrics on which we will not be able to report at this time, we will fill these gaps in 2012 as we move forward. Note that in some cases, data will reflect periods prior to 2011.

Critical Success Factor 5: Communities of Practice provide educational content and opportunities in appropriate web-based venues from which users do benefit.

Metrics: Outcomes (knowledge gained, behavioral intentions, and behavior changes), impacts, especially economic benefits, where appropriate.
Explanation: This Critical Success Factor/Metric focuses on the outcomes and impacts that result from Communities of Practice utilizing appropriate web-based venues to engage users in appropriate educational ways.

Metrics data: Where Communities of Practice focus their efforts on educational events that engage their Communities of Interest, primarily through webinars, documented outcomes (i.e., increased learning/knowledge, intent to apply this knowledge, and recommending these events to others) are significant and compelling. In 2011, nine Communities of Practice documented and reported outcomes achieved with their clientele through webinars, an embedded survey link on the public website, a social media campaign, and learning lessons (another 10 Communities of Practice focused on process oriented evaluations). In addition, 11 Communities of Practice reported they conducted 169 webinars in 2011 with 8,722 people participating. Also, the Communities of Practice reported that 14,447 people watched recorded webinars. To provide documentation for this critical success factor, evaluation reports from 22 Communities of Practice are posted to the Community of Practice Evaluation Reports page (http://create.extension.org/node/2699). A summary of outcomes documented by Communities of Practice in 2011 are presented below.

Community of Practice Outcome Evaluations for 2011

eOrganic

The eOrganic Community of Practice conducted 39 webinars from 12/09 to 4/11. A total of 3,712 individuals participated in the webinars.

Immediate feedback surveys were administered to participants in 22 of the webinars. The response rate was 66%. Results from these surveys indicated that:
• 76% of respondents reported that the webinar significantly or moderately improved their understanding of the topics.
• 76% of respondents indicated that they intended to apply the knowledge gained a lot or somewhat.
• 85% of respondents said they would recommend the webinars to others.

A follow-up survey was sent to participants of eight of the webinars 6-12 months after the webinars took place. Results from these surveys indicated that:
• 55% to 82% reported that they had applied the knowledge gained from the webinars in their farming operations.
• eOrganic webinars influenced construction of high tunnels, disease management, farm financial management, and the ability of advisers to better inform their constituents on these subjects.

Plant Breeding Genomics

Two half-day workshops and webinars were conducted and were attended by 132 participants. In addition, five 60 minute webinars were conducted with 197 participants. Participants included professional plant breeders, students, and related professionals.

Evaluations for the two half-day workshops and webinars were completed by 50% of the participants, and indicated that:
• 77% reported gaining new knowledge that is applicable to their work
• 74% reported that they plan to apply what they had learned
• 79% reported that they would recommend these sessions to others
Evaluations for the five 60 minute webinars were completed by 70% of the participants, and indicated that:
• 81% reported that the webinar improved their knowledge of the topic
• 74% reported that they intended to apply the knowledge gained in the webinars to their work
• 87% reported that they would recommend the webinars to others

Food Safety/Foodborne Illness

An embedded link to an evaluation survey was posted on several Food Safety foodborne illness pages on the eXtension public website. Thirty-two responses were received to the survey from 12 states. Selected survey results revealed that:
• Over 60% of respondents reported that Learning Lessons, Ask an Expert, and FAQs were very useful
• Respondents reported an increase in knowledge on foodborne illness topics before and after viewing information on the site in the areas of hand washing, washing produce, using a food thermometer, cross contamination, and temperature control

DAIReXNET

Eight webinars were conducted in 2011. Evaluation results included:
• 77% (72 of 95) of survey respondents indicated that their knowledge on the topics in the webinars greatly or moderately improved
• 96% (91 of 95) of survey respondents agreed or somewhat agreed that they could apply the knowledge learned in their work
• 95% (89 of 94) of survey respondents agreed or somewhat agreed that they would recommend the webinars to others

Extension Wildfire Information Network

Two webinars were conducted on Fire Adapted Communities: A New Concept for Fire Management and Materials and Design Considerations for Building in Wildfire Prone Areas. Evaluation results included:

Fire Adapted Communities: A New Concept for Fire Management
• 58 attended the live webinar and 150 viewed a recording of the webinar
• 62% rated the webinar as excellent
• 92% improved their knowledge on the topic
• 100% indicated they were very likely or somewhat likely to use the information presented
• 100% indicated they would very likely or somewhat likely share what they learned in the webinar with others
Materials and Design Considerations for Building in Wildfire Prone Areas
• 69 attended the live webinar and 122 viewed a recording of the webinar
• 84% rated the webinar as excellent
• 94% improved their knowledge on the topic
• 100% indicated they were very likely or somewhat likely to use the information presented
• 100% indicated they would very likely or somewhat likely share what they learned in the webinar with others

Financial Security for All

Ninety-four Cooperative Extension educators, America Saves and Military Saves campaign coordinators and others sent messages via Facebook and Twitter for America Saves Week, February 20 – 27, 2011. Multiple methods were used to evaluate this social media campaign.

Forty-five individuals responded to a follower and friend survey. Results included:
• 89% indicated the messages were very helpful or helpful
• 48% visited one or more links in the messages
• 11% joined America Saves, 32% said they planned to, and 18% were already American Savers
• Respondents provided comments including: Keep up the good work. These tips are timely and beneficial, I appreciate the effort to help us help ourselves. The tweets made me think about the ways we are managing our money. I welcome any and all suggestions for increasing my financial well-being.
Thirty-six collaborators who sent the messages responded to an online collaborator survey. Results included:
• Twitter Klout scores (a measure of engagement) increased from 11.22 to 19.68
• Twitter PeerIndex scores (a measure of social media engagement) increased from 4.94 to 11.52

Imported Fire Ants

A webinar Solve Your Fire Ant Problems with Help From the Experts was conducted in April, 2011. Ninety-eight individuals participated. Evaluation results included:
• 81% were concerned about fire ants because of human health issues related to stings, 50% because of pet issues, 62% because of environmental impact, 38% because fire ant mounds detract from the landscape, and 42% because fire ant mounds cause problems with equipment
• 96% indicated that they planned to adoption the information they received in the webinar

Livestock and Poultry Environmental Learning Center

A database on webinars conducted is maintained that currently includes information from September 2006 through April 2011. Selected results include:
• 55 webinars have been conducted with 7,282 live participants
• Of the 1,839 webinar participants who have completed evaluations, 75% indicated that they had improved their knowledge on the topics presented from moderately to significantly
• 47% of the participants use the information for recommendations for producers, 21% for farm management decisions, and 19% in developing manure management plans or permits
• The average viewer influences the decisions of 135 producers per year

HorseQuest

Evaluations were received from 29 users of HorseQuest online learning lessons. Selected results included:
• Based on a rating scale of 1 = little or no knowledge and 6 = very knowledgeable, the pre-learning lesson average knowledge score for respondents was 4.14; the post learning lesson knowledge score was 5.34, indicating a gain in knowledge (1.2) after viewing the learning lesson
• Based on a rating scale of 1 = not useful and 6 = very useful, 96% of respondents indicated a 5 or 6 when asked how useful the information in the learning lesson was to them
• Based on a rating scale of 1 = not likely and 6 = very likely, 97% of respondents indicated a 5 or 6 when asked how likely they would be to recommend the learning lesson to others