An Evaluation Proposal: Earth Sentinel Conversations for Environmental Sustainability Challenges
(Note: This evaluation proposal was written in the first half of 2021 as part of a Qualitative Research course)
Contents
Research and Monitoring and Evaluation Design. 9
Questionnaire 1 — Training questionnaire. 17
Questionnaire 2 — Biannual Tracker Survey. 27
Introduction and Overview
Sustainable development is not just a buzz word. If it is to be attained, all concerned hands have to be fully on board and committed. Earth Sentinel Conversations (ESC) believes that providing resources that empower environmental conversation and sustainability enthusiasts, potential advocates, and innovators, practicing advocates, and innovators would prevent fatalism, discouragement, and ensure that on every scale humans are advancing towards environmental sustainability and sustainable development. This proposal presents the details of implementation and monitoring and evaluation plan for an environmental education training and support network intervention designed by Earth Sentinel Conversations (ESC). This pilot training would take place in three cities in Nigeria — Abuja, Lagos, Port Harcourt.
Education for Sustainable Development (ESD) is the “provision of education and learning opportunities to enhance learners’ abilities to effectively understand and participate in the pursuit of sustainable development” (Didham & Ofei Manu, 2013). According to the United Nations (UN) Tbilisi declaration of 1977, one of the goals of environmental education is “to create new patterns of behavior of individuals, groups, and society as a whole towards the environment” (Athman & Munroe, 2001). A healthy environment is critical for sustainable development as the interlinkages and overlaps between environmental sustainability and sustainable development are evident in the sustainable development goals and other development agenda. A healthy environment means less diseases, and disease outbreaks, more food security, and a higher quality of life (UNEP, 2016). Hence, “the knowledge, values, attitudes, commitment, and skills” gained in education for sustainable development interventions also acquired for environmental sustainability and vice versa.
Not only should environmental education lead to behavior is modification, environmental education should also develop action competence — the ability to act with reference to environmental concerns — such that individuals are able to take on environmental action that is “intentional” and “intended” (Jensen & Schnack, 2006). This action includes effective public engagement (Buchs, Hinton, & Smith, 2015). Effective public engagement can be a catalyst for an increased concern about climate change and motivate the uptake of environment friendly attitudes and actions (Buchs, Hinton, & Smith, 2015).
In implementing this environmental education training and support network intervention, Earth Sentinel Conversations (ESC) would utilize a peer to peer framework and put in place support and mentorship infrastructures post training. ESC believes that this strategy would enhance collaboration and resource-sharing and reduce the risk of burn out by environmental sustainability enthusiasts and practitioners trained. ESC is interested in raising environmental sustainability educators, organizers, advocates, and innovators; educators who clarify issues and inspire love and commitment to environment health; community organizers honoring and developing the leadership potential of everyday people, helping them identify problems and solutions, and supporting them as they take action to make those solutions a reality’ (Foster & Loiue, 2010); “policy champions” who can bring about changes in public policy (Devlin-Foltz & Molinaro, 2010); innovators who create solutions to systemic environmental management challenges without redistributing or creating more problems.
This proposal for Earth Sentinel Conversations’ (ESC) environmental education training and support network intervention focuses on understanding to what extent Earth Sentinel Conversations (ESC) empowerment activities provide sufficient effective support for 14–21 years old youth and young adults (YYA) in Nigeria interested in environmental sustainability advocacy and innovation efforts. We have focused on youth and young adults so as to “catch them young.” Youths and young adults nowadays have become very interested in participating and taking active roles on social change issues. These YYA would develop a deeper understanding of these issues as they mature, be models to their peers, implement programmes targeted towards children and raise future environmental sustainability responsive YYAs.
Therefore, Earth Sentinel Conversations’ (ESC) environmental education training and support network intervention would involve trainings with peer learning discussions and support network that includes a mentorship system, provision of advocacy resources, and information on innovation competitions and scaling environmental sustainability efforts. These resources made available to participants are ESC’s inputs and independent variables. All participants would have access to these resources. Our outcome of interest, the dependent variable, would be the number and level of engagement in environmental advocacy and innovation efforts. This would be measured by creating an index that combines responses from environmental advocacy and innovation variables.
Dependent variables
Independent variables
Leading and participating in Environmental advocacy and innovation efforts
Exposure to peer learning discussions, trainings, advocacy resources, innovation competitions, mentorship structure, and networks
The ESC training program will be a 3 day training that includes a lot of opportunities for peer learning, conversations, and teamwork. Subject to the availability of funding, the program can be extended to 5 days. The 3 and 5-day module will have the same training content, with the 5 day module having more opportunities for conversation and bonding activities among participants. Questionnaires would be administered to participants pre and post training. These surveys would measure perceptions of participants about environmental issues especially the role of various actors and the impact of learnings during the training on their knowledge, skill, and confidence levels. There will be series of post training surveys administered to participants bi-annually for the next 3 to 5 years post training. The post training surveys would monitor advocacy and innovation activities and evaluate the usability of mentorship and support resources. All surveys would be web based and self-administered.
Literature Review
Engendering pro-environment attitudes is a topic of interest to many community based organizations, non-governmental organizations and other development actors. These organizations employ a variety of approaches based on their objectives and target groups for their interventions. The approaches include spending time outdoors and in green spaces especially for younger aged children to increase an appreciation of nature and for older aged children who can understand concepts, an awareness of ecosystem benefits. The methods used also includes trainings and facilitating peer-to-peer conversations. Environmental education trainings and ways of evaluating behavior modification and action competence have been developed by various actors. These resources often serve as a good starting point and/or reference point for persons interested environmental education.
In 2004, the Israeli Ministry of Education embarked on a course of action aimed at integrating Education for Sustainability (EfS) into the school curriculum. An EfS course was designed and implemented as a pilot program to encourage science and technology teachers to implement EfS processes in their schools. The time commitment for the teachers was 352 hours over the course of two years; 140 hours — sixteen 7-hour workshops every 2 weeks (Year 1) and 112 hours — four 7-hour workshops every month (Year 2). Two years later open-ended questionnaires and intensive interviews were conducted with teachers who participated in the training. Of the 28 junior high school science and technology teachers who had participated in the pilot EfS course, 13 volunteered, two years after its termination to provide feedback. Internal variables (motivation, sense of responsibility, sense of capability, values, and beliefs) seemed to remain highly stable resulting in environmentally responsible action in areas that were under the teacher’s control. External barriers, for example the unavailability of convenient resources, or resistance on the part of family, impeded the effectiveness of the teachers in implementing learnings and as advocates. The evaluation report did not measure within-school activities of teachers or obtain feedback on student learnings, behaviors, or advocacy activities but suggested that the principles of the EfS course should be the basis of courses that could be implemented among school principals, all teachers, and students. The researchers also recommended that the “implementation of post course support would encourage overcoming the gap between willingness to act and actual practice” (Abramovich and Lora, 2015).
In contrast, Geo-Environmental Resource Association (GERAS), an organization in Cameroun launched an environmental education project targeted towards school children between the ages of nine and sixteen. The project took place from April to October 2015 and commenced with a one day event in April 2015 with 44 students from 5 schools. The objective of the project is to ensure that the school children trained form environmental clubs in their schools after the training. GERAS believes that young people have a critical role to play to shape their future in the world threatened by the increasing impacts of climate change. GERAS used a resource from Eco-Schools titled “Pupil Environmental Review Key Stage 1 and 2”. The training in April included a tree planting activity, students recital of an ambassador’s pledge , and awarding of Young Climate Change Ambassadors certificate of achievement. In early October, GERAS prepared and distributed questionnaires to participating students and teachers in beneficiary schools. The questionnaires targeted the pupils who participated in the training program and their accompanying teachers. By October 2015, two schools had set up an environmental club and there were 328 secondary beneficiaries from the 44 students trained (GERAS, 2016).
In 2013, an environmental education intervention was also conducted in Germany with 114 secondary students from 14–19 year old. This evaluation employed an experimental approach. This training was a one day event because one day programs match the German curriculum better. There was a control group of 37 students. Like the Cameroon example above, this educational intervention also had clear objectives. The research wanted to understand the effectiveness of a one day program in influencing students’ environmental attitudes and connectedness with nature and the effect students’ degree of connectedness with nature has on their environ mental attitudes. After a module’s completion, students discussed their learnings with their peers and teachers. The evaluation utilized resources already in use within the environmental education field. The Two Major Environmental Values (2-MEV) and the Inclusion of Nature in Self scale in a pre-, post- and retention test design. The pre-test was administered 1 week before, the post-test directly after and the retention test 4 to 6 weeks after program participation. The researchers concluded among other things that depending on the programme’s content, short-term EE programmes indeed may have influence on participants’ environmental attitudes (Sellmann & Bogner, 2012).
Time spent in nature also helps clarify ecological concepts and leads to the development of more favorable attitudes toward the ecosystem. This 1999 study focuses on Spanish secondary school students 14–16 years of age. Before further research was conducted, an exploratory study was carried out consisting of an initial diagnosis of the pupils’ ideas; fieldwork materials were prepared and materials for the study of a freshwater ecosystem was designed, along with evaluation instruments. The independent variable consisted of a field trip; the dependent variable was the ecological concepts learnt and their application to the assessment of an environmental problem. The evaluation combined qualitative and quantitative research methods (Manzanal, Barreiro & Jiménez, 1999).
These papers have shown the importance of environmental education and its impacts on behavior modification and action competence. Most of the evaluations combine qualitative and quantitative approaches. The studies also show the importance of addressing barriers to taking learnings to the real world and setting expectations that can be evaluated.
Research and Monitoring and Evaluation Design
This evaluation would employ a nonexperimental approach design using a pre and posttest survey and biannual tracker surveys. ESC is interested in measuring and tracking the impacts of our training and post training resources on program participants. The training would ideally be a three day event with proposed schedule as follows:
Day 1
Pre training survey followed by peer to peer conversations exploring the future of climate change using a Futures thinking approach, and identifying factors that will aid pro-environment actions and mitigate negative actions. The day ends with an exercise where participants write individual confessions and affirmations to ground and center themselves on their environmental sustainability action journey.
Day 2
An overview of the climate crisis followed by training and discussions focusing on real world success stories and factors that led to their success. Followed by, a playing devil’s advocate activity where participants react to climate change denialist comments with effective rebuttals. There would be coaching on how to develop effective rebuttals and methods to disagree without being disagreeable.
Day 3
Will focus on how to how to conduct advocacy among environmental health enthusiasts, children, other youths, and young adults. Participants will be presented with Earth Sentinel Conversations post training resources, support network, and how to access them. The cohort would also form a Slack or WhatsApp group where they can stay in touch. Post training survey will be completed followed by a mini send forth party and presentation of participation plaques.
The pre and post training questionnaire is available in the Appendix. The follow up biannual tracker survey would involve counts of advocacy and innovation activities participants engage in and an assessment of the ease of use and value of our post training resources and mentorship structure.
Research Methods
This pilot phase of Earth Sentinel Conversations would take place in three cities in Nigeria; Abuja, Lagos, and Port Harcourt. Between thirty to forty five participants would be recruited for the training in each of these locations. The researcher would create awareness about this training in senior secondary schools and tertiary institutions within the state. There will be a link for interested participants to answer a few questions about their interest in the program. Based on these responses, participants who have the most compelling responses would be selected. From this batch of participants, final participants would be randomly selected to ensure that there is a mix of gender and age groups, and an equal representation for students from senior secondary schools and tertiary institutions. Participants selected would be given an invitation and a timeframe by which they should indicate their availability for the training.
Many of the activities during the training will take place in groups. There will be some activities where all group members will be from secondary schools to prevent power imbalances. There will be other activities where group members will be a mix or pairs of secondary school students and students from tertiary institutions to observe how the two groups will communicate, interact, and share ideas.
As mentioned earlier, there will be a pre and post training survey and biannual follow up surveys. On the first day of the training after an icebreaker session with introductions, participants will be given tablet computers to complete the pre training survey. Using the same device for the survey, will also ensure that all participants have the same experience when completing the survey. There will be a facilitator for this session and other staff available to assist participants in completing the survey. After the survey is completed, participants return the devices. This will be the same process for the post survey. Use of personal laptops will not be allowed during the training to prevent distractions.
The link for the follow up surveys will be sent out to participants once every six months. The purpose of this survey is to monitor their environmental advocacy and innovation efforts and engagement with Earth Sentinel Conversations support network and resources. These surveys would provide data to evaluate the long term objectives of the program and identify any improvements needed in the structure and delivery of the program. For a period of 3 years, there will be 6 follow up surveys per participant. The researcher envisages that there will be some participants who may not be accessible or may not want to participate in the follow up surveys. Therefore, the researchers have trained 45 participants to make room for a thirty percent drop out rate, so as to have a minimum of at least 30 participants for each location (Abuja, Lagos, and Port Harcourt) at the end of the 3 year period.
Some attitude and opinion questions will be repeated in the pre and post training survey to measure any changes between the pre and post training responses. Among others, questions in the pre training survey include number of environmental advocacy programs participant participated in, in the last twelve months and in what capacity participants participated. This question will be asked in the biannual tracker survey with the time frame changed to the last 6 months (Additional notes are provided in the Appendix: Questionnaire).
Results
Majority of the questions are Likert scale closed ended questions, with options ranging from strongly agree to strongly disagree, ordinal data coded from 5 to 1. Some questions are ratio scale — age of participants and number of environmental advocacy programs participant has participated in. Other open ended questions include recommendations for program improvement and how participant envisages that they will utilize ESCs resources.
Firstly, data will be examined using frequency tables. This would help understand the shape of the distribution of responses for various responses and identify outliers. There will also be cross tabulations with demographics such as location and participant type as banners. Descriptive statistics, especially the mean, will be included in all frequency tables and cross tabs. The mode can be easily identified from the crosstabs. Range will be included for numeric response variables. The data will be further analyzed using inferential statistics — the paired t-test analysis and correlation statistic.
The paired sample t-test will be used to test the difference in a set of questions asked to the participants before and after the training. Since the participants are the same persons and the questions are written in the same exact same language, a dependent sample or paired sample t-test is used to calculate the differences in the mean scores of participants. These questions focus on assessing the content of our training in making participants feel less anxious about environmental sustainability challenges, how they view collaborations between individuals, their perceptions of their capacity to lead change efforts, and what the role of individuals, corporations, and government should be.
The hypothesis for the paired sample t-test is a nondirectional hypothesis. The null (Ho) hypothesis for the paired sample t-test is that the mean of pretest and post test scores for the variables are equal to each other. The alternate hypothesis (Ha) is that the mean of pretest and post test scores for the variables are different. For some of the questions in the list of questions for which the t-test is being conducted a directional hypothesis would also be tested. The expected results from the t-tests are that the null hypotheses can be rejected and the alternative hypotheses, can be accepted, as statistically significant (at the 0.05 level) mean differences between pre and posttest responses. This will allow conclusions to be made, such as: the content of our training made participants less anxious and believe more in the power of individual level collaborations.
Correlation analysis would also be conducted between the number of environmental advocacy programs participants participated in in the last 6 months (a ratio variable) and perception of increase in knowledge, confidence, and skills from ESC’s training and support resources. Three correlation analysis would be conducted. The correlation analysis will be more relevant in the follow up phase during the analysis of the biannual tracker survey where participants have had the chance to participate or lead environmental sustainability efforts.
Frequency tables and the scatter plot of these variables would be used to identify outliers. The hypothesis for the correlation is a directional hypothesis. The null (Ho) hypothesis is that there is no, or zero, relationship between the two variables. The alternate hypothesis (Ha) is that there is a positive relationship between the dependent and the independent variable. For the three correlation analysis conducted, we expect that there is sufficient evidence to reject the null hypotheses at the 0.05 confidence level.. This will allow conclusions to be made, such as: the content of our training increase participants knowledge, confidence, and skills leading them to participate in more environmental advocacy programs.
Discussion
The findings from the paired t-test will let us know how well our training performs in reducing anxiety about environmental sustainability challenges and helps them in seeing the power of collaboration and a more proactive approach by individuals, corporations, and governments. The findings from the correlation analysis would reveal the connectedness between involvement in environmental sustainability action and increase in skills, confidence, and knowledge as a result of utilizing ESC’s resources and support network. Knowing the impact of post training resources on skills, confidence, and knowledge will guide program designers to design post training resources and focus on the factors that have the highest correlations with the action they are galvanizing support for.
The contribution to the field of evaluation is that this research provides more information about utilization of post training resources. We should expect a positive relationship from our correlations, if not it means additional research is necessary to find out reasons why. We might check responses to other questions such as engagement with resources and level of anxiety about environmental concerns to find out why increase in knowledge, confidence, or skills does not lead to more advocacy programs by the cohort.
The main limitation of this study is that those recruited are people who already care about the environment. Therefore, their scores may not differ so significantly from the pre to the post test. However, it is important to recruit people who already care about the environment and are interested in participating in environmental sustainability efforts, so as to optimize returns from the investments in this empowerment program.
Conclusion
Education for Sustainable Development (ESD) would remain relevant as we face the challenge of balancing environmental sustainability and economic growth. The aftermath of the slowdown in economic activities as a result of the now declining covid outbreak may lead to corporations and governments planning to build back bigger without concerns for the environment. It is important to raise the next generation through approaches that encourage them to advocate for environmental sustainability among their peers, and in society, and have the confidence to lead innovations that are environmentally friendly despite all odds.
References
Astramovich, R. (2016). Program evaluation interest and skills of school counselors. Professional School Counseling, 20 (1), 54–64. Retrieved April 17, 2021, from https://www.jstor.org/stable/90014835
Blake,J.(1999).Overcomingthe‘value-actiongap’inenvironmentalpolicy:Tensions between national policy and local experience. Local Environment, 4, 257–278.
Buchs M., Hinton E. and Smith G. (2015, October). Environmental Values, Vol. 24, №5, pp. 621–640. White Horse Press. Retrieved from: https://www.jstor.org/stable/43695266
David Devlin-Foltz D. and Lisa Molinaro L. (2010, August) Champions and “Champion-ness”: Measuring Efforts to Create Champions for Policy Change. Continuous Progress Strategic Services. Center for Evaluation Innovation.
Didham R. and Paul Ofei-Manu P. (2013) A framework for the main factors and important leverage points in the implementation of education for sustainable development in the Asia-Pacific region. Institute for Global Environmental Strategies.
Foster C. & Louie J. (2010, March) Evaluating Community Organizing. Grassroots Action and Learning for Social Change series. Center for Evaluation Innovation
Geo-Environmental Resource Association (GERAS) (2016, February 20). A Project Report of the pilot phase of the Climate Change workshop for Young People that took place from April to October 2015 in Kumba, South West Region of Cameroon
Manzanal F., Barreiro R., and Jiménez C (1999, March) Relationship between ecology fieldwork and student attitudes toward environmental protection. Wiley online library. Volume 36, Issue 4.
Sellmann D. and Bogner F. (2013, September) Effects of a 1-day environmental education intervention on environmental attitudes and connectedness with nature. European Journal of Psychology of Education , September 2013, Vol. 28, №3, pp. 1077–1086
United Nations Environmental Program (UNEP). (2016, May 19). Healthy Environment, Healthy People. https://www.unep.org/news-and-stories/story/healthy-environment-healthy-people.
Appendix
Questionnaire 1 — Training questionnaire
Link to training questionnaire
Questionnaire 2 — Biannual Tracker Survey
ESC for ESC Post-Training Biannual Questionnaire
In addition to questions with notes in the pre and post training questionnaire, other monitoring and evaluation questions in the biannual survey will touch on the following.
