Collaborative Program Evaluation: Successful Student Engagement Abroad

in Organizational Analysis and Professional Development Program Evaluation and Assessment on

Over a period of three years, CoreCollaborative International (CCI) worked with IFSA-Butler to evaluate the impact of IFSA’s “Exploring Community and Culture” program. Founded in 1988 on the campus of Butler University, the Institute for Study Abroad (IFSA) is an independent non-profit organization dedicated to providing students with high-quality and academically excellent learning environments in 19 countries. As an institution, IFSA commits itself to inclusive excellence, individualized learning, intercultural agility, and enduring impact.

At the heart of CCI’s project with IFSA was the question of whether IFSA’s Exploring Community and Culture program (C&C), which had been introduced in select study abroad sites, was effective in its goal of increasing participating students’ engagement with their host communities. IFSA had designed C&C as a voluntary, inclusive, and scaffolded opportunity that would provide students with additional context and touchpoints before, during, and after their abroad experience. Its goal was to help students to feel more fully connected to the communities in which they were living, to understand contemporary issues (social, political, economic) in a historical context, to develop a better understanding of their own cultural identities and world views, and to gain skills in analyzing and navigating cultural complexities – skills which students would carry with them to future endeavors.  

IFSA charged the CCI team in 2014 with assessing the actual impact of this supplemental academic program on student learning outcomes and student development. CCI worked with three semesters’ worth of study abroad students, along with faculty, staff, host families, and alumni, at three different international locations (London, Merida, and Valparaiso). Using a mixed methods approach, CCI employed both quantitative and qualitative measures including:

The evaluation was designed around a logic model – noting program inputs, activities, outputs, outcomes, and impacts – as a way of measuring whether the program was maintaining fidelity to the results IFSA had identified as important. In this case, C&C’s targeted outcomes were based around students’ sense of self and others, sense of belongingness, willingness to engage their host communities, and ability to transfer knowledge and skills from the experience abroad to future endeavors. 

At the end of the evaluation period, CCI delivered to IFSA-Butler a rich trove of data that was both cross-sectional and longitudinal, and both aggregate and parsed by site. Data included descriptive statistics on both C&C participants and non-participants; BEVI score gain comparisons; a thematic analysis of focus group results, including interpretation of differences between group responses; and a descriptive report of alumni perspectives on C&C participation.  A synthesis and interpretation of both quantitative and qualitative data were also provided in a full executive summary.  

IFSA-Butler used the program evaluation findings to renew focus on student readiness and differentiated learning, circling back to tighten up C&C’s learning outcomes and to refine training for staff and faculty, who benefited from CCI’s one-on-one support and professional development over the life of the project. CCI’s evaluation enabled the institution to strengthen its continuous feedback loop so that the Exploring Communities and Culture program goals aligned more fully with IFSA-Butler’s overall mission of intercultural learning. Findings were also shared with the wider field of international educators at conferences like WISE thus expanding significantly the sphere of learning.

The field of education abroad is increasingly reliant on outcomes assessment and program evaluation, as recent calls for more inclusive abroad opportunities, coupled with increased economic strain within higher education as a whole, have led to greater scrutiny of outcomes.  The tools and methodologies employed by CCI, through the IFSA project and others, are well suited to education abroad assessment. CCI’s one-on-one support and professional development for project staff is also impactful. One IFSA participant noted, for example, that CCI’s faculty and staff training around the project was “hugely impactful, and the ripple effect is still being felt years later! We covered so much and I was able to develop skills that I rely on in my daily work to this day.”  

Beyond education abroad, of course, program evaluation is essential for any institution seeking to determine whether its programming actually accomplishes what it sets out to accomplish, and is worth the institutional investment. This is the way that institutions can best support their learners as well as their bottom line. With intervention from CoreCollaborative International’s mixed-methods, stakeholder-driven approach to program evaluation, campuses, and programs can ensure that they are being responsible with their resources while delivering tangible effects to their students and to the wider communities of which they are a part. 

If you are interested in learning how program evaluation can benefit your organization, whether large or small, please reach out. Contact the CoreCollaborative International team at info@corecollaborative.com  to take the next steps.