National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Paper Response

We’ve invited thought leaders to provide a response to Occasional Papers that NILOA crafts. We hope that these pieces will spark further conversations and actions that help advance the field. Please sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

Response to Equity and Assessment: Moving towards Culturally Responsive Assessment

Response to Occasional Paper No. 29 - Equity and Assessment: Moving towards Culturally Responsive Assessment

Sheri Williams
Frank Perrone
University of New Mexico

In their paper on equity and assessment, Montenegro and Jankowski (2017) provided compelling arguments for moving towards culturally responsive assessment practices. The paper sounded an urgent call to improve assessments of student learning on our increasingly pluralistic campuses. The authors argued persuasively that mainstream assessment practices need to change in order to more appropriately measure the achievement gains of culturally diverse students.

We concur that culturally congruent curriculum, pedagogy and assessments are necessary to provide the conditions for diverse students to feel a sense of belonging and persist in college. As reflective educators in a minority-serving institution, we have come to know first-hand that we must address biased assessments and create alternative forms of assessments that are more accessible to all students. Below we provide examples of how our experiences further support Montenegro and Jankowski’s recommendations. We conclude by offering guidance based on our own experiences as part of faculty movement towards intentionally designing culturally-appropriate assessments.

Taking the Arguments to Higher Education Faculty

We used Equity and Assessment: Moving towards Culturally Responsive Assessment as the premise for a presentation at a statewide higher education assessment conference (Williams & Perrone, 2018). Our presentation engaged around 40 diverse participants from the state’s institutions of higher education. Participants engaged in small and large-group discussion on the promises of culturally-relevant assessment practices. We shared how our assessment practices had advanced upon reflection of the themes in the paper.

At the conference, we invited workshop participants to share their own experiences in creating and using culturally-fair assessment practices. Most of the audience indicated they were already committed to honoring multiple ways of knowing and integrating diverse perspectives in the assessment of student learning objectives. Participants’ responses reaffirmed Montenegro and Jankowski’s assertions and recommendations, most notably:

  • culturally-biased assessments are particularly unfair to students who are not fluent in certain cultural customs and traditions, or to students who are not proficient in the English language;
  • implicit bias has the effect of excluding the experiences of linguistically, culturally, and socioeconomically diverse students;
  • biased assessments fail to realize the strengths and assets of culturally diverse students; and
  • rubrics can establish the quality standards for evaluation of student work, but the demonstration of the work can vary, thus providing more equitable opportunities for diverse students.

A Rationale for Linking Assessments to Students’ Cultures and Ways of Knowing

Montenegro and Jankowski’s recommendations resonated strongly with faculty and students in our minority-serving institutions in the nation’s only majority-minority state. Our own institution had received funding to create a post-masters certificate program to prepare cohorts of predominantly American Indians for leadership in Indigenous-serving schools. As primary investigators, we wanted to assess our program’s usefulness and relevance among the cohorts of aspiring rural and Indigenous leaders. With each new cohort, we found increasing rationale for and benefits from linking assessments to students’ cultures and ways of knowing.

An external evaluation of our certificate program by McREL International revealed that culturally-relevant assessment practices were central to students’ success. Students reported that the assignments and assessments helped them reconnect with Indigenous values and leverage these values into strengths. Several themes emerged from the evaluation and these further reinforce the principles set forward by Montenegro and Jankowski (2017). Our Indigenous principal preparation students noted remarkable benefits from: the use of multi-cultural assessment tools involving video, print, and visual artifacts; the choice to develop one’s own authentic scenarios with feedback from peers and mentors; and the extended learning opportunities to co-present with faculty at seminars and conferences. The McREL report clearly demonstrated the benefits and need for culturally-relevant assessment. The following student quote aptly summarized the value of assessing learning via honoring Indigenous ways of knowing:

This program has really made us reflect and dig deep within ourselves to realize and discover what our Indigenious values are. We grew up with them in our culture, we were taught them, but they were kind of tucked away because the way that Western education is kind of drilled into your mind as, "This way, this way, this way." So, by enrolling in this program, it really allowed me to realize that these [values] are my strengths, that this is where I'm grounded, this is where I'm from.

Making Assessment Practices Valid for Indigenous Students

Nelson-Barber and Trumbull (2017) have argued that making assessment valid for Indigenous students requires the “use of local wisdom, the recognition of culture, and active involvement of community” (p. 133). These actions contribute strongly to constructing appropriate assessments for culturally-diverse students. An example of the ways in which Indigenous students can demonstrate their academic and social learning comes from Davidson’s work on reclaiming Indigenous methodologies (2017). She explored the question, what are the experiences of western-educated students when Indigenous thought is made central to the manner in which students are assessed? Instead of writing a conventional paper, Davidson argued that decolonizing methodologies, involving collective processes like reflection, sharing, and celebration, can tell a powerful academic story.

Faculty can take the lead from these thought leaders to push the limits of what counts in the assessment of learning outcomes. We contend that instructors must be culture builders and boundary spanners who lead from a vision that fully respects cultural interests. The value of using local knowledge and culture in assessments has been shown in many works, including a longitudinal study conducted in rural Alaska. The investigators, composed of elders, local teachers, and university faculty, developed curricula that explicitly connected assessments to the knowledge of elders and the local culture (Lipka & Adams, 2004). Similarly, Murphy (2015) has argued that, in order to grow student “sense of self” identity and self-awareness, new perspectives on learning must be the platform for authentic assessments (p. 45). Others have argued that the “best evidence about student learning outcomes or proficiencies will be found in authentic student work. . . in field-based contexts such as practicums or service learning.” (Jankowski, Hutchings, Ewell, Kinzie, & Kuh et al., p. 51)

Some Implications for Assessors of Academic and Social Learning

We offer a few suggestions from our own work in designing culturally-appropriate assessments that document student learning in visual, written, and oral means:

  • Provide options for students to visualize their learning through exhibits, artistic performances, story boards, posters incorporating visual or symbolic illustrations, cultural artifacts, photo collages, and the like.
  • Scaffold the writing process. It can be intimidating for second language learners to put their thoughts in writing. Start with silent reflection. Group in pairs/triads for dialogue to promote listening and reflection before writing.
  • Create assignments that allow students to demonstrate their learning through the spoken word. Make space for cultural narratives. Honor language preservation and encourage students to introduce themselves in their home languages whenever they present.

Our experience informs us that assessment approaches must be rooted in the learning and problem-solving traditions of culturally-diverse students. This means that institutions of higher education will need to make strategic investments in developing faculty members’ assessment literacy. If institutions are to improve their capacity to appropriately assess academic and social learning, the work of Indigenous researchers and community members must be tapped as sources of important expertise who can help “affirm and protect Indigenous student identities” (Khalifa, Gooden, & Davis, 2016, p. 1282). As Giroux wrote so persuasively in 1997:

Teachers need to not only understand the curriculum they teach, but understand how experiences produced in the various domains and layers of everyday life give rise to the different “voices” students use to give meaning to their own worlds. . . without recognition of learner identities and the ways in which self-regulation, self-assessment, curiosity, cultural and social interpretations and priorities are understood and mediated in the classroom we may not be helping [students] to learn. (p. 110)

We conclude that faculty can make culturally-relevant assessments commonplace by minimizing assessment bias, providing explicit expectations that honor diverse ways of knowing, building in respect for the local culture, and engaging students in peer- and self-assessment for equitable assessment of academic and social learning goals.  

References

Davidson, C.E. (2018). A methodology of beauty. In R. Minthorn & H. Shotton (Eds.), Reclaiming Indigenous Research Methodologies in Higher Education (pp. 36-46). New Brunswick, NJ: Rutgers University Press.

Giroux, H. (1997). Pedagogy and politics of hope: Theory, culture and schooling. Oxford: Westview Press.

Jankowski, N., Hutchings, P., Ewell, P., Kinzie, J., & Kuh, G. (2013). The Degree Qualifications Profile: What it is and why we need it now. Change: The Magazine of Higher Learning45(6), 6-15.

Khalifa, M. A., Gooden, M. A., & Davis, J. E. (2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research86(4), 1272-1311.

Lipka, J., & Adams, B. (2004). Culturally based math education as a way to improve Alaska Native students' math performance. Appalachian Collaborative Center for Learning, Assessment, and Instruction in Mathematics (ACCLAIM). Working Paper Series published at Ohio University, Athens, Ohio by the ACCLAIM Research Initiative.

Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment (Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

Murphy, J. F. (2015). Notes on the profession. UCEA Monograph Series. Charlottesville, VA: University Council for Educational Administration. Available at http://www.ucea.org/member_journals/ucea-monographs-series/

Nelson-Barber, S., & Trumbull, E. (2007). Making assessment practices valid for Indigenous American students. Journal of American Indian Education, 46(3), 132-147.

Williams, S. & Perrone, F. (2018, February 22). Promoting culturally-rooted assessment practices. Presentation at the 2018 annual meeting of the New Mexico Higher Education Assessment and Retention Conference (NMHEAR). Albuquerque, NM: Hotel Albuquerque.

About the authors:

Sheri Williams is an Assistant Professor in Educational Leadership, University of New Mexico. Her scholarship focuses on transformational and culturally rooted leadership in tribal, rural, and urban schools and communities. She has served as Principal of four schools and as Superintendent of two rural school districts in Colorado and New Mexico.

Frank Perrone is an Assistant Professor of Educational Leadership at the University of New Mexico. His research focuses on school principal preparation programs and design along with the school leaders’ roles in teacher recruitment, burnout, professional development, and retention.



For additional responses to this paper please click below:

Video response by Dr. Eboni Zamani-Gallaher

Response by Dr. Thomas F. Nelson Laird & Dr. Allison BrckaLorenz

Response by Melissa Wright

Response by Pamela Felder

Response by Jan McArthur

Response by Jodi Fisler

Response by Ereka Williams

Response by Masahiro Arimoto and Ian Clark