National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Viewpoints

We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

 

Promoting an Improvement Culture

Claudia J. Stanny, Director of the Center for University Teaching, Learning, and Assessment, University of West Florida

 

An interesting theme percolated through discussions at the recent meeting of the Association for the Assessment of Learning in Higher Education (AALHE) in Salt Lake City: How can campuses move beyond systematic collection of assessment evidence toward building and sustaining a “culture of improvement” (Suskie, 2018)? Findings from an AALHE & Watermark survey, presented at the conference, highlighted the important role assessment should play in improving student learning and facilitating conversations among faculty and staff. Assessment experts sought concrete examples of institutions that have made good use of assessment evidence, reflected on the quality of student learning and academic programs, and revised curricula or adopted new approaches to teaching and learning, with the goal of improving student learning.

In its 2015 volume, Using Evidence of Student Learning to Improve Higher Education, the National Institute for Learning Outcomes Assessment argue that assessment can and does produce significant improvements in student learning. However, assessment leaders repeatedly lament the absence of evidence of “significant change” in academic program quality motivated by reflection on and use of assessment evidence (e.g., Banta, Jones, & Black, 2009; Suskie, 2015, 2108). Linking improved assessment practices to curriculum modifications and/or new approaches to instruction and then linking both to improved student learning will always be a challenge, in part because all three types of change are incremental (Fulcher, et al., 2017).

The NILOA report stands in stark contrast to the commentaries of faculty skeptics, who believe faculty and institutions devote significant time and effort to assessment without producing evidence of a meaningful impact on student learning (e.g., Worthen, 2018). Certainly, assessment can be done badly and simply consume resources. The compliance response is a strong temptation, especially when stakes are high. External demands can distort internal processes and create cumbersome “make-work” assessment tasks that produce little more than a check-off for a mandated process. Institutions may devote so much effort to collecting data and documenting assessment processes to meet external demands that they have few resources to devote to interpret their findings or reflect on what the findings mean for teaching, advising, and curriculum design.

Roscoe (2017) argues for an “improvement paradigm,” which “would place at the forefront collective conversations about curricula and instruction.” Indeed, meaningful faculty conversations are at the heart of successful initiatives that improve student learning. Finding times and places to ensure that these conversations occur is a challenge, especially in large and complex institutions. Faculty at all institutions face increased demands for scholarship, grant writing, service to the community, and learning and managing new technology tools for teaching and research. Nevertheless, institutions must find ways to implement assessment processes that pose meaningful questions about curriculum, courses, and learning; engage faculty in deep discussion; and motivate efforts to improve.

The AAHE Principles of Good Practice(Astin et al., 1992) continue to guide assessment practices that promote improved student learning:

  • Assessment efforts have a clear purpose. They pose questions about teaching and learning that faculty care about.
  • Effective assessment is ongoing, not episodic. The power of assessment is manifested in cumulative, incremental change. Collection of and reflection on assessment evidence should be an integral part of the normal ebb and flow of faculty work.
  • Effective assessment involves faculty across and within academic programs in meaningful discussions of the structure of courses and curriculum, assignments and teaching strategies that promote learning, and meaningful ways to assess learning and provide feedback to students.

The September issue of New Directions for Teaching and Learning, Assessment in Action: Evidence-Based Discussions about Teaching, Learning, and Curriculum, collects a dozen examples from a variety of institutions that have succeeded in facilitating campus discussions about the quality of teaching and learning, informed by assessment evidence. The examples illustrate how institutions leveraged key elements of the AAHE principles to create processes that facilitate improvement. These campuses create structures (offices, processes, supportive technology) that engage faculty across the institution, develop faculty expertise, and facilitate ongoing dialog and reflection among faculty across disciplines. Campus initiatives create opportunities for faculty to develop expertise and carve out time to interpret assessment findings, develop new courses, design new assignments, and discuss teaching initiatives that promise to improve the quality of student learning. Six chapters describe initiatives related to general education, inquiry skills, and student writing. Another six chapters describe campus efforts to engage faculty within specific academic programs in discussion about the interpretation of assessment findings and campus-wide discussions of assessment practices and use of evidence.

Must significant change occur only when a big initiative introduces large changes quickly? The culture today seems to be in love with disruptive innovation (e.g., Christensen & Eyring, 2011), which suggests that the only way to make significant change is to supplant the old with something completely different. If we define “curriculum change” only as a significant overhaul (e.g., of a general education program or the curriculum for an academic major), we will find scant evidence of change, much less evidence of change that produces dramatic improvement in student learning. Even in the best cases, successful initiatives to improve student learning take multiple years to plan, implement, and document impact (Fulcher, et al., 2017). Unless we pay attention, we may fail to notice incremental change. In this case, assessment is our best friend. When we systematically assess and track changes made to the curriculum and note associated changes in student performance, we can discover and document larger changes that take several years to accrue. Systematic assessment practices make “invisible” incremental change visible.

The examples presented in Assessment in Action represent initiatives founded on a commitment to incremental change. For example, Isabella and McGovern (chapter 10), describe a more than decade-long evolution of a writing program based on thoughtful reflection on evidence about the impact of courses and assignments on the quality of student writing. Such gradual transformations require intentionality and persistent leadership among faculty and administrators. Other chapters describe the role Centers for Teaching and Learning play, serving as sources for expertise, facilitating faculty discussions, and providing leadership to shepherd efforts to develop assignments, develop faculty skill with active learning strategies and assignments, and facilitate discussions about curriculum change within specific departments or across the general education curriculum .

Contributors to Assessment in Action describe campus assessment structures and processes that faculty experience as meaningful and beneficial for both curriculum development and advancement of assessment skill. Faculty can lose sight of the progress made when change occurs incrementally. Systematic assessment documents incremental change and makes this progress visible. This volume highlights the benefits created when assessment processes and findings are made visible. The chapters describe models that can be adapted for other institutional contexts. When institutions engage faculty in ongoing discussions of student learning, informed by assessment findings, they enable faculty to identify and celebrate the real progress they achieve through long-term change initiatives.

References

Astin, A.W., Banta, T.W., Cross, K.P., El-Khawas, E., Ewell, P.T., Hutchings, P., Marchese, T., McClenney, K., Mentkowski, M., Miller, M. and Moran, E. (1992). Principles of good practice for assessing student learning. AAHE Bulletin, 45(4).

Banta, T. W., Jones, E. A., & Black, K. E. (2009). Designing effective assessment: principles and profiles of good practice. San Francisco, CA: Jossey-Bass.

Christensen, C. M., & Eyring, H. J. (2011). The innovative university: Changing the DNA of higher education from the inside out. John Wiley & Sons.

Fulcher, K., H., Smith, K. L., Sanchez, E. R. H., Ames, A. J., & Meixner, C. (2017). Return of the pig: standards for learning improvement. Research & Practice in Assessment, 11, 10-40.

Isabella, M., & McGovern, H. (2018). Identity, values, and reflection: shaping (and being shaped) through assessment. New Directions for Teaching and Learning, 155, 89-96.

Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence of student learning to improve higher education. San Francisco, CA: Jossey-Bass.

Roscoe, D. D. (2017, Winter). Toward an improvement paradigm for academic quality. Liberal Education, 103 (1). https://www.aacu.org/liberaleducation/2017/winter/roscoe

Stanny, C. J. (Ed.). (2018). Assessment in action: Evidence-based discussions about teaching, learning, and assessment. New Directions for Teaching and Learning, 2018 (155). Hoboken, NJ: Wiley. https://onlinelibrary.wiley.com/toc/15360768/2018/2018/155

Suskie, L. (2015). Five dimensions of quality: a common sense guide to accreditation and accountability. San Francisco, CA: Jossey-Bass.

Suskie, L. (2018). Assessing student learning: a common sense guide. San Francisco, CA: Jossey-Bass.

Worthen, M. (2018, February 23). The misguided drive to measure ‘learning outcomes.’ [Opinion]. The New York Times. https://www.nytimes.com/2018/02/23/opinion/sunday/colleges-measure-learning-outcomes.html

 

Check out our past Viewpoints:

Promoting an Improvement Culture
Claudia J. Stanny

Rubrics: Lessons from the Field
Laura Massa & Margaret Kasimatis

In Appreciation of Clifford Adelman
Peter T. Ewell

Why Are We Assessing?
Higher Education Assessment Practitioners

The EEQ CERT: A New Way to Assure and Communicate Program Quality, Relevance, & Value
Melanie Booth

Bringing Student Voices to the Table: Collaborating with our Most Important Stakeholders
Ann E. Damiano

Responses to “The Misguided Drive to Measure ‘Learning Outcomes’”
Natasha Jankowski

Collaborating for Individual and Institutional Success: Libraries as Strategic Campus Partners
Jennifer Duncan, Kacy Lundstrom & Becky Thoms

Rethinking the Role of Work in Higher Education
David W. Marshall

Demand interoperability to dramatically improve the educational ecosystem
Jeff Grann

The Comprehensive Student Record at Dillard University
Demetrius Johnson

NILOA Remembers Assessment Pioneer Sister Joel Read of Alverno College
Peter Ewell, Pat Hutchings, & Russ Edgerton

The Neuroscience of Learning and Development: How can Evidence Legitimize Self-Reflection?
Marilee Bresciani Ludvik

Taking Stock of the Assessment Movement – Liberal Education, Winter, 2017
Peter Ewell, Pat Hutchings, Jillian Kinzie, George Kuh & Paul Lingenfelter

Eight Years On: Early—and Continuing—Lessons from the Tuning Project
Daniel J. McInerney

Real-time Student Assessment: Prioritizing Enrolled Students’ Equitable Progress toward Achieving a High-Quality Degree
Peggy Maki

Academic and Student Affairs Sides of the House: Can We Have an Open Concept Learning Design?
Darby Roberts

Just Assessment. Nothing More. Nothing Less.
Wayne Jacobson

Design for a Transparent and Engaging Assessment Website
Frederick Burrack and Chris Urban

Improvement Matters
Peter Felten

Working Together to Define and Measure Learning in the Disciplines
Amanda Cook, Richard Arum, and Josipa Roksa

The Simplicity of Cycles
Mary Catharine Lennon

Helping Faculty Use Assessment Data to Provide More Equitable Learning Experiences
Mary-Ann Winkelmes

Ignorance is Not Bliss: Implementation Fidelity and Learning Improvement
Sara J. Finney and Kristen L. Smith

Student Learning Outcomes Alignment through Academic and Student Affairs Partnerships
Susan Platt and Sharlene Sayegh

The Transformation of Higher Education in America: Understanding the Changing Landscape
Michael Bassis

Learning-Oriented Assessment in Practice
David Carless

Moving Beyond Anarchy to Build a New Field
Hamish Coats

The Tools of Intentional Colleges and Universities: The DQP, ELOs, and Tuning
Paul L. Gaston, Trustees Professor, Kent State University

Addressing Assessment Fatigue by Keeping the Focus on Learning
George Kuh and Pat Hutchings, NILOA

Evidence of Student Learning: What Counts and What Matters for Improvement
Pat Hutchings, Jillian Kinzie, and George D. Kuh, NILOA

Using Evidence to Make a Difference
Stan Ikenberry and George Kuh, NILOA

Assessment - More than Numbers
Sheri Barrett

Challenges and Opportunities in Assessing the Capstone Experience in Australia
Nicolette Lee

Making Assessment Count
Maggie Bailey

Some Thoughts on Assessing Intercultural Competence
Darla K. Deardorff

Catalyst for Learning: ePortfolio-Based Outcomes Assessment
Laura M. Gambino and Bret Eynon

The Interstate Passport: A New Framework for Transfer
Peter Quigley, Patricia Shea, and Robert Turner

College Ratings: What Lessons Can We Learn from Other Sectors?
Nicholas Hillman

Guidelines to Consider in Being Strategic about Assessment
Larry A. Braskamp and Mark E. Engberg

An "Uncommon" View of the Common Core
Paul L. Gaston

Involving Undergraduates in Assessment: Documenting Student Engagement in Flipped Classrooms
Adriana Signorini & Robert Oschner

The Surprisingly Useful Practice of Meta-Assessment
Keston H. Fulcher & Megan Rodgers Good

Student Invovlement in Assessment: A 3-Way Win
Josie Welsh

Internships: Fertile Ground for Cultivating Integrative Learning
Alan W. Grose

What if the VSA Morphed into the VST?
George Kuh

Where is Culture in Higher Education Assessment and Evaluation?
Nora Gannon-Slater, Stafford Hood, and Thomas Schwandt

Embedded Assessment and Evidence-Based Curriculum Mapping: The Promise of Learning Analytics
Jane M. Souza

The DQP and the Creation of the Transformative Education Program at St. Augustine University
St. Augustine University

Why Student Learning Outcomes Assessment is Key to the Future of MOOCs

Wallace Boston & Jennifer Stephens

Measuring Success in Internationalization: What are Students Learning?
Madeleine F. Green

Demonstrating How Career Services Contribute to Student Learning
Julia Panke Makela & Gail S. Rooney

The Culture Change Imperative for Learning Assessment
Richard H. Hersh & Richard P. Keeling

Comments on the Commentaries about "Seven Red Herrings"
Roger Benjamin

Ethics and Assessment: When the Test is Life Itself
Edward L. Queen

Discussing the Data, Making Meaning of the Results
Anne Goodsell Love

Faculty Concerns About Student Learning Outcomes Assessment
Janet Fontenot

What to Consider When Selecting an Assessment Management System
R. Stephen RiCharde

AAHE Principles of Good Practice: Aging Nicely A Letter from Pat Hutchings, Peter Ewell, and Trudy Banta

The State of Assessment of Learning Outcomes Eduardo M. Ochoa

What is Satisfactory Performance? Measuring Students and Measuring Programs with Rubrics
Patricia DeWitt

Being Confident about Results from Rubrics Thomas P. Judd, Charles Secolsky & Clayton Allen

What Assessment Personnel Need to Know About IRBs
Curtis R. Naser

How Assessment and Institutional Research Staff Can Help Faculty with Student Learning Outcomes Assessment
Laura Blasi

Why Assess Student Learning? What the Measuring Stick Series Revealed
Gloria F. Shenoy

Putting Myself to the Test
Ama Nyamekye

From Uniformity to Personalization: How to Get the Most Out of Assessment
Peter Stokes

Transparency Drives Learning at Rio Salado College
Vernon Smith

Navigating a Perfect Storm
Robert Connor

It is Time to Make our Academic Standards Clear
Paul E. Lingenfelter

In Search for Standard of Quality
Michael Bassis

Avoiding a Tragedy of the Commons in Postsecondary Education
Roger Benjamin