Methodological Challenges in International Comparative Post-Secondary Assessment Programs: Lessons Learned and the Road Ahead
Studies in Higher Education (2015), Volume 40, Issue 3 – Raffaela Wolf, Doris Zahner, Roger Benjamin
This article summarizes the lessons learned from the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) project with the goal of improving future multinational educational assessment studies. It identifies numerous methodological challenges that pertain to test design and development, translation, adaptation, student sampling, scoring, reporting, and the validity of score interpretation. The article specifically focuses on analyzing and discussing the significance of the different item types employed by AHELO – performance tasks and selected-response items – and issues with assessments composed of multiple item formats. The authors discuss case studies that further our understanding of the challenges of measurement equivalency and the feasibility of measuring higher order skills across countries for cross-national score comparisons.
Two Questions About Critical-Thinking Tests in Higher Education
Change: The Magazine of Higher Learning (2014), Volume 46, Issue 2 – Roger Benjamin
Critical-thinking skills are applicable over an array of academic disciplines and can be assessed and improved by teaching. Standardized tests of such skills have the advantage of being reliable, valid, and comparable. Value-added models can be used to estimate the growth in learning between freshman and senior years of college. There are significant inter- and intra-institutional variations in the effectiveness of efforts to develop critical-thinking skills.
Teach, Learn, Assess
McGill-Queen’s University Press (2013) – Roger Benjamin
Chapter 6 of "Measuring the Value of a Postsecondary Education" (pp. 123 – 140) discusses the rationale and strategy for assessment.
The Principles and Logic of Competency Testing in Higher Education
Sense Publishers (2013) – Roger Benjamin
The aim of this chapter in Modeling and Measuring Competencies in Higher Education is to argue that more attention should be devoted to the assessment of student learning in higher education, using multiple types of instruments, some of which link directly to teaching and learning.
Performance Tasks and the Pedagogy of Broadway
Change Magazine (2012), Volume 44, Issue 5 – Marc Chun
Performance task development shares much in common with playwriting. Important choices must be made about style, focus, integration, scope, timing, authenticity, relevance, and accessibility.
The Seven Red Herrings About Standardized Assessment in Higher Education
NILOA Paper # 15 (2012) – Roger Benjamin
This occasional paper outlines the role of standardized assessment in higher education by addressing familiar arguments against standardized assessments.
Selecting Value-Added Models for Postsecondary Institutional Assessment
Assessment & Evaluation in Higher Education (2012), Volume 37, Issue 6 – Jeffrey Steedle
The research described in this paper compares the statistical properties of various options for estimating institutional value-added scores and provides a rationale for the current CLA value-added model.
The New Limits of Education Policy Avoiding a Tragedy of the Commons
Edward Elgar Publishing (2012) – Roger Benjamin
This study provides an explanation as to why the improvement of teaching and learning is not a high priority for the stakeholders involved.
Generalizability Theory in Assessment Contexts
In C. Secolsky & D. B. Denison (Eds.), Handbook on Measurement, Assessment, and Evaluation in Higher Education. (2011). – Noreen Webb, Richard Shavelson, Jeffrey Steedle
This chapter provides an introduction to Generalizability Theory and demonstrates its application to data from the CLA.
A Well-Rounded Education for a Flat World
College Outcomes Project (2011) – Richard Hersh, Matt Bundick, Richard Keeling, Corey Keyes, Amy Kurpius, Richard Shavelson, Daniel Silverman, Lynn Swaner
This paper provides the conceptual underpinnings of the College Outcomes Project and the efficacy outcomes while briefly sketching out work on measurements and indicators.
What Do They Measure? Comparing Three Learning Outcomes Assessments
Change Magazine (2010) Volume 42, Issue 4 – Jeffrey Steedle, Heather Kugelmass, Alex Nemeth
This article provides CAE’s perspective on the results of the FIPSE Test Validity Study.
Taking Teaching to (Performance) Task: Linking Pedagogical and Assessment Practices
Change Magazine (2010) – Marc Chun
This article reviews CAE’s work to engage faculty in creating performance tasks for use in the classroom.
Measuring College Learning Responsibly: Accountability in a New Age
Stanford University Press (2010) – Richard Shavelson
This innovative work explores how new demands for accountability and new technologies are changing the way student learning is assessed.
On the Foundations of Standardized Assessment of College Outcomes and Estimating Value Added
In K. Carey & M. Schneider (Eds.), Accountability in American Higher Education (2010) – Jeffrey Steedle
This chapter explores the history of and the rationale for measuring outcomes in higher education using standardized tests.
The Collegiate Learning Assessment: Setting Standards for Performance at a College or University
RAND Corporation Technical Report (2009) – Chaitra Hardison, Anna Marie Vilamovska
This report describes the application of a technique for setting standards on the Collegiate Learning Assessment (CLA), a measure of the value-added to critical thinking at higher-education institutions.
Test Validity Study Report
FIPSE (2009) – Stephen Klein, Lydia Ou Liu, James Sconing, Roger Bolus, Brent Bridgeman, Heather Kugelmass, Alex Nemeth, Steve Robbins, Jeffrey Steedle
This study examines whether commonly used measures of college-level general education outcomes provide comparable information about student learning.
The Case for Comparative Institutional Assessment of Higher Order Thinking Skills
Change Magazine (2008) – Roger Benjamin
This article asserts that comparative assessments are essential and practical: although we cannot measure all dimensions of learning, it is perfectly possible to comparatively assess the higher-order thinking skills that are etched in most collegiate mission statements.
Assessing School Effectiveness
Evaluation Review (2008) – Stephen Klein, David Freedman, Richard Shavelson, Roger Bolus
In this paper, we outline the methods used by the CLA to determine value added.
Recreating a Faculty Role in University Governance
Anker Publishing (2007) – Roger Benjamin
This chapter in the book Fixing the Fragmented University considers whether and how governance and faculty participation might be redefined in the public research university in a way that encourages more faculty participation.
The Collegiate Learning Assessment: Facts and Fantasies
Evaluation Review (2007) – Steve Klein, Roger Benjamin, Richard Shavelson, & Roger Bolus
This article describes the CLA's measures and what they do and do not assess, how dependably they measure what they claim to measure, and how CLA scores differ from those on other direct and indirect measures of college student learning.
Assessing Student Learning Responsibly: From History to an Audacious Proposal
Change Magazine (2007) – Richard Shavelson
This article suggests how we might approach the task of assessing the full range of collegiate learning.
A Brief History of Student Learning: How We Got Where We Are and a Proposal for Where to Go Next
AAC&U (2007) – Richard Shavelson
The author offers historical context to consider as institutions across the country continue to develop new methods of assessment in response to renewed calls for greater accountability and, more importantly, the urgent need to raise levels of student achievement.
An Approach to Measuring Cognitive Outcomes Across Higher Education
Research in Higher Education (2005) – Stephen Klein, Marc Chun, Laura Hamilton, George Kuh, Richard Shavelson
This paper reports the results of an eﬀort to assess important aspects of student learning in higher education.
IMS Collection (2005) – Stephen Klein
This article appears in the book Probability and Statistics: Essays in Honor of David A. Freedman.
The Environment of American Higher Education: A Constellation of Change
The ANNALS of the American Academy of Political and Social Science (2003) – Roger Benjamin
The American university is one of society’s key institutions, perhaps the lead institution available today to respond to changing societal imperatives. However, for the university to continue to play a leading role, it is important to match the functions of the institution with the societal imperatives presented by a changed environment.
Responding Responsibly to the Frenzy to Assess Learning in Higher Education
Change Magazine (2003) – Richard Shavelson, Leta Huang
The problem is not that assessment is bad; it is that the assessment is currently being advanced as a policy “solution” in the absence of a coherent conceptual framework that would align assessments with the valued outcomes of higher education.
Looking Where the Light is Better: A Review of the Literature on Assessing Higher Quality Education
Peer Review (2002) – Marc Chun
If we take as our starting point that one of the central purposes of higher education is student learning, the obvious question arises: Are we indeed measuring what we should be measuring?