Research

Publications

Publications

Two Questions About Critical-Thinking Tests in Higher Education
Change: The Magazine of Higher Learning (2014), Volume 46, Issue 2 – Roger Benjamin 

Critical-thinking skills are applicable over an array of academic disciplines and can be both improved by teaching and assessed. Standardized tests of such skills have the advantage of being reliable, valid, and comparable. Value-added models can be used to estimate the growth in learning between the freshman and senior years. There are significant inter- and intra-institutional variations in the effectiveness of efforts to develop critical-thinking skills.

Teach, Learn, Assess
McGill-Queen’s University Press (2013) – Roger Benjamin

Chapter 6 of "Measuring the Value of a Postsecondary Education" (pp. 123 – 140) discusses the rationale and strategy for assessment.

The Principles and Logic of Competency Testing in Higher Education
Sense Publishers (2013) – Roger Benjamin

The aim of this chapter in Modeling and Measuring Competencies in Higher Education is to argue that more attention should be devoted to student-learning assessment in higher education, using multiple types of instruments, some of which link directly to teaching and learning.

Performance Tasks and the Pedagogy of Broadway
Change Magazine (2012), Volume 44, Issue 5 – Marc Chun

Performance task development shares much in common with playwriting.  Important choices must be made about style, focus, integration, scope, timing, authenticity, relevance, and accessibility.

The Seven Red Herrings About Standardized Assessment in Higher Education
NILOA Paper # 15 (2012) – Roger Benjamin

This occasional paper outlines the role of standardized assessment in higher education by addressing familiar arguments against standardized assessments.

Selecting Value-Added Models for Postsecondary Institutional Assessment
Assessment & Evaluation in Higher Education (2012), Volume 37, Issue 6 – Jeffrey Steedle

The research described in this paper compares the statistical properties of various options for estimating institutional value-added scores and provides a rationale for the current CLA value-added model.

The New Limits of Education Policy Avoiding a Tragedy of the Commons
Edward Elgar Publishing (2012) 
 Roger Benjamin

This study provides an explanation as to why improvement of teaching and learning is not a high priority for the stakeholders involved.

Generalizability Theory in Assessment Contexts
In C. Secolsky & D. B. Denison (Eds.), Handbook on Measurement, Assessment, and Evaluation in Higher Education. (2011). – Noreen Webb, Richard Shavelson, Jeffrey Steedle

This chapter provides an introduction to Generalizability Theory and demonstrates its application to data from the CLA.

A Well-Rounded Education for a Flat World
College Outcomes Project (2011) – Richard Hersh, Matt Bundick, Richard Keeling, Corey Keyes, Amy Kurpius, Richard Shavelson, Daniel Silverman, Lynn Swaner

This paper provides the conceptual underpinnings of the project and efficacy outcomes while briefly sketching work on measurements and indicators.

What Do They Measure? Comparing Three Learning Outcomes Assessments
Change Magazine (2010) Volume 42, Issue 4 – Jeffrey Steedle, Heather Kugelmass, Alex Nemeth

This article provides CAE’s perspective on the results of the FIPSE Test Validity Study.

Taking Teaching to (Performance) Task: Linking Pedagogical and Assessment Practices
Change Magazine (2010) – Marc Chun

This article reviews CAE’s work to engage faculty in creating performance tasks for use in the classroom.

Measuring College Learning Responsibly: Accountability in a New Age
Stanford University Press  (2010) – Richard Shavelson

This innovative work explores how new demands for accountability and new technologies are changing the way student learning is assessed.

On the Foundations of Standardized Assessment of College Outcomes and Estimating Value Added
In K. Carey & M. Schneider (Eds.), Accountability in American Higher Education (2010) – Jeffrey Steedle

This chapter explores the history of and rationale for measuring outcomes in higher education using standardized tests.

The Collegiate Learning Assessment: Setting Standards for Performance at a College or University
RAND Corporation Technical Report (2009) – Chaitra Hardison, Anna Marie Vilamovska

This report describes the application of a technique for setting standards on the Collegiate Learning Assessment (CLA), a measure of critical-thinking value-added at higher education institutions.

Test Validity Study Report
FIPSE (2009) – Stephen Klein, Lydia Ou Liu, James Sconing, Roger Bolus, Brent Bridgeman, Heather Kugelmass, Alex Nemeth, Steve Robbins, Jeffrey Steedle

This study examines whether commonly used measures of college-level general educational outcomes provide comparable information about student learning.

The Case for Comparative Institutional Assessment of Higher Order Thinking Skills
Change Magazine (2008) – Roger Benjamin

This article asserts that comparative assessments are essential and practical: although we cannot measure all dimensions of learning, it is perfectly possible to comparatively assess the higher-order thinking skills that are etched in most collegiate mission statements.

Assessing School Effectiveness
Evaluation Review (2008) – Stephen Klein, David Freedman, Richard Shavelson, Roger Bolus

In this paper, we outline the methods used by the CLA to determine value added.

Recreating a Faculty Role in University Governance
Anker Publishing (2007) – Roger Benjamin

This chapter in the book Fixing the Fragmented University considers considers whether and how governance and faculty participation might be redefined in the public research university in a way that encourages more faculty participation.

The Collegiate Learning Assessment: Facts and Fantasies
Evaluation Review (2007) – Steve Klein, Roger Benjamin, Richard Shavelson, & Roger Bolus

This article describes the CLA's measures and what they do and do not assess, how dependably they measure what they claim to measure, and how CLA scores differ from those on other direct and indirect measures of college student learning.

Assessing Student Learning Responsibly: From History to an Audacious Proposal
Change Magazine (2007) – Richard Shavelson

This article suggests how we might approach the task of assessing the full range of collegiate learning.

A Brief History of Student Learning: How we Got Where We Are and a Proposal for Where to Go Next
AAC&U (2007) – Richard Shavelson

The author offers historical context to consider as institutions across the country continue to develop new methods of assessment in response to renewed calls for greater accountability and, more importantly, the urgent need to raise levels of student achievement.

An Approach to Measuring Cognitive Outcomes Across Higher Education
Research in Higher Education (2005) – Stephen Klein, Marc Chun, Laura Hamilton, George Kuh, Richard Shavelson

This paper reports the results of an effort to assess important aspects of student learning in higher education.

Characteristics of Hand and Machine-Assigned Scores to College Students’ Answers to Open-Ended Tasks

IMS Collection (2005) – Stephen Klein

This article appears in the book Probability and Statistics: Essays in Honor of David A. Freedman.

The Environment of American Higher Education: A Constellation of Change
The ANNALS of the American Academy of Political and Social Science  (2003) – Roger Benjamin

The American university is one of society’s key institutions, perhaps the lead institution available today to respond to changing societal imperatives. However, for the university to continue to play a leading role, it is important to match the functions of the institution with the societal imperatives presented by a changed environment.

Responding Responsibly to the Frenzy to Assess Learning in Higher Education
Change Magazine (2003) – Richard Shavelson, Leta Huang

The problem is not that assessment is bad; it’s the fact that it is currently being advanced as a policy “solution” in the absence of a coherent conceptual framework that would align assessments with the valued outcomes of higher education.

Looking Where the Light is Better: A Review of the Literature on Assessing Higher Quality Education
Peer Review (2002) – Marc Chun

If we take as our starting point that one of the central purposes of higher education is student learning, the obvious question arises: Are we indeed measuring what we should be measuring?