Institution Report Review Page

How Do Students in Your Institution Measure Up?

Interpret your students’ performance-based assessment scores using the tools and resources below.

Congratulations! Your institution has taken an important step to prepare your students for the future. Now that students have completed CAE’s performance-based assessment, let’s dig into the reports you received.

What you’ve learned so far:
You’ve received an overview report of your institution’s scores, along with a side-by-side comparison of how your students compare to students in CAE’s US norm sample. 

Take it a step further:
The scoring rubric and Mastery Level details below will allow you to evaluate the specific skills and subskills that would benefit from additional development, in order to better prepare your students for their futures. 

Ready to dig in?
Your institution report shows you a summary and average associated with student scores on the Performance Task and Selected Response Questions, alongside an average Mastery Level score based on students’ level of proficiency from the summary scores. Learn more detail about what the scores mean below.

Comparing Your Institution with National Averages
(US Norm Sample Details) 

In order to provide your institution with a benchmark, CAE utilizes a sample of approximately 120,000 students within the United States as a norming group for comparison. The group comprises participants that are demographically representative of students across the United States in terms of race/ethnicity, gender, primary home language, and socioeconomic proxies.

By comparing your institution’s results to our norming sample, you will gain a better understanding of where your institution stands, allowing you to make data-driven decisions that enhance the educational experience for your students.

What Your Institution Mastery Level Means

To learn more about your students can move to higher Mastery Levels, click on any of the categories below.

Students at the Emerging level can improve by:

  • Stating a claim or decision and supporting it with evidence from the information in the documents
  •  Refraining from using opinions when citing evidence 
  • Countering opposing positions
  • Understanding the purpose of data
  • Analyzing the data, but not necessarily perfectly
  • Organizing responses in a manner that reads logically 
  • Paying attention to language and vocabulary conventions

Students at the Developing level can improve by:

  • Stating a claim or decision and supporting it with evidence from the information in the documents
  •  Refraining from using opinions when citing evidence 
  • Countering opposing positions
  • Understanding the purpose of data
  • Analyzing the data, but not necessarily perfectly
  • Organizing responses in a manner that reads logically 
  • Following language and vocabulary conventions

Students at the Proficient level can improve by: 

  • Making a claim and supporting that claim using some evidence that is based upon the information provided in the documents
  • Understanding the data that are presented in the task and using and interpreting the data, although not perfectly
  • Presenting a basic argument, but may miss the opposing position(s) 
  • Writing simple sentences that generally follow language and vocabulary conventions

To become Accomplished, students can improve by: 

  • Better identifying the reliability of information
  • Synthesizing information across multiple documents
  • Identifying areas where data and arguments are conflicted
  • Making accurate predictions based upon data
  • Identifying information bias
  • Evaluating alternate conclusions

To become Advanced, students can improve by: 

  • Being much more thorough in their analyses of the information
  • Understanding the complexity of the issue or situation and generating a response that makes this evident
  • Acknowledging that there are valid points on the opposing side, but still being able to refute that position 
  • Identifying gaps in logic and reasoning 
  • Demonstrating command of the written language

How we defined the mastery levels:
CAE completed a standard setting study using industry best practice (AERA, APA, & NCME, 2014; Cizek & Bunch, 2007). A demographically and geographically diverse panel of judges who have expertise in students’ critical thinking and written communication skills was convened to determine performance standards for five levels of mastery: Emerging, Developing, Proficient, Accomplished, and Advanced. Results of the standard setting study are used for awarding the levels of mastery for students’ results.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. American Educational Research Association.

Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. SAGE Publications Ltd.

Understanding Average Scores in Each Skill Area

To learn more about student scores in each skill group, click on the overall institution score you received below.

Analysis and Problem Solving

Making a logical decision or conclusion (or taking a position) and supporting it by utilizing appropriate information (facts, ideas, computed values, or salient features) from the Document Library.

  • May state or imply a decision/ conclusion/ position.
  • Provides minimal analysis as support (e.g., briefly addresses only one idea from one document) or analysis is entirely inaccurate, illogical, unreliable, or unconnected to the decision/ conclusion/ position.

  • States or implies a decision/ conclusion/ position.
  • Provides analysis that incorporates a few supporting ideas, some of which is inaccurate, illogical, unreliable, or unconnected to the decision/ conclusion/ position.

  • States or implies a decision/ conclusion/ position.
  • Provides some valid support, but omits or misrepresents critical information, suggesting only superficial analysis and partial comprehension of the documents.
  • May not account for contradictory information (if applicable).

  • States an explicit decision/ conclusion/ position.
  • Provides valid support that addresses multiple pieces of relevant and credible information in a manner that demonstrates adequate analysis and comprehension of the documents; some information is omitted.
  • May attempt to address contradictory information or alternative decisions/ conclusions/ positions (if applicable).

  • Organizes response in a logically cohesive way that makes it fairly easy to follow the writer’s arguments.
  • Provides valid elaboration on facts or ideas related to each argument and cites sources of information.

  • Organizes response in a logically cohesive way that makes it very easy to follow the writer’s arguments.
  • Provides valid and comprehensive elaboration on facts or ideas related to each argument and clearly cites sources of information.

Writing Effectiveness

Constructing organized and logically cohesive arguments. Strengthening the writer’s position by providing elaboration on facts or ideas (e.g., explaining how evidence bears on the problem, providing examples, and emphasizing especially convincing evidence).

  • Does not develop convincing arguments; writing may be disorganized and confusing.
  • Does not provide elaboration on facts or ideas.

  • Provides limited, invalid, overstated, or very unclear arguments; may present information in a disorganized fashion or undermine own points.
  • Any elaboration on facts or ideas tends to be vague, irrelevant, inaccurate, or unreliable (e.g., based entirely on writer’s opinion); sources of information are often unclear.

  • Provides limited or somewhat unclear arguments; presents relevant information in each response, but that information is not woven into arguments.
  • Provides elaboration on facts or ideas a few times, some of which is valid; sources of information are sometimes unclear.

  • Organizes response in a way that makes the writer’s arguments and logic of those arguments apparent but not obvious.
  • Provides valid elaboration on facts or ideas several times and cites sources of information.

  • Organizes response in a logically cohesive way that makes it fairly easy to follow the writer’s arguments.
  • Provides valid elaboration on facts or ideas related to each argument and cites sources of information.

  • Organizes response in a logically cohesive way that makes it very easy to follow the writer’s arguments.
  • Provides valid and comprehensive elaboration on facts or ideas related to each argument and clearly cites sources of information.

Writing Mechanics

Demonstrating facility with the conventions of standard written English (agreement, tense, capitalization, punctuation, and spelling) and control of the English language, including syntax (sentence structure) and diction (word choice and usage).

  • Demonstrates minimal control of grammatical conventions, with many errors that either makes the response difficult to read or provides insufficient evidence to judge.
  • Writes sentences that are repetitive or incomplete, and some are difficult to understand.
  • Uses simple vocabulary, and some vocabulary is used inaccurately or in a way that makes meaning unclear.

  • Demonstrates poor control of grammatical conventions, with frequent minor errors and some severe errors.
  • Consistently writes sentences with similar structure and length, and some may be difficult to understand.
  • Uses simple vocabulary, and some vocabulary may be used inaccurately or in a way that makes meaning unclear.

  • Demonstrates fair control of grammatical conventions, with frequent minor errors.
  • Writes sentences that read naturally but tend to have similar structure and length.
  • Uses vocabulary that communicates ideas adequately but lacks variety.

  • Demonstrates good control of grammatical conventions, with few errors.
  • Writes well-constructed sentences with some varied structure and length.
  • Uses vocabulary that clearly communicates ideas but lacks variety.

  • Demonstrates very good control of grammatical conventions.
  • Consistently writes well constructed sentences with varied structure and length.
  • Uses varied and sometimes advanced vocabulary that effectively communicates ideas.

  • Demonstrates outstanding control of grammatical conventions.
  • Consistently writes well constructed, complex sentences with varied structure and length.
  • Displays adept use of vocabulary that is precise, advanced, and varied.

What the Research Says

 {Global Report} Does Higher Education Teach Students to Think Critically?

A study conducted between 2016 and 2021 used CAE’s performance-based Collegiate Learning Assessment (CLA+) to assess the proficiency of more than 120,000 participants across seven countries.

The research findings indicate:

  • It is possible to measure these skills using a valid and reliable performance-based assessment in an international context
  • Only 50.2% of students exiting higher education are proficient in these skills
  • There is only a 2.7 percentage point increase in the share of proficient students between entering and exiting students

“The results outlined in this report demonstrate the importance of assessing critical thinking skills and how such assessments can contribute to the higher education policy agenda at the national and international levels. Since proficiency in these essential skills predicts positive academic and post-university outcomes, higher education institutions should explicitly teach and assess these skills.”

—Doris Zahner, Ph.D., Report co-editor and CAE chief academic officer

Next Steps

 

To discuss your results with CAE experts, please email [email protected] to schedule a meeting.

CAE also offers professional development, critical thinking instruction, and custom assessments to help you improve your students’ essential academic and career skills of critical thinking, problem solving and written communications.