One-quarter of graduating students score below adequate on measures of literacy, numeracy
One of the first major attempts to measure employment-related skills in university and college students on a large scale shows that students are experiencing some gains in literacy, numeracy and critical-thinking scores over the course of their undergraduate studies. Yet, one in four graduating students scored below adequate in measures of literacy or numeracy, and less than a third scored at superior levels, according to findings by the Higher Education Quality Council of Ontario (HEQCO).
HEQCO completed two large-scale trials involving more than 7,500 students at 20 Ontario universities and colleges to measure literacy, numeracy and critical-thinking skills in entering and graduating students. The results of the trials are contained in the report On Test: Skills, Summary of Findings from HEQCO’s Skills Assessment Pilot Studies.
The first trial, the Essential Adult Skills Initiative (EASI), began in 2016. It administered the Education and Skills Online (ESO), an assessment widely used by the Organization for Economic Co-operation and Development that measures literacy, numeracy and problem-solving abilities of adults using everyday scenarios. More than 4,600 first- and final-year students participated in the pilot.
The second trial, the Postsecondary and Workplace Skills (PAWS) project, used the HEIghten Critical Thinking assessment, a test that is designed to evaluate students’ ability to analyze evidence, understand implications and consequences, and develop valid arguments. More than 2,900 students at two institutions participated in the study, which was conducted by the Education Policy Research Initiative at the University of Ottawa in partnership with HEQCO.
Together, the two trials aim to get at the heart of the ongoing debate over the skills gap among postsecondary graduates. They also aim to encourage postsecondary institutions to teach, measure and credential skills that are highly sought by employers and the labour market rather than discipline-specific content alone.
Non-disciplinary skills are becoming increasingly important in today’s economic climate where many graduates will end up not working in their field of study, and where they can expect to hold several jobs over the span of their careers, argue the authors, Harvey P. Weingarten and Martin Hicks. Even graduates who stay with one employer may find their jobs rapidly evolving. “For these workers, non-disciplinary skills matter just as much and often more so than discipline-specific skills,” they write. “For mid-career workers whose jobs may disappear, the key to successful and speedy re-entry into the labour market is a strong foundation of transferable skills that will help them pivot into a new work environment,” they add.
The results of the EASI trial show that final-year students had somewhat higher scores in literacy and numeracy than their first-year counterparts, although there was considerable variation among programs. About 25% of participating students scored at ESO Levels 1 and 2, 45% scored at Level 3 — the minimum required for graduates to perform well in today’s work world — and 25% to 30% scored at Level 4/5, the highest level achievable.
The results of the PAWS trial indicated little difference between the test scores of incoming and graduating students in critical-thinking abilities, although it too showed considerable variation among programs. In the second phase of the study, currently underway, researchers will link students’ HEIghten test scores and academic performance with their income tax data to assess the impact that critical thinking has on labour market outcomes.
The trials conducted by HEQCO and its partners demonstrate that large-scale testing of employment-related skills across multiple institutions is feasible, Weingarten and Hicks conclude. Based on the findings, HEQCO recommends that such assessments be implemented across all institutions and involve all students, rather than just a sample, and that they be integrated into program requirements.
On Test: Skills, Summary of Findings from HEQCO’s Skills Assessment Pilot Studies is written by Harvey P. Weingarten and Martin Hicks, Higher Education Quality Council of Ontario.
Measuring Essential Skills of Postsecondary Students: Final Report of the Essential Adult Skills Initiative is written by Harvey P. Weingarten, Sarah Brumwell, Ken Chatoor and Lauren Hudak, Higher Education Quality Council of Ontario.
Measuring Critical-thinking Skills of Postsecondary Students is written by Ross Finnie, Michael Dubois, Dejan Pavlic, Eda Suleymanoglu (Bozkurt), Education Policy Research Initiative.