Quality Standards for VET Examinations

Send by emailPDF version

Abstract “Quality Standards for VET Examinations”

On behalf of the German Federal Institute for Vocational Training (BiBB, Bonn) Univation Institute for Evaluation, Cologne, Germany analysed quality issues of examinations with relevance for the field of vocational education and training (VET). The purpose of this project is to give a comprehensive overview so that procedures to establish standards in the field of innovative VET examinations have a profound basis to start from. There is a special interest in quality standards for ‘innovative’ examination procedures aside from paper-& -pencil tests and demonstrations of technical skills in the laboratory setting of the training/examination workshop. The inquiry is based on desk research, especially national and international literature review.
The project critically examines the terms „standards“, „criteria“ and „indicators“ as consistent definitions do not exist and especially criteria and indicators are often used in an inconsistent vague way. Referring to its etymological origins the three terms are defined:
„Standard“ relates to something good, or worth striving for. As in the context of quality management they express minimal requirements. Maximum standards on the other hand are more often found as guidelines for development in educational and human services settings.
“Criterion” relates to judgemental statements of something (more) good or (more) poor. Criteria are denominators of success or failure or serve as threshold points dividing a continuum of data into sectors, each of them expressing a distinct level of high or low worth.
“Indicator” is a neutral denominator indicating that something exists which cannot be observed directly. Indicator is inseparably connected with measurement and operationalization. Indicators can be used to make criteria as well as standards ‘observable’ or measurable.
Codified standard sets concerning (innovative) examinations could be directly found in the context of vocational education and training, e.g. quality standards of assessments in initial and continuing vocational education as well as of personnel certifications. Many interesting developments could be identified in Australia.
Further relevant quality standards could be identified in adjacent fields as assessment-centers, universities, and schools. Quality standards for assessment-centres concentrate on the training of assessors, concepts, and feasibility. In the context of universities quality standards for examinations are mostly vague. For accreditation purposes only questions but not standards have been formulated. For the evaluation in schools important standards have been developed around the four attributes: propriety, utility, feasibility and accuracy.
More than 400 often similar standards have been analysed and devised within the following categories: standards concerning persons, institutions, processes, and products. There are often contradictions and incompatibilities – also within the categories mentioned - and not all standards can be fulfilled by the same examination or examination type. Most of the standards are not related to special types of examinations like oral or written examinations. Many standards are embedded in the context of their (national) VET systems and therefore not easy to transfer to other systems or generalizable.
As only explicit standards were analysed within this project, future research projects should focus on implicit standards in relevant literature and textbooks as well.
It is planned to publish parts of the study within the discussion paper series of the German Federal Institute for Vocational Training (BiBB). 

Contact: Wolfgang Beywl