Program Evaluation Models in Technical and Vocational Education and Training: A Systematic Literature Review of Scopus-Indexed Studies (2019–2026)

Muhammad  'Ariqsyah; Ambiyar Ambiyar; Fahmi  Rizal

doi:10.56806/jh.v7i3.456

Authors

Muhammad 'Ariqsyah Universitas Negeri Padang
Ambiyar Ambiyar Universitas Negeri Padang
Fahmi Rizal Universitas Negeri Padang

DOI:

https://doi.org/10.56806/jh.v7i3.456

Keywords:

TVET, program evaluation, CIPP model, Kirkpatrick model, systematic literature review

Abstract

Program evaluation is a strategic instrument for quality assurance, accountability, and continuous improvement in Technical and Vocational Education and Training (TVET). Yet TVET evaluation practices remain fragmented across paradigms, models, indicators, and stakeholder configurations. This systematic literature review synthesizes Scopus-indexed empirical and conceptual studies published from 2019 to 2026 on TVET program evaluation. Following PRISMA 2020, 400 Scopus records retrieved through Publish or Perish were screened; 46 relevant records were identified and 33 core TVET studies were retained for thematic synthesis. Keyword co-occurrence mapping was conducted in VOSviewer at two occurrence thresholds. The findings show that high-visibility TVET literature is dominated by vocational education, systematic reviews, technology-enhanced learning, and employability, while standard program-evaluation models such as CIPP, Kirkpatrick, Stake, and Proves remain under-represented. Most studies emphasize reaction and learning outcomes, whereas behavior-level impact, results-level impact, context evaluation, and utilization of findings remain limited. The review recommends context-sensitive, mixed-method, longitudinal, and stakeholder-inclusive evaluation designs integrated with accreditation and SDG indicators.

Downloads

Download data is not yet available.

References

Abdurrahman, Parmin, & Muryanto, S. (2022). Evaluation on the automotive skill competency test through ‘discontinuity’ model and the competency test management of vocational education school in Central Java, Indonesia. Heliyon, 8(2). https://doi.org/10.1016/j.heliyon.2022.e08872

Acosta, J. L. B., Navarro, S. M. B., Gesa, R. F., & Kinshuk, K. (2019). Framework for designing motivational augmented reality applications in vocational education and training. Australasian Journal of Educational Technology, 35(3). https://doi.org/10.14742/ajet.4182

Alsalamah, A., & Callinan, C. (2021). Adaptation of Kirkpatrick’s Four-Level Model of Training Criteria to Evaluate Training Programmes for Head Teachers. Education Sciences, 11(3), 116. https://doi.org/10.3390/educsci11030116

Ana, A. (2020). Trends in Expert System Development: A Practicum Content Analysis in Vocational Education for Over Grow Pandemic Learning Problems. Indonesian Journal of Science and Technology, 5(2), 246–260. https://doi.org/10.17509/ijost.v5i2.24616

Antonietti, C., Cattaneo, A., & Amenduni, F. (2022). Can teachers’ digital competence influence technology acceptance in vocational education? Computers in Human Behavior, 132, 107266. https://doi.org/10.1016/j.chb.2022.107266

Central Bureau of Statistics of Indonesia. (2025). Condition of the Labor Force in Indonesia August 2025. https://www.bps.go.id/id/publication/2025/12/19/42a75ee61332755586fdfcdd/keadaan-angkatan-kerja-di-indonesia-agustus-2025.html

Beer, P., & Mulder, R. H. (2020). The Effects of Technological Developments on Work and Their Implications for Continuous Vocational Education and Training: A Systematic Review. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.00918

Billett, S. (2020). Perspectives on enhancing the standing of vocational education and the occupations it serves. Journal of Vocational Education & Training, 72(2), 161–169. https://doi.org/10.1080/13636820.2020.1749483

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Cattaneo, A. A. P., Antonietti, C., & Rauseo, M. (2022). How digitalised are vocational teachers? Assessing digital competence in vocational education and looking at its underlying factors. Computers & Education, 176, 104358. https://doi.org/10.1016/j.compedu.2021.104358

Chatterjee, S., Bhattacharjee, K. K., Tsai, C.-W., & Agrawal, A. K. (2021). Impact of peer influence and government support for successful adoption of technology for vocational education: A quantitative study using PLS-SEM technique. Quality & Quantity, 55(6), 2041–2064. https://doi.org/10.1007/s11135-021-01100-2

Chiang, F.-K., Shang, X., & Qiao, L. (2022). Augmented reality in vocational training: A systematic review of research and applications. Computers in Human Behavior, 129, 107125. https://doi.org/10.1016/j.chb.2021.107125

Choi, S. J., Jeong, J. C., & Kim, S. N. (2019). Impact of vocational education and training on adult skills and employment: An applied multilevel analysis. International Journal of Educational Development, 66, 129–138. https://doi.org/10.1016/j.ijedudev.2018.09.007

Dahalan, F., Alias, N., & Shaharom, M. S. N. (2024). Gamification and Game Based Learning for Vocational Education and Training: A Systematic Literature Review. Education and Information Technologies, 29(2), 1279–1317. https://doi.org/10.1007/s10639-022-11548-w

Eck, N. van, & Waltman, L. (2009). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3

Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No. 67. Medical Teacher, 34(5), e288–e299. https://doi.org/10.3109/0142159X.2012.668637

George, W. K., Silas, E. I., Pandey, D., Pandey, B. K., George, W. K., Silas, E. I., Pandey, D., & Pandey, B. K. (2024). Utilization of Industry 4.0 Technologies in Nigerian Technical and Vocational Education: A Conundrum for Educators [Chapter]. IGI Global Scientific Publishing. (utilization-of-industry-40-technologies-in-nigerian-technical-and-vocational-education). Https://Services.Igi-Global.Com/Resolvedoi/Resolve.Aspx?Doi=10.4018/978-1-6684-9962-7.Ch014. https://doi.org/10.4018/978-1-6684-9962-7.ch014

Indrawati, S. M., & Kuncoro, A. (2021). Improving Competitiveness Through Vocational and Higher Education: Indonesia’s Vision For Human Capital Development In 2019–2024. Bulletin of Indonesian Economic Studies, 57(1), 29–59. https://doi.org/10.1080/00074918.2021.1909692

Jabarullah, N. H., & Iqbal Hussain, H. (2019). The effectiveness of problem-based learning in technical and vocational education in Malaysia. Education + Training, 61(5), 552–567. https://doi.org/10.1108/ET-06-2018-0129

Jayalath, J., & Esichaikul, V. (2022). Gamification to Enhance Motivation and Engagement in Blended eLearning for Technical and Vocational Education and Training. Technology, Knowledge and Learning, 27(1), 91–118. https://doi.org/10.1007/s10758-020-09466-2

Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training evaluation. Association for Talent Development.

Korber, M., & Oesch, D. (2019). Vocational versus general education: Employment and earnings over the life course in Switzerland. Advances in Life Course Research, 40, 1–13. https://doi.org/10.1016/j.alcr.2019.03.003

Lavrentieva, O. O., Arkhypov, I. O., Kuchma, O. I., & Uchitel, A. D. (2020). Use of simulators together with virtual and augmented reality in the system of welders’ vocational training: Past, present, and future. Augmented Reality in Education?: Proceedings of the 2nd International Workshop (AREdu 2019), Kryvyi Rih, Ukraine, March 22, 2019, (2547), 201–216. http://ceur-ws.org/Vol-2547/paper15.pdf

Li, J., & Pilz, M. (2023). International transfer of vocational education and training: A literature review. Journal of Vocational Education & Training, 75(2), 185–218. https://doi.org/10.1080/13636820.2020.1847566

Ling, Y., Chung, S. J., & Wang, L. (2023). Research on the reform of management system of higher vocational education in China based on personality standard. Current Psychology, 42(2), 1225–1237. https://doi.org/10.1007/s12144-021-01480-6

Lund, H. B., & Karlsen, A. (2020). The importance of vocational education institutions in manufacturing regions: Adding content to a broad definition of regional innovation systems. Industry and Innovation, 27(6), 660–679. https://doi.org/10.1080/13662716.2019.1616534

McGrath, S., Ramsarup, P., Zeelen, J., Wedekind, V., Allais, S., Lotz-Sisitka, H., Monk, D., Openjuru, G., & Russon, J.-A. (2020). Vocational education and training for African development: A literature review. Journal of Vocational Education & Training, 72(4), 465–487. https://doi.org/10.1080/13636820.2019.1679969

McGrath, S., & Yamada, S. (2023). Skills for development and vocational education and training: Current and emergent trends. International Journal of Educational Development, 102, 102853. https://doi.org/10.1016/j.ijedudev.2023.102853

Mertens, D. M., Hall, J. N., & Wilson, A. T. (2025). Program evaluation theory and practice. Guilford Publications.

Mulyanti, B., Purnama, W., & Pawinanto, R. E. (2020). Distance Learning in Vocational High Schools during the COVID-19 Pandemic in West Java Province, Indonesia. Indonesian Journal of Science and Technology, 5(2), 271–282. https://doi.org/10.17509/ijost.v5i2.24640

Mutohhari, F., Sutiman, S., Nurtanto, M., Kholifah, N., & Samsudin, A. (2021). Difficulties in implementing 21st century skills competence in vocational education learning. International Journal of Evaluation and Research in Education (IJERE), 10(4), 1229–1236. https://doi.org/10.11591/ijere.v10i4.22028

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

Patton, M. Q. (2008). Utilization-focused evaluation. Sage publications.

Radosavljevic, S., Radosavljevic, V., & Grgurovic, B. (2020). The potential of implementing augmented reality into vocational higher education through mobile learning. Interactive Learning Environments, 28(4), 404–418. https://doi.org/10.1080/10494820.2018.1528286

ReioJr., T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017). A Critique of Kirkpatrick’s Evaluation Model. New Horizons in Adult Education and Human Resource Development, 29(2), 35–53. https://doi.org/10.1002/nha3.20178

Sangsawang, T. (2020). An Instructional Design for Online Learning in Vocational Education according to a Self-Regulated Learning Framework for Problem Solving during the CoViD-19 Crisis. Indonesian Journal of Science and Technology, 5(2), 283–298. https://doi.org/10.17509/ijost.v5i2.24702

Semali, L. (2024). Leveraging multistakeholder partnerships in technical, vocational education, and training. Vocation, Technology & Education, 1(1), e3. https://doi.org/10.54844/vte.2024.0558

Shaji George, A., Baskar, T., & Siranchuk, N. (2025). Navigating the AI Disruption: Strategic Frameworks for Workforce Transformation and Innovation in the Age of Intelligent Automation. Partners Universal International Innovation Journal, 3(3), 83–95. https://puiij.com/index.php/research/article/view/189

Spöttl, G., & Windelband, L. (2021). The 4th industrial revolution – its impact on vocational skills. Journal of Education and Work, 34(1), 29–52. https://doi.org/10.1080/13639080.2020.1858230

Stufflebeam, D. L. (2000). The CIPP Model for Evaluation. Dalam D. L. Stufflebeam, G. F. Madaus, & T. Kellaghan (Ed.), Evaluation Models: Viewpoints on Educational and Human Services Evaluation (hlm. 279–317). Springer Netherlands. https://doi.org/10.1007/0-306-47559-6_16

Stufflebeam, D. L., & Coryn, C. L. (2014). Evaluation theory, models, and applications. John Wiley & Sons.

Suharno, Pambudi, N. A., & Harjanto, B. (2020). Vocational education in Indonesia: History, development, opportunities, and challenges. Children and Youth Services Review, 115, 105092. https://doi.org/10.1016/j.childyouth.2020.105092

Syauqi, K., Munadi, S., & Triyono, M. B. (2020). Students’ perceptions toward vocational education on online learning during the COVID-19 pandemic. International Journal of Evaluation and Research in Education (IJERE), 9(4), 881–886. https://doi.org/10.11591/ijere.v9i4.20766

van Eck, N. J., & Waltman, L. (2023). VOSviewer Manual. http://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.1.pdf

Wagino, W., Habibullah, H., Nasution, I., Giatman, M., Koto, R. D., Samala, A. D., Criollo-C, S., & Polin, K. C. (2024). Enhancing Academic Achievement and Critical Thinking through e-Learning: Exploring Motivation, Creativity, Participation, and Gadget Utilization in Higher Education. International Journal of Information and Education Technology, 14(12), 1679–1687. https://doi.org/10.18178/ijiet.2024.14.12.2198

Wagino, W., Maksum, H., Purwanto, W., Krismadinata, K., Suhendar, S., & Koto, R. D. (2023). Exploring the Full Potential of Collaborative Learning and E-Learning Environments in Universities: A Systematic Review. TEM Journal, 1772–1785. https://doi.org/10.18421/TEM123-60

Wagino, W., Tymofiiv, V., Polin, K., Milana, M., Astuti, M., & Koto, R. D. (2026). Longitudinal Analysis of Sustainable Impact of e-Learning Integration on Academic Capabilities, Critical Thinking Skills, and Digital Behavior of Automotive Engineering Students. International Journal of Information and Education Technology, 16(1), 259–277. https://doi.org/10.18178/ijiet.2026.16.1.2500

Wahjusaputri, S., & Bunyamin, B. (2022). Development of teaching factory competency-based for vocational secondary education in Central Java, Indonesia. International Journal of Evaluation and Research in Education (IJERE), 11(1), 353–360. https://doi.org/10.11591/ijere.v11i1.21709

World Economic Forum. (2025). The Future of Jobs Report 2025. World Economic Forum. https://www.weforum.org/publications/the-future-of-jobs-report-2025/

Xu, S., Ren, J., Li, F., Wang, L., & Wang, S. (2022). School Bullying Among Vocational School Students in China: Prevalence and Associations With Personal, Relational, and School Factors. Journal of Interpersonal Violence, 37(1–2), NP104–NP124. https://doi.org/10.1177/0886260520907360

Zeng, Y., Wang, G., Xie, C., Hu, X., & Reinhardt, J. D. (2019). Prevalence and correlates of depression, anxiety and symptoms of stress in vocational college nursing students from Sichuan, China: A cross-sectional study. Psychology, Health & Medicine, 24(7), 798–811. https://doi.org/10.1080/13548506.2019.1574358