Four Levels of Evaluation
The Four Levels of Evaluation, also referred to as the Kirkpatrick Evaluation Model, was created by Donald Kirkpatrick, Ph.D. to define the four levels of training evaluation. The four levels of evaluation are: (1) the reaction of the student and their thoughts about the training experience; (2) the student's resulting learning and increase in knowledge from the training experience; (3) the student's behavioral change and improvement after applying the skills on the job; and (4) the results or effects that the student's performance has on the business.
Donald Kirkpatrick developed the Kirkpatrick Evaluation Model for evaluating training during the 1950s. The model was initially published in a trade journal and later in Kirkpatrick’s 1975 book, Evaluating Training Programs. Today, it is the most recognized method of evaluating the effectiveness of training programs.
Each of the four levels is explained below.
Level 1 - Reaction
Level 1 solicits opinions of the learning experience following a training event or course. Typical questions concern the degree to which the experience was valuable (satisfaction), whether they felt engaged, and whether they felt the training was relevant. Training organizations use that feedback to evaluate the effectiveness of the training, students’ perceptions, potential future improvements, and justification for the training expense. A variety of sources estimate that approximately 80 percent of training events include Level 1 evaluation.
Level 2 - Learning
Level 2 measures the degree to which participants acquired the intended knowledge, skills and attitudes as a result of the training. This level is used by instructors and training executives to determine if training objectives are being met. Only by determining what trainees are learning, and what they are not, can organizations make necessary improvements. Level 2 can be completed as a pre- and post-event evaluation, or only as a post-evaluation.
Level 3 - Behavior
Level 3 measures the degree to which participants’ behaviors change as a result of the training – basically whether the knowledge and skills from the training are then applied on the job. This measurement can be, but is not necessarily, a reflection of whether participants actually learned the subject material. For example, the failure of behavioral change can be due to other circumstances such as individual’s reluctance to change. Level 3 evaluation involves both pre- and post-event measurement of the learner’s behavior.
Level 4 - Results
Level 4 seeks to determine the tangible results of the training such as: reduced cost, improved quality and efficiency, increased productivity, employee retention, increased sales and higher morale. While such benchmarks are not always easy or inexpensive to quantify, doing so is the only way training organizations can determine the critical return on investment (ROI) of their training expenditures. One typical challenge is to identify whether specific outcomes are truly the result of the training. Level 4 requires both pre- and post-event measurement of the training objective.
In 2011, Atlanta-based Kirkpatrick Partners modified the learning and evaluation model to more easily calculate the return on expectations (ROE) of stakeholders. Kirkpatrick Partners contends that ROE is the “ultimate indicator of value.” While ROI and ROE are common methods for evaluating and justifying training, many training organizations still struggle with the four levels, particularly quantifying levels three and four.
As learning content becomes increasingly available through informal channels, learners and training organizations are finding ways to use and incorporate this type of material into personal enrichment or professional training methods. Most if these informal channels, however, do not include pre- and post-evaluations – or any type of measuring tools – to measure effectiveness. To mitigate this issue, advancements like Experience API (or Tin Can API) allow training organizations to effectively track and measure informal learning activities.
Contributors: Donald L. Kirkpatrick, James D. Kirkpatrick, and Wendy K. Kirkpatrick
Written for TrainingIndustry.com