Introducing Path Analysis as a Tool for Modeling and Quantifying Impact of Training Theories, Part III- Path Analysis and Reduced Form Models
This is the third article in a series introducing Path Analysis as a tool for developing macro-level models of training impact. The first article introduces the path analysis framework for testing theories of cause and effect relationships and explains the reduced form path model as the basis for quantifying training impact at a macro-level. The second article explores how a path model is created and introduces the concepts of direct and indirect effects. The purpose of this article will show how analysis of path models is conducted to estimate magnitudes of causal connections between variables from which a reduced form model can be derived and used to quantify training impact.
Path Analysis and Reduced Form Models
Current quantitative approaches to assessing training impact are not predicated on constructing a theory, although many of the fine books and papers on training impact and ROI do suggest the usefulness of examining the “chain of impact” from intervention to business result. What path analysis does is explicitly model this chain of impact.
Most, if not all, training impact studies have been conducted at, what I call, the micro-training impact view. As such, these sources recommend experimental design approaches such as control group designs, trend analyses or subjective assessments of impact to isolate the effects of the training intervention, typically on targeted programs. These methodological approaches are not suited to a macro-training impact view. They instead take the isolated program, view the relationship between training intervention and business result as a “black-box” and seek only to quantify the relation of output to input to demonstrate training impact. They cannot speak directly to how the specific impact developed; for this, a theory would be needed.
The path analysis framework, which is considered a “non-experimental” analysis for testing theories represented by path models, does explicitly represent the causal relationships among all the variables in the theory and can be used to derive a quantitative relationship between endogenous and exogenous variables. It is this relationship between endogenous and exogenous variables that allows us to quantify training impact, since in effect, the relationship between endogenous and exogenous variables reduces to the micro-training impact view. However, the relationships imbedded in the “black-box” accounting for the observed impact are illuminated by the path model and lead to a deeper understanding and explanation of how the training impact is achieved.
To see how path analysis can be used for training impact, let’s take the path model from the previous article and introduce the concept of path coefficient, bi, which represents the magnitude of the cause and effect relationship between any two variables. Below is what the path model of training impact taken from the previous article together with path coefficients looks like.
The path coefficient b1 can be interpreted as follows: If “Training Investment” changes by one unit, then “Skill Development” changes by b1 units. Thus the path coefficients in this context can be thought of as regression coefficients in a multiple regression model. Further, note that for the path model, “Training Investment” is the exogenous variable, while the remaining variables are endogenous.
Given the variables and path coefficients for the path model, you can express the model by the following set of equations:
The set of equations representing the path model are referred to as structural equations and used to represent training impact once the path coefficients are estimated. The mechanics of estimation are not going to be described here. There are a variety of regression and correlational techniques available for estimating path coefficients and path analysis programs exist as parts of larger statistical packages such as SAS and SPSS, or as web-available programs such as AMOS.
Once the structural equations are represented and coefficients estimated, path analysis enables any of the endogenous variables in the model to be represented in terms of the exogenous variable. In this case, “Training Investment” would enable us to quantify the impact of training investment on skill development, work performance or business result. The resulting equations expressing the endogenous variables in terms of exogenous variables only are called the reduced form equations. The reduce form equations for our path model are obtained from the structural equations by straightforward substitution of the “Skill Development” equation into the “Work Performance” equation and (re-written) “Work Performance” equation into the “Business Result” equation as follows:
Eliminating the intervening steps leads to the following reduced form equations:
Given the reduce form equation and knowing the values of the path coefficients, for any unit change in “Training Investment,” you would expect to see a b3 * b2 * b1 change in “Business Result.” Again, all the existing path analysis programs can calculate reduced form coefficients as part of their output.
This example shows that the reduced form coefficients integrate the several direct and indirect paths that characterize the total effect of the exogenous variable on each endogenous variable. It is this total effect that quantifies the impact of training. The path model represents how that effect is produced.
In these series of articles, the basics of path modeling and path analysis have been presented to demonstrate how this methodology can be used to help model macro-theories of (training) impact and quantify effects. The purpose in writing these articles is to introduce path modeling and path analysis concepts to the broader training community in the hope that organizations will see value in applying the concepts to their theories of training impact.