Accreditation – Billie Beane, the Oakland A’s and Crosswalking Curriculum and Evaluation

This was another one of those challenging blogs for me.  Crosswalking the curriculum and evaluation is an important topic.  It makes sense and seems straight forward.  However, in practice it can quickly become complicated and overwhelmingly detailed.  As I found when trying to write this blog. There were way too many drafts that were too dense, too theoretical, too dry, too long, etc.  And then I remembered the movie Moneyball…

I love the story about how the Oakland A’s baseball team was transformed by Billie Beane’s belief in meaningful data (eventually tempered with personal openness and the ‘human touch’).  Billie Beane and his advisors created a complicated statistical process to identify players with potential, and then used formative assessment and program evaluation to customize training and team position.  Every element of their training program (ie: curriculum) was paired with assessment and evaluation measures.  The curriculum for training and the evaluation measures were completely integrated.  Evaluation was mapped onto curriculum.

In order to answer questions about the effectiveness of training – Where is the program working?  Where does it need to be changed? – the curriculum and the evaluation/assessment measures need to be mapped onto each other.  When they are integrated, transformation can occur.

One practical method for integrating curriculum and evaluation is to “crosswalk” between the two – pairing every curriculum component with specific evaluation or assessment measures.  This includes specifying the method and timing of the evaluation.  At its simplest, crosswalking is the creation of a series of 2×2 tables with curriculum on the Y axis and evaluation on the X axis.  Of course, with sufficient resources, one can also have the advanced quality improvement, data analytic approach used by the Oakland A’s!

Originally, the concept of crosswalking was derived from mathematics and information systems.  It is a technique for mapping the data elements from one schema onto the data elements of another related schema.  You are probably familiar with crosswalk tables that map the elements of ICD-9 onto ICD-10. CMS uses crosswalk tables to assist program administrators prepare for site visits. This idea of applying one schema to another has been used by industry and the government in curriculum development, as a way to link learning/training experiences with specific outcome competencies.

In creating a crosswalk between curriculum and evaluation, the curriculum is the foundation.  Evaluation is “mapped” onto curriculum.  To crosswalk curriculum with evaluation means to specify the method and timing of each evaluation/assessment for each curricular element.  Within the Self Study Guide, Appendix C is an example of cross-walking evaluation onto curriculum for the first 5 elements in Standard 2.

For your individual program, you can create a crosswalk for your curriculum by using Standard 2 as a foundation to build out your curriculum, then map your evaluation process onto the curriculum.  This approach strengthens program planning and delivery, and facilitates the accreditation review process.

Below is a 2×2 table that I made up as an example.  In this example, I integrated the some of Standard 2’s structural elements (1-5) and competency domains elements (6-11) into one crosswalk table.

In the movie Moneyball, Brad Pitt, playing Billie Beane said: “We’ve got to use every piece of data and piece of information, and hopefully that will help us be accurate with our player evaluation. For us, that’s our life blood.”  The same is true for NP postgraduate training.

Until next time,

Candice