Building a certification program, Step 7: Blueprinting

Posted on
Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone

NOTE: This is an ongoing series. To view all articles in the series, click here.

Blueprinting is the process of creating a blueprint that you will use to design both instructional materials and exams. In the previous installment of this series we discussed in-depth the whys and wherefores of conducting a task analysis as a key activity when you are developing a certification program. As part of the task analysis we mentioned that a key objective was identifying the tasks that those in the role to be certified perform. Keeping this in mind, the obvious goal of a task analysis is to break down each task into its requisite steps.

The question then arises, how should the requisite steps gleaned from the task analysis be sequenced to maximize learning? Sequencing the steps and objectives is one of the primary goals of a learning/assessment blueprint. Just like a construction blueprint guides a general contractor through the steps of a construction project, so too will a learning blueprint guide an educator through the sequence of tasks and objectives for an effective learning experience.

That being said, it is clear that a learning/assessment blueprint is not clearly understood even among many learning and development professionals. What I have discovered in my research is that most learning professionals, especially those involved with certification, tend to speak about assessment blueprints as though everyone knows what an assessment blueprint consists of.

Few if any learning professionals, however, especially those involved with certification, speak of learning blueprints with the same passion that they speak about assessment blueprints. And no one until now has discussed an instrument that will allow a learning and development professional to scope and sequence the steps and objectives of an instructional event and thoroughly plan out and monitor the summative and formative assessments that are used to evaluate the effectiveness of the learning experience. That, in short, is the goal of this article — to introduce you to a learning/assessment blueprint instrument.

Before we go too far, I think it would be wise to explore some of the most common descriptions of the phenomenon known as blueprinting:

BLUEPRINTING: A definition

In a grade school setting, educators think in terms of curricula, units, and lesson plans. Lesson plans, which are used to sequence instruction, contain the specific, day-to-day activities that make up a unit of instruction. In a university setting, people think in terms of program study, syllabi, and classes. A syllabus is the scope and sequence description for a single course, while classes are individual meetings held regularly throughout the semester.

In a business or government setting, instruction is often approached with regard to competencies and certification (Brown and Green, 2011). So, no matter whether you are working with lesson plans, syllabi, or competencies and certification for any instructional event, the amount of information, and the order in which the information will be presented, must be determined.

No matter how big or how small an instructional event is determined to be, the scope and sequence must be developed at the outset in order to form a clear picture of the intervention’s requirements. The scope and sequence is documented in a learning blueprint which is the equivalent of the grade school lesson plan.

According to Brown and Green (2011), a traditional K-12 lesson plan contains five main sections. These are as follows:

1) The background, including the number of students and their grade level
2) The goal of the lesson
3) The objective of the lesson
4) The activities for the lesson
5) The evaluation method and criteria for the lesson

According to Patil et al. (2015) a “blueprint is a map and a specification for an assessment program which ensures that all aspects of the curriculum and educational domains are covered by assessment programs over a specified period of time. The term ‘blueprint’ is derived from the domain of architecture which means ‘detailed plan of action.’ In simple terms, a blueprint links assessment to learning objectives. It also indicates the marks carried by each question. It is useful to prepare a blueprint so that the faculty who sets question paper knows which question will test which objective, which content unit and how many marks it would carry.”

According to Shrock and Coscarelli (2007) one of the key purposes of an assessment blueprint in certification is for legal defensibility.

BLUEPRINTING: The history of the learning assessment blueprint instrument

Around 2008 or 2009, as I began my doctoral work, while I was still teaching and involved with certification training, I had an epiphany while reading a text by Shrock and Coscarelli (2007). I had never prepared an assessment blueprint and did not know what made up a good assessment  blueprint. Furthermore, the light really came on when I realized that one of my pet peeves (Wyrostek, 2008a) about certification assessments would be addressed if the scope and sequence of an assessment was well aligned with the instructional event(s) that supposedly were designed to prepare students for these certification assessments.

So often throughout the 1990s and into the 2000s, many of the major IT vendors failed to publish certification tests that were even remotely aligned with the available training. This caused a lot of bad feelings toward a number of the major vendors and pushed educators like me to begin the development of an instrument that would ensure that assessments were well aligned, at least in the planning stage, with the instructional event(s) leading up to the assessments.

In order to design a working instrument, my wife — who was a high school science teacher for more than 30 years — and I went out to dinner one night. We used several paper napkins to draft what has turned out to be a working solution, a blueprint instrument. This learning/assessment blueprint instrument functions in multiple ways; it can work as a basic lesson plan, as a simple assessment blueprint, or as full-bore blueprint for certification training and assessment, ensuring that the assessment aligns with the preparatory training. That is where this learning/assessment blueprint instrument began — as a high school science lesson plan.

Next let’s look at the first page of the leaning /assessment blueprint instrument which covers the lesson plan, or the learning portion of the instructional event.

BLUEPRINTING: The Learning Page

Blueprinting is the process of creating a blueprint that you will use to design both instructional materials and exams.

Figure 1: The Learning Page Of The Learning/Assessment Blueprint

The learning page of the learning/assessment blueprint can be used as a daily lesson planner, as the requisite documentation for planning out a five-day technical course with or without certification, or as a planning instrument for a semester long program. It can be used as a standalone plan for teaching or training purposes, or in conjunction with the assessment page to ensure that the assessment is aligned with the prerequisite instruction.

To say that this page, as part of this instrument, is flexible is an understatement. In that light this blueprinting instrument has been a work in progress for over 10 years. I am most open to reader feedback on ways it can be designed to be more efficient and effective.

To begin using this instructional instrument, it is assumed that you can provide some basic information. Consider the following sample data:

Learning Title: Master of Integrated Networking Capstone Course (Wyrostek, 2008c).
Blueprint Developer: W.Wyrostek
SME Developer: Sharon W.
Course Code: MINCC04000
Learning Goal: To prepare students for the MIN Certification Performance Assessment
Date Blueprint Completed: 10.31.18
Date SME Reviewed 11.1.18
Learning Format: Instructor-led lecture (50 percent) with hands-on labs (50 percent)
Learning Prerequisite: can show evidence of having successfully taken the courses leading to a major LAN and WAN credential.
Learning Tone: Technical
Highest Level of Primary Audience: Master CNE and CCIE

Now that you have captured the high-level data, it is now time to enter your learning objectives and the information needed to meet those objectives. You will notice that, if you completed the task analysis as discussed in the previous article, much of the required information will be done for you already. The information required for each objective is the following:

Objective ID: NOVL65 CNA01
Learning/Testing Objective: Create a new user in the application
Audience / Job / Role / Task / Duty: Super Users
Activities:
— Discussion
— Demonstration
— Hands-on Practice
Reference(s): User Manual
Evaluation: M/C QUESTIONS
Setup: Access to environment, logins, etc
Objective Prerequisites: Optional
Schedule or Time Required to complete: 0.25 hours
Knowledge Level (1-6): 4 — From KNOWLEDGE (1) to SYNTHESIS (5) and EVALUATION (6)
Frequency Used (1-5): 1 — From RARELY USED (1) to ALWAYS USED (5)
Importance (1-5): 5 — From SLIGHTLY IMPORTANT (1) to MISSION CRITICAL (5)
Content Available (0-5): 2 — From LITTLE OR NO CONTENT (0) to A LARGE AMOUNT  OF CONTENT COVERED (5)
Content Development Level of Effort (days or hours): 2.5 hours
Assumptions: 100 percent dedicated developer

Now that you have a sense of what is involved with the learning portion of the blueprint, it is time to examine the assessment page.

BLUEPRINTING: The Assessment Page

Blueprinting is the process of creating a blueprint that you will use to design both instructional materials and exams.

Figure 2: The Assessment Page of The Learning/Assessment Blueprint

The assessment page of the learning/assessment blueprint can be used to plan out quizzes and knowledge checks as well as all types of formative and summative exams, including all high stakes and low stakes certification exams. It can be used as a standalone plan for teaching/training purposes or in conjunction with the learning page to ensure that the assessment is aligned with the prerequisite instruction.

To say that this page, as part of this instrument, is flexible is an understatement. In that light, this blueprinting instrument has been a work in progress for more than 10 years. I am most open to reader feedback on ways it can be designed to be more efficient and effective.

To begin using this instructional instrument it is assumed that you can provide some basic information. Such as with sample data included:

Assessment Goal: To validate a candidates’ performance and knowledge of the role of MASTER OF INTEGRATED NETWORKING
Assessment Name; MASTER OF INTEGRATED NETWORKING CERTIFICATION ASSESSMENT
Blueprint Developer: W. WYROSTEK
SME Developer: SHARON W.
Date Blueprint Completed: 10.31.18
Date SME Reviewed 11.1.18

Now that you have captured the high-level data, it is now time to enter our assessment objectives and the information needed to meet those objectives, including the test questions that correlate to each objective. You will notice if you completed the task analysis, or the learning page, as discussed earlier, much of the required information will be done for you already. The information required for each objective is the following:

QUESTION ID: Include all questions (test items) related to each objective
Such as: NOVL65CNA01Q001-Q003 with included content
Objective ID: NOVL65 CNA01
Learning/Testing Objective: Create a new user in the application
Knowledge Level (1-6): 4 — From KNOWLEDGE (1) to SYNTHESIS (5) and EVALUATION (6)
Frequency Used (1-5): 1 — From RARELY USED (1) to ALWAYS USED (5)
Importance (1-5): 5 — From SLIGHTLY IMPORTANT (1) to MISSION CRITICAL (5)
Content Available (0-5): 2 — From LITTLE OR NO CONTENT (0) to A LARGE AMOUNT  OF CONTENT COVERED (5)
Job/ Duty/Task/Role Being Tested: SUPER USER

Blueprinting is the process of creating a blueprint that you will use to design both instructional materials and exams. This next set of metrics is where you can monitor your assessment by the objective. It includes the following parameters:

Number of Items Needed: 6
Number of Items Created: 3
Variance: -3
Total number of Hot Spot: 0
Total number of Image: 0
Total number of Matching: 0
Total number of Multiple Choice: 6
Total number of Multiple Select: 0
Total number of True/False: 0
Percentage of Hot Spot: 0 percent
Percentage of Image: 0 percent
Percentage of Matching: 0 percent
Percentage of Multiple Choice: 100 percent
Percentage of Multiple Select: 0 percent
Percentage True/ False: 0 percent
Reference(s): See Shrock and Coscarelli (2007)
Comment(s): NONE

BLUEPRINTING: Putting it all together

And there you have it: a proven method for preparing a learning/assessment blueprint, which will provide you with a solid foundation for a certification program. Our next step will be to assimilate all the data from our earlier analyses, along with the task analysis and blueprint, and begin designing and developing a learning intervention and assessment that follows our blueprint and that leads to certification. Next stop: instructional systems design.


REFERENCES

1) Brown, A., & Green, T. D. (2011). The Essentials of Instructional Design: Connecting Fundamental Principles with Process and Practice (2nd ed.). Boston: Prentice Hall.

2) Dick, W., Carey, L., & Carey, J. O. (2009). The Systematic Design of Instruction (7thed.). Upper Saddle River, N.J.: Merrill/Pearson.

3) Morrison, G. R., Ross, S. M., Kalman, H. K., & Kemp, J. E. (2013). Designing Effective Instruction (7th ed.). Hoboken, NJ: Wiley.

4) Patil, S. Y., Gosavi, M., Bannur, H. B., & Ratnakar, A. (2015). Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in pathology. International journal of applied & basic medical research, 5(Suppl 1), S76-9.

5) Shrock, S. A., & Coscarelli, W. C. C. (2007). Criterion-referenced Test Development: Technical and Legal Guidelines for Corporate Training (3rd ed.). San Francisco, CA: Pfeiffer.

6) Smith, P. L., & Ragan, T. J. (2005). Instructional Design (3rd ed.). Hoboken, N.J.: J. Wiley & Sons.

7) Wyrostek, W. (2008a) The Top 10 Problems with IT Certification in 2008.  Retrieved from http://www.trainingindustry.com/it/TO_Article.asp?ID=8144

8) Wyrostek, W. (2008b) Now What? First Steps into IT, 2008 Edition.  Retrieved from http://www.informit.com/articles/article.aspx?p=1183904

9) Wyrostek, W. (2008c) An Integrated Certification Plan.  Retrieved from http://www.informit.com/articles/article.aspx?p=118559

Share on Google+Share on LinkedInShare on FacebookShare on RedditTweet about this on TwitterEmail this to someone
Warren Wyrostek

ABOUT THE AUTHOR

Warren E. Wyrostek is a Solutions Oriented Educator and Leader, a Certified Trainer and Facilitator, an Experienced DACUM Enthusiast, and an innovative Certification and Assessment expert in demand. Warren holds a Doctorate in Education in Curriculum and Instruction. Currently Warren is an Adjunct at Valdosta State University and the owner of 3WsConsulting - Providing Efficient And Effective Top To Bottom Solutions To Learning Issues. Warren can be reached at wyrostekw@msn.com.

Posted in Resources|

Comment:

Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>