Tag: training effectiveness

Wednesday, June 8th, 2011

Competency Frameworks: A First Step

Listen with webreader

I’ve been seeing the phrase “competency frameworks” a lot lately.  I’m glad.  I’ve long been concerned about the disconnect between training content and job performance.  In a quarter decade of business training, I have rarely felt much attention was given to the question: “What do these people need to know to do better in their jobs?”  I often felt that training was designed from the starting point of “here’s what I know so that’s what I’ll teach”.

So, what is a competency framework and how will it improve the effectiveness of training?

Ratings of Exceptional, Exceeds Expectations, Meets Expectations, etc.The HR Dictionary defines competency framework as “the set of duties or tasks performed as part of a job with the standards which should be achieved in these duties”.

OK, so for every training course we design, we need to know:

  • What job are we training for?  In other words, what duties or tasks are we teaching someone how to do?
  • What are the standards that we will measure against?  How will we know if our students have learned enough of the right things to perform those duties?  How will we know our training accomplished this?

In my mind, competencies for education are fairly well-defined and adhered to by a very strict accreditation system.  It is relatively easy to accurately measure students’ understanding of geometry, grammar, or DaVinci’s work.  Education provides foundational knowledge; training is the application of that knowledge in a specific situation.  My brother-in-law (a math whiz) was always amazed at how his grandfather used calculus in his machine working job.  But Granddad didn’t actually know calculus; he knew some rules for machining.  My brother-in-law, with his foundational knowledge, can apply what he knows about math to just about any situation.

The difficulty with business training is that job descriptions (and their related competencies) change frequently.  People in those jobs come from varying backgrounds.  Often, people have to adapt to new job requirements because that’s the best thing for the company.  An example would be that of typists.  There’s no such thing as a typing pool any more.  For awhile, typists were converted to word processors (using machines of the same name).  That transition required an entirely new competency: using a computer.

Businesses try to fill the gap between “knowledge/skill” of workers (old, young, new, tenured) and what they need at that moment, with training.  Not only is it difficult to determine what training is required for that gap, it is even harder to measure if the training is effective.  Sadly, it is even more difficult because often the very people in charge of these efforts are not competent in training design or testing!  I’m hoping that with increased emphasis on it from a software view, there will be some attention to the concept itself.

Much the same as when mapping a process, the people who do the job should be involved in the determination of the necessary competencies.  Mind Tools™ has posted an excellent article on the subject, which includes a step by step guide to get it done.  As they say, it will take a lot of effort; effort by the people who actually know the positions.

The US Army is very good at defining job duties and training to them. Every job, at every classification, has defined skills within the MOS system. (Note: this term varies by branch of service, but the structure is very similar.)  Here is an example for a US Army Corp of Engineers Diver for five skill levels.  Notice how this also includes required scores on fitness and written tests, as well as other requirements.  Those developing the training would start with these requirements, not with what they felt like teaching!

I encourage you to read as much as you can about the concept of competency frameworks (start with this Wikipedia article), browse through the Army’s MOS listings (for ideas on how to structure yours), and do your own Internet searches.  To read more on how competency frameworks are critical to the success of your business, visit my blog for earlier posts (such as this one) on testing in a business environment and this one on Purpose-Objectives-Goals for business training.  Future posts are planned for how Moodle supports competency frameworks through grades, scales, and outcomes.


Tags: , , , , , , , , , , , , ,

Tuesday, August 3rd, 2010

Creating Purpose-Objectives-Goals for a Business Training Course

Listen with webreader

In my post on creating course outlines, I wrote that two pages in any course should be the Purpose-Objectives-Goals (POG) page and the Summary page. That sounds simple enough, right? Well, maybe not…

What is the POG for a business training course? Is it the same as it would be for a university course? Does it come from the business case for delivering the training? Is it related to the mission of the business?

Let’s start with some assumptions:

  • Business training differs from academic education (note the different uses of “training” and “education”).
  • Academic education seeks to impart not just information to students, but to equip them to think about new scenarios, to integrate ideas, and to build upon their education as they experience life. This is done through a foundation of knowledge. We don’t simply learn that “2+2=4”, but why it’s so.
  • Business training, while sometimes is for the sole enrichment of employees, is usually targeted to improve a business metric. Or, it is intended to be.

A while back, I wrote a post on assessing the effectiveness of business training. I have observed a huge gap between the intended or desired outcome of business training and what it actually delivers. My “hypothesis” (which I have empirical evidence to support) is that this gap exists because of three things:

  • The lack of proper evaluation of training effectiveness
  • The failure to align training objectives with business objectives
  • The failure to create and deliver training to the objectives, if they had been aligned in the first place

The first item is discussed in the post; the third item is a deeper subject known as instructional design. This post addresses the second item, aligning training objectives with those of the business.

So, how do you align training goals and objectives with the goals and objectives of your business?

  1. First, understand your business goals and objectives. Where are your “problem areas”? What do you want to improve? Where do you want to reduce risk? Some likely business examples:
    • prevent accidents
    • reduce errors
    • improve customer service
    • improve efficiency
    • reduce waste
    • improve communication
    • reduce time to market
    • leverage knowledge
    • protect intellectual property
    • improve work environment (physical)
    • improve quality of work (emotional)
    • increase promotion opportunities
    • increase market share
    • reduce redundancy/confusion from department to department
  2. Second, understand to at least some degree, who in the organization can affect these goals. Your delivery driver might have a strong influence on several goals, but she isn’t going to have anything to do with reducing the time to market of a new product. A RACI chart would be a useful tool for this.
  3. Based on the RACI chart, decide what level of training should be provided to each position in topics aimed at achieving each goal. Bloom’s rose would be a great reference for this.
  4. Determine what those topics, tools, and methods are. You will need to seek the assistance of subject matter experts to accomplish this.
  5. Create a curriculum (map out all of the training).
  6. Write a POG for each course in that curriculum. Please note that the terminology can be highly variable.. I’ve seen many instances where Goals were defined as more general than Objectives. Still others use them interchangeably or use completely different terminology. It doesn’t matter. The important things are that you use the terminology consistently, in a manner that your students understand, and that these three words combine to define the scope of the course.

Purpose (a.k.a. Aim): These statements should be formulated with phrases similar to these: “to provide an overview of…”, “to provide the framework for…”, “an in-depth discussion of…”, “to advance the knowledge from Course 101”, “to apply knowledge to field examples in…”.

While Control Charts have a solid history of use in manufacturing, they are excellent tools for use in monitoring and controlling transactional processes as well. This course demonstrates the construction and use of control charts, providing both scenarios and corresponding example control charts.

(Learning) Objectives: These are essentially from Bloom’s categories (Cognitive domain) and more specific than the purpose of the course. There are usually a few objectives.

1. Explain the purpose and proper use of control charts.
2. Introduce the six basic types of control charts.
3. Provide examples of how control charts can help stop trends and identify potential problems in the processes.

Goals (a.k.a. Learning outcomes): SMART goals directly related to the objectives.

It is important that you leave this course knowing:
1. Which type of control chart is best suited to different situations.
2. How to construct and use a control chart.
3. How control charts fit into larger quality initiatives.

These examples are taken from the SPC 101 course at BeeLearn.com. They are not perfect. Yours probably won’t be, either. But, they are “good enough” to define the scope of the course, set expectations, and to build content around.

The Summary page of every course should tie back to the POG. The course exams and activities should be built in support of the POG. The content should be built to the POG. If you do this, you’ll have created a course that serves a purpose; to make your business stronger by providing training that is aligned with and effective at meeting your goals and objectives.


Tags: , , , , , , , , ,

Monday, May 10th, 2010

Six Easy Steps to Convert Your Content to e-Learning

Listen with webreader
Dazed and confused by all the choices!

Which way to go?

Recently I wrote a post about how e-Learning can benefit entrepreneurs and small businesses by reaching a larger audience and leaving more time to spend on higher value activities. The biggest barrier to entry for those of us sans-IT department is not the cost or the skill, or even breaking the paradigm of teaching face to face; it is knowing where to start and what steps to take next.

Since I went through this myself, I know how bewildering it can be! I had several training courses developed and a pretty good feeling about myself as a teacher. I thought that was enough. But without a budget or a course designer as my partner, I had no focus and no clearly defined path. I wandered aimlessly, trying to find the right LMS, the right amount of content to start with, and the best way to market it. If I knew then what I know now, I would have done this:

1. Start with a book (published or not), a slide presentation that you can talk to, or an existing training course. If you don’t have something already put together, you can still use e-Learning as your “publishing platform”. But be prepared for a lot of time, money, and effort. You’ll be tackling two huge tasks at once: compiling and organizing your thoughts plus the vastness of e-Learning options. Looking on the bright side, if you don’t already have something put together, you can do so with your e-Learning course in mind. 

2.Determine your budget! I had one client who told me that her budget was “angel’s wings”. This made it very difficult for me to design a plan for her. If I were an architect and you asked me to build a house, I’d need to know how big and how fancy you wanted it. It’s no different for e-Learning design and creation.

3. Find a course designer who will work within your budget to:

  • Determine your audience and their constraints, as well as your learning objectives for this course
  • Create the course and curriculum architecture (even if you start with just one, you’ll probably want to be able to add more)
  • Determine the best blend of teaching methods for your audience
  • Create methods of evaluation for both participant competencies and how your course meets their objectives
  • Determine the best hosting solution (shared, branded site) for your situation
  • Create a course (or courses) to fit all of the above.

No one, not even the best instructional designer in the world, can create a course without knowing these things. Even if you “give them all the materials”, they need to ask these questions. If you find someone who does NOT ask you these questions, move on!

4. Prioritize your efforts and budget to meet the needs of your audience first. Do what you can to reduce the costs and timeline of designing and launching the course, but don’t get in the way. Make sure the course designer you choose is willing to “share the load” with you, if that’s what you want. Remember, she is working for you, not the other way around!

I have a client for whom I’m creating several courses. He does not have “angel’s wings” for a budget, so we discuss what must be included, what should be included, and what we can leave until another time. I give him examples for things such as quiz questions, so that he can write his own. He’s not as good as I am at writing questions, but he simply can’t afford to hire someone to do it all. I don’t see this as cutting into my work; I would rather have some business from him and help him, than to not work with him at all. In return, he trusts me to make instructional design decisions so that he can focus on taking care of his clients.

5. Stay focused and within your original scope. There is a simple tool often referred to as a multi-generation product plan (MGPP). For an e-Learning course, it might look like this [click here]. It allows you to have a lofty goal while still accomplishing important milestones along the way. There are numerous examples of how well this works; start small, earn and learn, improve and grow.

6. Gather and analyze data on participant scores, feedback, and participation so that you can adjust and improve. Short term revenue is not a good measure of long term success! You must understand if the course met your objectives, if the course met the objectives of your audience (and their superiors if they paid for it), and in general, what people thought of it. You’ll know whether to add to or edit the content, or simply change the way you market the course.

If you follow these steps, in order, you’ll find that you’ll have your e-Learning course or site up and running before you know it.  Determine your budget and stick to it! Create an MGPP and stick to it!  Don’t try to do too much because if you do, you won’t be able to do any of it!

Next time I’ll be writing in more detail about the web hosting aspects of e-Learning for small businesses and how to find a course designer.


Tags: , , , , , , , , , , ,

Wednesday, April 21st, 2010

Protecting your eLearning content – Is this something to worry about?

Listen with webreader

I get a lot of questions about how to keep eLearning content safe from being copied, stolen, and/or plagiarized. My response is always “the only way to stop people from using your ideas is to keep them to yourself“. Both copyright and patent laws protect the owners of ideas, concepts, and designs from being copied outright in their original format. But you can’t stop someone from taking your ideas and making them better or combining them with the ideas of others for yet another way to look at things. Having others build on your ideas (and even copy some of them) is a true measure of how good your ideas really are.

Let’s be honest, how many of us have created some new process, program, or graphical layout that is totally from scratch? I haven’t. Everything I have ever taught, written, and/or created is a compilation of what I read in text books, what my teachers taught me, and what I’ve seen and otherwise experienced. What makes my training, coaching, and consulting valuable is ME. My training content is really an example of my skill as a teacher and subject matter expert. Yours should be, too.

Ask yourself:

  • Can anyone else do a better job teaching my content than I can?
  • Have I been able to capture everything I know on paper? Is that all there is to it?
  • Of all the text books, magazine articles, online copy, examples, etc., that I’ve ever written, read, or taught, how much of it was “stolen”? Isn’t it true that almost everything is built upon something else?

Throughout my 25 year consulting career, I watched in amazement as my training material – and that of my mentors and colleagues – showed up in the “work of others”, sometimes as part of very large training programs. Plagiarism is one thing; it’s illegal and unethical, but it’s hard to stop. When I was less wise than I am today, I was enraged by this. Then I realized that even though my chart might be in someone else’s course content, I was still the one getting the rave teacher reviews. “They were coming to see ME”.

The only thing you can do is to stay ahead of those who would steal your content to sell it as their own. While they’re peddling last month’s idea, you’re launching this month’s better idea. And you’re doing it better because you’re the one who can teach it best.

If you’re concerned about one person printing off your eLearning content and giving it to everyone else – thus cutting into your revenues, this can be easily prevented by making your online version worth spending the money on and harder than blazes to copy. Make people want the original recording…not a scratchy copy with background noise (metaphorically speaking). (Click here to read my previous post on what makes eLearning GREAT).

If you have the best jelly recipe on the planet, give away peanut butter to entice people to buy the jelly. Even better, give away the jelly recipe to prove that the real secret is the way it’s made! If you are a wonderful teacher with an effective way of teaching, let everyone know how good you are; let them know this content is yours and that if they want to learn more, they’ll have to come to YOU. Let others try to copy it! Michael Port gives away a book chapter to show people how good the book is. Williams-Sonoma gives away recipes (both in stores and online) to show how much they know about cooking (and to get you to buy the equipment). In The Martha Rules, Ms. Stewart tells readers to “Profit by giving information away”. These people are onto something…

If you want to teach something or you want to sell your ideas, you have to be willing to accept that “imitation is the sincerest form of flattery”. [Charles Caleb Colton] However, no one is as good as the original – YOU. Spend your time, energy, and money making your content great and leave the worrying about theft to diamond dealers and fine art museums.


Tags: , , , , , , ,

Monday, April 5th, 2010

Assessing the Effectiveness of Business Training

Listen with webreader

Whether you are the department head paying for it, or the consultant delivering it, justifying the expense of training is more important now than ever.  While it might not be easy to quantify the long term financial benefits of a trained workforce, it is possible to demonstrate knowledge, skill, and practical application gained from a specific training workshop. 

While there is no SAT or any other standardized test to administer in most business training, we’re so programmed to take (and give) tests, every business training course seems to end in one.  Typically, everyone passes with flying colors.  Does this mean the training was effective?  No.  It doesn’t even mean that any knowledge was transferred.  What we really want to know is if this training changed the way people do their jobs, which is the whole point of business training!  This is rarely measured or tracked, but it can (and should) be.

During the workshop, test knowledge and the ability to apply it, in a variety of ways: 

  • Administer well-written tests, not just tests.  These tests should be developed by people who know how to write valid tests; being a great trainer does not always lead to being a good test writer.  Knowing the subject upside down and sideways is no guarantee of test expertise.
  • Scatter quick quizzes (written or verbal) throughout the workshop.
  • Conduct group activities with a skilled observer who can assess whatever qualities you’re interested in; leadership, ability to work together, problem solving, time management, etc.
  • Include written assignments, to be graded for content as well as writing ability, if writing is an important skill in using the workshop knowledge.  If writing isn’t important, don’t make the students write!
  • Include a speaking assignment (such as a project presentation) if it’s important to be able to use this new knowledge.  If it is important, grade it.  If it’s not important, leave it out!

The most valuable assessment comes after the workshop.  This is where the knowledge is applied to the job, which is where the benefits lie. 

  • Coaching (usually by the trainer) to guide the participants through the first-time application of their new-found knowledge is not only a valuable assessment tool, but it generally improves the effectiveness of the training.
  • Outside of class assignments – graded by the trainer/coach – are a great way to assess the ability of the participant to apply what was taught in the workshop
  • Follow-up surveys (to obtain self-perception) in conjunction with project/work audits can be used to measure actual implementation of the workshop knowledge.

Stay tuned for future posts to include examples of good tests and surveys and how to improve the effectiveness of training (once you’ve measured it, you can make it better!).


Tags: , , , ,

  • LinkedIn LinkedIn Facebook LinkedIn newsletters
  • Archived Posts
  • Archived Newsletters
  • Sign up for Albany Analytical Newsletters
    * = required field
    I would like to receive the following newsletters:

  • Test

    Testing Sidebar 2

© 2010, All rights reserved, Albany Analytical, Inc.

Blossom Theme by RoseCityGardens.com

/***Google Analytics Code ***/ /***End of Google Analytics Code ***/