Blog Article

Home/BLOG/Blog Article

In 2003 I attended a talk given by Prof Ronald Harden, who at that time was the Director of the International Medical School (IVIMEDS). During his talk he showed a cartoon which has stuck with me ever since.

A fundamental part of education and training is to establish if it has been effective. That is the job of assessment. Assessment isn’t easy. We need to design assessments to match what we are assessing.

Traditionally, training course defined learning objectives which show what should be learned. Increasingly, nowadays, training is linked to becoming competent through links to a competency map (see another blog post “Are you competent” for more on competencies). The format of learning objectives and competencies is very similar. Some examples might be:

• Analyze the impact of information system risks on the organization (CPA Canada Competency)
• Apply financial information to analyse, appraise and resolve high-level issues of strategic financial management and organisational planning in the public sector context (African Professionalisation Initiative Competency)
• Interpret and analyse the range of ESG reporting standards and frameworks, including the SDGs and their indicators (IASeminars Course Learning Objective)

Note that competences and learning objectives usually start with an “action” verb. These verbs are very significant because there is a hierarchy from basic to complex linked to the level of cognitive comprehension. This was first recognised in 1956 by Benjamin Bloom.

Bloom's Taxonomy (from Jessica Shabatura,University of Arkansas)

In fact, this hierarchy is like the proficiency levels used in competency frameworks. It is typical that the lowest proficiency levels use Understand and Remember, the next level uses Apply and Analyse and the highest proficiency levels use Evaluate and Create. This means that when we are deciding how to assess if an objective/competence has been achieved we need to be aware of the complexity of that task and decide what is an appropriate way to assess it.

A lot of detailed thought about methods of assessing accounting training has been carried out. For example, the International Accounting Educational Standards Board (IAESB) through the IES (International Educational Standards) 2017 Handbook of International Education Pronouncements said:

“Assessment activities are those activities designed to assess specific areas of professional competence. During IPD (initial professional development), assessment activities can be selected to match the particular aspect of professional competence being assessed.

Examples of assessment activities may include, but are not limited to:

1. Written examinations;
2. Oral examinations;
3. Objective testing;
4. Computer-assisted testing;
5. Workplace assessment of competence by employers; and
6. Review of a portfolio of evidence on completion of workplace activities.”

A more detailed set of assessments methods was provided in “Competency-Based Education and Assessment for the Accounting Profession: A Critical Review”, 2003, J.E.Boritz and C.E.Carnaghan, Canadian Accounting Perspectives CAP Vol. 2 No. 1.

“Suggested sources of information for assessment purposes include:

• samples of work from the workplace;
• evidence from prior learning;
• multiple-choice tests;
• written responses (short answer, extended answer);
• oral questioning;
• project reports;
• assignments;
• log books, portfolios;
• self-reports;
• simulations (e.g., competency tests, skill tests, proficiency tests, projects/assignments);
• direct observation of work activities;
• indirect observation;
• supervisor assessments/ratings”

Obviously, many of these assessment methods require work to be submitted for assessment by experts. This is effective, though time-consuming and demands skilled assessors. Increasingly there is a demand for objective and automated assessment which has the benefits of being both consistent and immediate.

It is this challenge of assessing higher cognitive processes in an objective and automated fashion that faces the IASeminars customised online learning team on a regular basis. If you face similar problems, please contact us to discuss our innovative approaches.