Improving Assessments

We can work with you to improve assessments in a number of ways:

1. Review Existing Questions - Are they clearly written and do they objectively measure the intended subject areas? More

   We first agree the overall aims of the assessment and then analyse questions on the basis of this and any existing response data to ensure:

   1. Question Scope

      - Questions are assigned to the appropriate subject area
      - The question set provides complete and balanced topic coverage
      - The question set contains the right mix of easy and difficult questions (for the scoring to polarise the respondents in the desired way)

   2. Question Wording

      - Phrases are a suitable length/complexity
      - Vocabulary/terminology/abbreviation used is appropriate for the audience
      - The wording is clear and consistent without any cues (clues in the wording which might accidentally indicate the correct answer)

   3. Question Order

      - The topics proceed logically
      - The questions are, if appropriate, randomised correctly (Totally/semi-random selection and/or ordering)

   We feedback any comments and suggested changes in an electronic document. If you feel it would be useful, we are also happy to discuss these edits in more detail.

2. Author New Questions – what questions should be asked and how? More

   We first agree the overall objectives of the assessment, the discrete topic areas (to enable modular scoring and feedback) and then author the questions to ensure:

   1. Question Scope

      - Questions objectively measure the various knowledge domains (and allow scoring to polarise the respondents in the desired way)
      - The questions can be easily assigned to a specific topic area
      - The question set provides complete and balanced topic coverage yet remains efficient?

   2. Question Wording

      - Phrases are a suitable length/complexity
      - Vocabulary/terminology/abbreviation used is appropriate for the audience
      - The wording is clear and consistent without any cues (clues in the wording which might accidentally indicate the correct answer)

   3. Question Order

      - The topics proceed logically
      - The questions are, if appropriate, randomised correctly (Totally/semi-random selection and/or ordering)

   We then provide a draft question set in an electronic document along with any comments or queries. As necessary, we are also happy to discuss/edit these questions, trial and validate as appropriate (see below for further details).

3. Trial New/Existing Questions - Are the questions functioning correctly? More

   However carefully questions are written, it is always worth trialling them with a sample of the target audience. We can deliver the sample questions using our on-line platform and also check the validity and reliability of the questions by analysing the response data with a variety of quantitative and qualitative techniques.

4. Improve Question Delivery - Are the questions being asked and answered effectively? More

   We analyse the question context (such as the instructions respondents are given beforehand), question presentation, data capture mechanisms and, where appropriate, suggest improvements. We can also administer questions for you using our on-line platform.

5. Analyse Response Data - How well are the questions performing and what do the answers actually mean? More

   We help you extract the full value of information from responses and pin point any limitations and caveats. We do this by analysing the response data using a variety of quantitative and qualitative techniques and, in turn, gauging the quantity and quality of useable data. We can also provide a cleansed response data set (i.e. without spurious and irrelevant entries) for further 3rd party analysis.

6. Train Question Authors - what are the principles and processes behind good question design? More

   We can train your staff on the various step and nuances of effective question authoring. We can also cover the analysis and interpretation of response data in order to validate new or existing content.

Where can we help?

Good assessment design adds value wherever an organisation wants to measure knowledge and understanding. However, it is particularly important in the following areas:

1. Training Needs Analysis - Do employees know what they need to know? Key Benefits:
   • Identify those employees who need training (therefore allowing most appropriate use of training resources)
   • Identify employees who, due to misunderstanding, put the organisation at risk by potentially doing things incorrectly (therefore requiring the most urgent interventions)
   • Train employees only in the specific areas in which they need training (saving time and money)
   • Map knowledge risk against a competency framework (to quantify the overall operational risk)
   • Secure buy-in from employees that they need training (Research shows people learn more when made explicitly aware of their training needs)
2. Training Evaluation - How much have employees learnt from training? Key Benefits:
   • Measure return on training investment (particularly effective if also testing some months after training to measure retention and, implicitly, application on the job)
   • Improve the accuracy and validity of L&D benchmarking and KPIs (by having objective measures of understanding to compare before and after training interventions and/or different types of intervention teaching the same content)
   • Create credible qualification programmes that are valued by employees and which can be linked to their competency frameworks
   • Map attainment directly against a competency framework
   • Identify opportunities to improve training
   • Save time and money by avoiding ineffective training
3. Regulatory Compliance - Are the guidelines, processes and controls truly understood? Key Benefits:
   • Demonstrate to regulators and customers an active and responsible approach to compliance
   • Reduce risk of a regulatory breach
   • Train or retrain employees only as necessary
4. Knowledge Management - How well is knowledge being obtained, retained and actively shared? Key Benefits:
   • Understand what is known and by whom
   • Evaluate knowledge management efficacy (how much is being codified and shared)
   • Improve knowledge codification
   • Improve knowledge networks

   To find out more about us or to discuss your requirements please Contact us.