There are many things RTOs need to monitor and look into once granted registration—the annual declaration, AVETMISS reporting, marketing compliance, etc. Amidst the long list of things to monitor stands the task most RTOs dread: validation.

We have written several articles about validation, but let’s pedal back and unlock the term. ASQA defines validation as a quality review of the assessment process.

In other words, it is the process of confirming which areas or aspects of an RTO’s assessment process are done right and identifying which elements can be improved. And with the correct understanding of its key components, validation won’t be too intimidating a task.

Clause 1.8 of the SRTOs 2015 states that RTOs need to make sure their assessment systems, including RPL, are compliant with the requirements of the training package and “is conducted in accordance with the Principles of Assessment and Rules of Evidence.”

The standards require us to conduct two types of validation.

The first type of assessment validation ensures that your RTO’s assessment complies with the assessment requirements of the training package in your scope.

The next ensures that you conduct your assessment following the principles of assessment and the rules of evidence.

So, this gives us an idea that we validate before we do the assessment and after we do the assessment. This article will focus on the first type of validation—assessment tool validation.

 

What are the Two Types of Assessment Validation?

assessment validation process - two types of validation

As mentioned earlier and in one of our previous blogs (How to Validate Assessment Tools), validation is split into two processes: (1) assessment tool validation and (2) post-assessment validation. 

On one hand, assessment tool validation, also called pre-assessment validation or verification, relates more to the first part of the clause, which discusses ensuring that all unit requirements are addressed. It is focused on the idea that all workbooks are 100% compliant.

Post-Assessment Validation, on the other hand, relates more to the implementation side of things. Registered Training Organisations must ensure that assessment is conducted in accordance with the Principles of Assessment and Rules of Evidence.

For this article, we will be focusing on assessment tool validation.

 

How do you Conduct Assessment Tool Validation?

rto's assessment system must include valid assessment judgements and assessment decisions aided by assessment tool validation

Now that we’ve reviewed the difference between the two types of validation, let’s get to the nitty gritty of assessment tool validation.

When do you conduct assessment tool validation?

Assessment tool validation aims to ensure all elements, performance criteria, and performance and knowledge evidence are addressed by your assessment tools.

This means anytime you purchase new learning resources, you conduct assessment tool validation before you let your student use the resources.

You don’t need to wait for the next validation schedule in your 5-year cycle. When you purchase a new set of resources, you validate them immediately because you need to ensure they’re all good before letting your students use them.

However, this is not the only reason to conduct this type of validation. You can also conduct assessment tool validation when you:

  • update your resources
  • add new training products on scope
  • review your course against training product updates
  • identify your learning resources as a risk during your risk assessment

Since the Australian Skills Quality Authority applies a risk-based approach to regulating RTOs, they also expect RTOs to conduct risk assessments regularly. So, if you receive complaints from your students about your learning resources, this would be the perfect time to conduct assessment tool validation.

Which training products will you validate?

Again, remember that this type of validation aims to ensure all learning resources are compliant before use. All RTOs are expected to validate all resources for each unit.

What Resources do you Need to Get Started on your Assessment Tool Validation?

validation outcomes may depend on the resources youre willing to use for your validation activities and assessment practices

Learning Resources

Given that you’re validating your assessment tools, you would need the full suite of your learning resources:

  1. Mapping tool – this is the first document you would need to investigate. This shows you which assessment items address which unit requirements.  This document will help you validate faster.
  2. Learner/student workbook – when you conduct assessment tool validation, you also need to check if the student workbook is suitable to be used as an assessment tool. Are the instructions to the students clear? Do the answer fields provide enough space for the students to supply their answers? This is actually a common gap.
  3. Assessor guide/marking guide – aside from checking instructions for students, you also need to check if there are sufficient instructions for the assessor and clear benchmarks for each assessment item. Having clear benchmarks for assessors to follow is vital in achieving reliable assessment outcomes.
  4. Other related resources – could be checklists, registers, and templates developed separately from the workbook and marking guide. Validate them to ensure they are appropriate for the assessment task and address unit requirements.

Validation panel

Clause 1.11 outlines the requirements for members of the validation panel. From the clause, you can see that validation can be done by one or more persons. However, RTOs normally require all trainers and assessors to attend. Some even invite industry experts.

Collectively, your validation panel must have:

  • Vocational competencies and current industry skills relevant to the unit being validated
  • Current knowledge and skills in vocational teaching and learning
  • Either of the following training and assessment credentials:
  • TAE40116 Certificate IV in Training and Assessment or its successor, or
  • TAE40110 Certificate IV in Training and Assessment, and one of the following:
  • TAELLN411 Address adult language, literacy and numeracy skills or its successor or
  • TAELLN401A Address adult language, literacy and numeracy skills

and one of the following:

  • TAEASS502 Design and develop assessment tools or its successor or
  • TAEASS502A Design and develop assessment tools or
  • TAEASS502B Design and develop assessment tools.
  • TAEASS00011 Assessor Skill Set or its successor, or
  • TAEASS00001 Assessor Skill set, and one of the following:
  • TAEASS502 Design and develop assessment tools or its successor or
  • TAEASS502A Design and develop assessment tools or
  • TAEASS502B Design and develop assessment tools.

Validation tool/template

Having a validation tool helps you with both the validation process and documentation. Using a validation tool makes it easier to look at how each assessment item maps against each unit requirement.

At the same time, it can serve as your document evidence that you have validated your resources before letting the students use them.

ASQA doesn’t have a recommended or required template for assessment tool validation, but heaps of assessment tool validation templates are available online. Most of these tools have the validators look at the tools as a whole and determine if they meet the principles of assessment.

Principles of Assessment Yes/No/Partially Comments
1. Fair
2. Flexible
3. Valid
4. Reliable

 

And while validation templates like these are easier to accomplish, it’s easy to make errors in judgement since you practically have no space to comment on each assessment item.

We highly recommend using a more detailed template to inspect each unit requirement and the assessment items that map against them. Below is an example:

Element Performance Criteria Assessment Instructions Benchmarks Assessment Instrument Rectification Recommendations
1.

1.1

1.2

 

What do you Need to Check?

quality control process should follow the principles of assessment to be guided on what to check

As we discussed in our blog post Common Problems In Assessment Tools, you must ensure that your assessment tools allow your trainers to adhere to assessment principles and evidence rules.

Principles of Assessment

  1. Fairness – Does everyone have equal opportunity and access to the assessment process?
  2. Flexibility – Does the assessment provide different options to demonstrate competence according to different needs and preferences?
  3. Validity – Does the assessment assess what it is meant to assess? Is it a valid tool for assessing the required skill or knowledge?
  4. Reliability – Will the assessment achieve the same results every time, regardless of who conducts the training? Will different assessors make the same decision on whether someone has a skill or not?

Rules of Evidence

  1. Validity – Does the evidence show that the candidate has the skills, knowledge and attributes described in the module or unit of competency and associated assessment requirements?
  2. Sufficiency – Is there enough evidence to ensure that the learner has the skills and knowledge required?
  3. Authenticity – Does the assessment tool prove that the work is the candidate’s own?
  4. Currency – Are the assessment tools based on current units of competency and up-to-date industry practice?

Despite these being constantly addressed in VET-related professional development and nationally recognised training, heaps of tools still have problems addressing these requirements.

To avoid using learning resources that leave some unit requirements unaddressed, make sure to follow these guides:

  1. Walk the talk

Pay attention to the verbs used in the unit requirements and ensure they are addressed by the assessment item. For example, in the unit CHCECE032 Nurture babies and toddlers, one of its performance evidence requirements requires students to:

Complete each of the following at least once with two different babies under 12 months of age in a safe environment, using age-appropriate verbal and non-verbal communication and according to service and regulatory requirements:

  • change nappies
  • prepare bottle, bottle feed babies and clean equipment
  • prepare solid food and feed babies
  • respond appropriately to baby signs and cues
  • prepare and settle babies for sleep
  • monitor and encourage age-appropriate physical exploration and gross motor skills

Having the students describe the process of changing nappies of babies under 12 months of age does not directly address the unit requirement. So unless the unit requirement aims to assess underpinning knowledge (i.e. knowledge evidence), your students should be walking the talk.

  1. Watch out for the s!

Pay attention to the numbers. In our example on one of the unit requirements of CHCECE032, this single unit requirement calls for the students to complete the tasks at least once on two different babies under 12 months of age. Having students complete the tasks listed twice on just 1 baby won’t cut it.

  1. All or NC

Pay attention to lists. Again, in our example above, if students are asked to perform just half of the tasks listed, this is non-compliant. Each assessment item must address all requirements, or it’s not yet competent for the student and non-compliant for the assessment tool.

  1. Can you be more specific?

Each assessment item must have clear and specific benchmark answers to guide the assessor’s judgement on the student’s competence. With this in mind, it’s also important that your instructions do not confuse your students and assessors. For example:

What kind of information can be included in a work package?

The answer may include:

  • Required resources
  • Applicable costs
  • Activity duration
  • Assigned roles and responsibilities

If an assessment item calls for multiple answers, specify the number of answers a student needs to provide. This way, your assessment can be conducted reliably, and the evidence you collect will be valid.

The same can be said for assessment items with double-barrelled questions or questions that simultaneously ask for more than one answer from the student. This can confuse both the student and the assessor, as in the sample question below:

Name a hazard and/or environmental issue in the work area and choose the most effective hazard control hierarchy.

Answers may include, but are not limited to:

  • Weather conditions – isolation of work area, engineering, PPE
  • Work area and ground conditions – elimination, isolation, engineering
  • People – isolation, engineering, administration
  • Structural hazards – substitution, isolation, engineering
  • Chemical hazards – isolation, engineering, administration
  • Equipment or machinery – isolation, engineering, administration

Avoiding double-barrelled questions makes it easier for students to answer and for assessors to accurately judge a student’s competence.

Looking at these requirements, you might think, “That’s such a lot of work. Don’t learning resource developers have audit guarantees?” But the thing with these audit guarantees is that you’d have to wait to be audited first before they help you rectify instances of noncompliance auditors would find in your resources. This automatically tacks onto your compliance history, so it’s infinitely better to take the safe and compliant route.