Validation can be confusing - not least because the term is used in different ways in Australian Vocational Education and Training (VET), and because there are a number of different types of validation activities that may be carried out by an RTO.
Here is some clarity on some of the differences.
Pre-use (pre-implementation) validation
Quality Assurance process
Before use
Answers the questions:
Will this assessment tool collect all evidence required?
Is this tool suitable for use?
Will this assessment tool do as it says it will?
This is an analysis of the materials before they are put into use, and is carried out to verify or confirm requirements will be fulfilled.
Pre-use validation might include a validation of mapping to confirm coverage against the unit evidence requirements, and a validation of the assessment tool, which is a check that the instruments have been written to collect the right type of evidence as required by the training package/unit of competency.
Validation of the assessment tool also confirms that the assessment tool has been designed to allow for the provision of sufficient evidence that is valid, reliable, authentic, and current.
Validation of the mapping document - when done well - shows what gaps (if any) exist in the assessment tool compared to what evidence it should be collecting to meet unit requirements. It also confirms the accuracy of the mapping document as a mechanism to evidence how the assessment tool collects required evidence. That is, which questions/tasks in the assessment collect evidence against which parts of the unit. An accurate and precise mapping document is the foundation for quality assessment.
Whilst it is best practice to validate materials to confirm they are fit-for-purpose, pre-use validation is not mandated by the Standards for RTOs 2015.
However, it has been included as a requirement in the unit of competency TAEASS413 - Participate in assessment validation which was released as part of the most recent TAE Training Package update. It is also a practice included in the draft revised Standards for RTOs and likely to remain in the final updated version due for implementation 1.1.2025.
For RTOs based in Western Australia, note that the state-based regulator TAC, expects that all assessment resources, whether purchased or designed internally, must be validated prior to use in accordance with the Principles of Assessment and Rules of Evidence.
Assessment review
Quality Check process
In use
Answers the questions:
Is this assessment tool working as anticipated?
Is it doing as it said it would?
Do we need to make any adjustments?
The assessment review process would typically take place on materials already in the system. It would be looking to confirm the same things as a pre-use validation activity as well as confirming that assessments are fulfilling requirements.
Assessment review type of validation is not mandated by the Standards for RTOs 2015.
Post-implementation validation (Formal validation)
Quality Review process
After use
Answers the questions:
Did this tool allow for consistent, valid judgements to be made on assessment evidence?
Did the RTO's assessment system allow for consistent, reliable use of the tool?
This validation activity is the formal process as required by the Standards for RTOs 2015. This a quality review process that every RTO must undertake.
The aim of formal validation is to confirm that your RTO's assessment system can consistently produce valid assessment judgments.
It is typically conducted after assessment is complete - hence post-implementation validation.
Within this process, validators are to consider the validity of both assessment practices and assessment judgements (how the assessment has been conducted/the assessment system and how the assessments have been marked). It is the latter which can be confused with moderation - more on that below.
In a post-implementation validation, checks are done on the assessment tools to confirm they have produced valid, reliable, sufficient, current and authentic evidence—evidence that has allowed an RTO to make reasonable judgements about whether training product requirements have been met. Hence, it makes sense to have tools validated at pre-use stage and/or in an assessment review activity to catch any issues before getting to formal validation stage.
The Standards also require the following when it comes to formal validation:
A systematic process in place (a validation plan)
A plan which covers specific quantities over set time-frames (five-year cycle to validate all training products on an RTO's scope; 50% to be completed within the first three years of any five-year cycle)
Details within the plan as to what will be validated when, and by who, and how any actions will be recorded and acted upon
Sampling and statistically valid sample sizes to be used
Improvements to be documented and acted upon - including any identified rectification requirements (again, here is a good point for why pre-use validation is so valuable)
(From the Big Book of kNOw - Best Practice Validation of Assessment)
RTOs must comply with all requirements associated with post-implementation validation.
Further information about this can be viewed at:
ASQA's Fact Sheet - Conducting Validation
Clauses 1.9, 1.10 and 1.11 of the Standards (Link to ASQA's User's Guide provided here)
Post-implementation validation is mandated by the Standards for RTOs 2015.
Comentários