Bespoke Assessment Design
CompAssess has developed an extensive library of assessment
exercises from which to draw. These exercises form the back
bone of most our assessment solutions.
Our proven expertise in assessment design has also equipped
us with the capability to develop bespoke assessment designs
for our clients, applying the same development rigour that
applies to the development of CompAssess designs.
The CompAssess Design Process
Our approach to assessment design is to ensure a clear separation
of roles between assessment design and reviewing and critiquing.
In our experience this ensures efficient project management
and the highest quality output.
The CompAssess assessment design process operates as follows.
» Click
here to view assessment design process diagram.
- Review Target Behaviours
The first step in assessment design process is a thorough
review of the targeted competencies and associated definitions,
and behavioural indicators.
From this we are able to complete a draft competency coverage
grid, showing the link between each competency and planned
exercise, as well as demonstrating that each competency
is well covered by appropriate assessment exercises
- Scenario Development
The next step in the CompAssess design process is to
generate face valid and content valid scenarios that will
provide realistic stimuli and triggers for the targeted
behavioural indicators.
CompAssess then 'pressure tests' the scenarios to ensure
that they will provide sufficient stimuli for participants
to demonstrate all the required behavioural indicators for
each competency.
The culmination of this step is a brief synopsis of the
exercise scenarios, supported by notes detailing the contextual
behaviours that will be stimulated against each of the behavioural
indicators.
These provide the context or background for each exercise.
- Draft Exercise Design
Our assessment design team drafts the content for each
exercise before submitting them for scrutiny.
This process may be partially iterative with the design
team making adjustments and amendments as requested until
all parties agree the exercise is ready for piloting.
- Assessment Material Development
This important step guides the preparation of the additional
assessment material that requires drafting before the exercise
may be piloted.
Assessment materials requiring drafting at this step include:
- Background Notes
These provide the context for the design and in some
instances are provided to participants a week prior
to a assessment/development centre. They set the scene
for the role the participant is adopting and provide
background information on the fictitious organisation
the participant will 'work for' for the duration of
the assessment centre.
- Role-player Instructions (where appropriate)
Role-player instructions provide succinct guidance regarding
the behavioural indicators that are being targeted by
the exercise, details of the roles being played in the
exercise, general guidance on how to role-play effectively
and exercise-specific guidance as to how to respond
effectively in the different situations that may arise
within the given scenario, depending on the approach
taken by the participant.
- Participant Instructions
Participant instructions provide all the required information
for the participant to successfully complete the exercise.
Obviously, this information is exercise specific and
differs depending on the exercise format.
- Evaluation Forms
Exercise specific evaluation forms are developed, guiding
the assessors by detailing the effective and ineffective
behaviours that may be displayed against each of the
competency behavioural indicators.
- Assessor Notes
CompAssess also develops supplementary assessor notes,
as a companion to the evaluation forms. Well designed
and exercise specific evaluation forms reduce the reliance
on assessor notes. We do, however, provide additional
background information, and more extensive detail regarding
the range of possible approaches that may be taken by
a participant in each exercise and guidance as to how
assess each approach.
All the above documentation is reviewed, critiqued and approved
before moving forward to piloting.
- Piloting, Editing and Calibration
The Piloting Process
The piloting of exercises represents a critical step
in the assessment design process.
Our approach is to pilot all exercises internally before
presenting them for final review, minimising the probability
of exercise failure or the need for substantial re-work.
We pilot each exercise with a "naive" participant,
that is, one who has had no previous exposure to any of
the assessment materials. The role-playing and assessing
is completed by individuals who have not been previously
involved in the project, and who therefore bring no vested
design interest and less subjectivity to the pilot.
Editing
On completion of the piloting process, adjustments and
edits are completed.
Calibration
Where parallel exercises are required, the calibration
of the exercises for equal opportunity to demonstrate target
behavioural indicators and degree of difficulty is critical.
Calibration of exercises is measured in three key areas:
- Face validity - perceptions of difficulty and comparability
- Reliability - comparable levels of rater accuracy
in the assessment process
- Performance - comparable levels of degree of difficulty,
such that a participant will achieve the same score
irrespective of which exercise they complete
Our approach to determining the comparability and inter-changeability
of parallel exercises is to pilot them as previously discussed
but with the same participant applying the same effort in
both exercises.
The debrief process then covers:
- Participant reactions to different exercise content,
with an emphasis on any differences between the two
- Assessor reactions to usability of assessor evaluation
forms and notes for both exercises, with an emphasis
on any differences between the two
- Inter-rater reliabilities on both exercises, with
an emphasis on where there are discrepancies and why
- Differing participant performance across the two exercises,
with an emphasis on understanding the differences between
the two performances, and any associated disparities
in opportunities to demonstrate effective behaviour
against the target behavioural indicators.
- Finalisation of Materials / Assessor Training
The final step in the process is to complete a proof of
the documentation for presentation and typographical errors
before dispatching the materials for printing.
If applicable, an Assessor Training session also takes place.
This is most important in order to ensure the CompAssess
and/or client assessors are familiar with the exercises,
their role as assessors/role players and are able to conduct
accurate evaluations of the exercises. Data integration
and feedback to participants is also canvassed through assessor
training.
» View
Client Case Study
» Contact CompAssess
to learn more about our Assessment Design Services.
|
