‘Assessment refers to measuring learner performance (either before or after a teaching intervention, or both). Thus, assessment can be part of an evaluation, but assessment and evaluation aren’t synonymous.’
Educause - Measuring Success: Evaluation Strategies for Distance Education.
The article above suggests that, in order to evaluate the success of your (e-assessment) programme, you may first need to define what you mean by ‘success’. Can success be measured in terms of higher uptake, for example because you’re able to reach a wider group of learners using online assessment? Is it related to higher retention rates, better learner results or higher learner satisfaction? If e-assessment is to work effectively, meeting the needs of all users, including teaching professionals and support staff, as well as learners, each of these factors will probably be relevant. Therefore you may need a comprehensive and carefully planned evaluation plan.
Your evaluation plan could include regular surveys to capture information on learner results, staff views and technical and procedural issues. Gathering this information should allow you to monitor how well e-assessment does or doesn’t contribute to supporting your centre’s objectives, and so could also inform future plans.
You may be able to use your centre VLE to survey learners and centre staff, or one of the wide range of online survey tools available for creating questionnaires, delivering them and analyzing the responses.
Another potential evaluation tool is the ‘Interview’, which can be used instead of, or to supplement, a questionnaire. Whichever approach you use, you’ll need to decide on the areas you wish to evaluate. The Educause article suggests that most areas of interest fall under inputs (resources, personnel etc), outcomes (performance, attitudes etc) or implementation issues. Subcomponents of the implementation issues, for example, on which to focus evaluative questions, could include:
This resource from Oklahoma State University (2002) offers advice on designing questions for both questionnaires and interviews -
Questionnaire and Interview as Data-Gathering Tools
As mentioned elsewhere, good e-testing systems will have Teacher Reporting facilities to allow you to monitor learner attainment in tests taken across different groups of learners and over time, as well as identifying which areas of the tests present problems for learners. These systems should be able to generate reports about the performance of individual learners, a class, or a number of classes.
Your centre will be involved at some level in coordinating, or participating in, internal and external quality assurance processes, designed to improving standards in the centre, and in education generally. These processes may require monitoring, tracking or benchmarking of learners’ progress, and could take place, for example, through:
If learners are using e-assessment, you may be able to gather data from the recommendations, findings or comments resulting from these quality processes which could contribute to your evaluation of the e-assessment provision.