The University is changing its approach to module and programme evaluation and has adopted a new tool to deliver it. This approach is being driven by the PVC (Education). The implementation project is being led by Dr Tony Fawcett (Dean of Education), Andrew Rennie (Head of the Curriculum, Learning and Assessment Service), Dr Malcolm Murray (eLearning Manager) and Julie Mulvey (eXplorance Expert). It draws on the findings of the MEQ Task & Finish Group led by Prof Gordon Love. Where decisions are required they have been obtained from the University Education Committee. Data Transfer has been approved by the University Information Governance Unit.
In the past, the University has used a range of tools to deliver module evaluations, most recently Blackboard’s Enterprise Survey Tool. During 2016-17, the University ran a successful year-long pilot of eXplorance Blue with Durham Business School. At the same time, Education Committee ran a Task & Finish Group looking at the module evaluation process. The result was a desire to produce a common set of questions at the level of the University and then within a Department. This will allow aggregation of the data and better reporting. To this end, in May 2017, the University signed a 5 year deal with the Canadian company Explorance Inc. They are a leading provider of course/module evaluation software with clients from across the globe.
IMPLEMENTATION TIMETABLE
The project began in 2017-18 with end of year undergraduate module evaluations. The eXplorance Blue module evaluation solution has been made available to all departments during the Easter term of 2018 and this tool should be used for all evaluations. After completing Undergradute MEQs, the project will then expand to cover Postgraduate MEQs and programme-level evaluations. Staff who need to create evaluations for interim module evaluations or modules that finish early in the academic year) should use Blackboard’s survey tool and are being advised not the Enterprise survey tool. Once eXplorance Blue takes over the MEQ process fully, the Enterprise Survey tool in Blackboard will become read-only.
Enterprise Surveys: Please note that are are not training any new Evaluation Managers for the Enterprise Survey tool and recommend that if you are still using it then you only allow staff who know how to use the tool to have access.
Blackboard Survey tool: This is a tool that sits within each duo modules and allows a survey to be build and run anonymously. You can copy a survey template from one course into others and instructions can be found on the Blackboard Help area within the module.
THE NEW APPROACH
STRUCTURED QUESTIONS
Standard questions have been defined to provide a more consistent student experience and allow aggregation of the data (so results from modules with low numbers of students can still contribute to a broader, programme-level picture). The question style and responses have been designed to follow those used in the NSS. There was also an acknowledgement that some existing questionnaires were too long, risking a low return rate. Some existing MEQs contained questions that should really be asked earlier in interim evaluations when there is still time to make changes for the current cohort. As such the number of questions has been limited, so that evaluations should be a similar length across the University.
Evaluations will be driven by the data in the student record system. As such we will be evaluating modules by module code, not duo course codes or any other way of aggregating students.
Evaluations are made up of distinct sets of questions. The explanation that follows focuses on the MEQ process. It also applies to programme evaluations – simply replace the word ‘Module’ with ‘Programme’ in the examples below. It uses screen-shots from a bespoke tool (the QuBE) developed by members of the LTT to combine responses from the departments and create the data files used by eXplorance Blue. As such the styling may differ from the final evaluations delivered to students.
THE ANATOMY OF AN EVALUATION
There are some questions that will be standard to all evaluations delivered across the University. These occur in two places in the evaluation – at the start and at the end.
Here are the opening questions: