Assessment Design

 

June 2011

ASSESSING A UNIT OF STUDY

Introduction

The Moodle course ‘Discover’ is a short, 7-week information literacy course for adult learners. It sits outside national standards or other accreditation processes, however, it has the potential to become part of formal education or workplace training programs, awarding learners credits for successful performance. Information literacy is fundamental to 21st century learning, life and work. It has various definitions, including one coined by The Prague Declaration (2003) which is upheld by UNESCO.

The ‘Discover’ course is fully online, with the first week as a familiarisation module, and the successive six weeks as study modules. The study modules are sequential and each module aims to develop learner proficiency in a particular skill of information literacy.

Each module incorporates formative assessment to monitor learning progression and summative assessment to measure skill proficiency. Hence, assessment is based wholly on coursework, rather than on an end-of-course event. Literature reports that “students consider coursework to be fairer than exams and to measure a greater range of abilities than exams” (Gibbs & Simpson, 2004, p. 7).

The rest of this article outlines the assessment strategy for one module of the ‘Discover’ course.

Assessment Focus

The third formal unit of study (taken in week four) covers the skill of ‘Evaluating Websites’. The learning outcomes and assessments for this unit are:

Learning Goal:
It is expected that by the end of the week, learners will have developed an understanding of the importance of evaluating websites and have learned how to select authentic and credible information.

Assessed Learning Outcome:
Demonstrate the ability to review and select website information that is credible (authoritative), reliable (verifiable), free of bias and current.

Assessment of Learning Outcome:

    1. Formative Assessment: Complete a group project to develop criteria for evaluating websites and then apply the criteria to a website of choice. Post the group’s report to a discussion forum.
    2. Formative Assessment: Participate in discussion forums for the week, including the forum for group reports (for motivation purposes, participation in forums is graded at 5% of the total grade for the course).
    3. Summative Assessment: Complete an online quiz as individual work, to demonstrate knowledge and skill in evaluating websites (quiz score is graded at 15% of the total grade for the course).

Quiz Format:
The quiz comprises 12 questions; a mix of multiple choice, matching, short answer and essay questions. It is a timed quiz (45 mins) and partially auto-assessed. There is automatic grading of multiple choice, matching and short answer questions. The course facilitator will manually grade essay-type questions as the essay questions require responses in the form of short explanations supported by evidence. There is pre-programmed feedback on multiple choice questions. The course facilitator can add manual feedback to matching and essay questions.

Rationale

Garrison & Anderson (2003) stated that feedback is valuable to learning and should be ongoing, frequent and comprehensive (cited in Jenkins, 2004). Accordingly, the ‘Discover’ course aims to provide learners with feedback in each and every unit of study.

The third unit on ‘Evaluating Websites’ enables feedback through formative assessment activities. Literature states that formative assessment allows learners to “put forward their initial ideas and understanding for ‘critique’ and sharing” in a safe environment (Jenkins, 2004, p. 74). Formative assessment which is not accompanied by a score increases the likelihood of learners attending to the feedback (Gibbs & Simpson, 2004).

The formative assessment tasks for the ‘Evaluating Websites’ unit comprise discussion forums and a group project. Discussion forums help to create what Garrison & Anderson (2003) term a “critical community of inquiry” (cited in Jenkins, 2004, p. 74) and they also enable “reflective styles of learning” (Jenkins, 2004, p. 69). The group project can facilitate these same benefits, but also offer:

    a) social, collaborative learning
    b) inquiry-based learning
    c) the opportunity for learners to interpret criteria (Brown, 2004, p. 85)
    d) peer assessment (Gibbs & Simpson, 2004, p. 20)
    e) high expectations for learning (Chickering & Gamson, 1987)

The actual task design for the group project is one that follows the precedent of the University of Gloucestershire where “students work in small groups on a negotiated assignment to produce a short report” (Jenkins, 2004, p. 75). In the case of ‘Discover’, however, learners are required to use materials provided on Moodle (and any other relevant resources) to develop and agree a set of criteria for evaluating information on websites. The criteria is formatted as a table. The group then evaluates an actual website using their criteria and compiles a short report of evaluation findings. Both the table of evaluation criteria and the report of evidence of applying the criteria are published to a discussion forum titled ‘Presentation Forum’. Each group’s work receives feedback from peers and a tutor.

This group project has been designed not only to enable feedback to learners, but to enable learners to develop deeper-level understanding of the evaluation criteria to use to evaluate information on websites before sitting a summative assessment (quiz).

The summative assessment (computer-based quiz) at the end of the unit of study is designed to make marking efficient and avoid tying tutors up in hours of marking. It also enables learners to receive relatively quick notice on their proficiency in the course, due to the auto-grading and pre-programmed feedback options of computer-aided assessment and the additional prompt, manual grading and feedback of explanatory-type answers by tutors. It is important to note that the purpose of summative assessment is to “measure” learner proficiency. There will be some feedback from the assessment, but the primary purpose of the quiz is to quickly and efficiently measure learning performance. The course facilitator can then identify whether there are any ‘at risk’ learners, and learners themselves will know whether they are meeting the performance expectations for the course. This is important in a course where sequential acquisition of knowledge and skills is desired.

In conclusion, the assessment strategy for the third unit of study for the ‘Discover’ course ensures that learners are provided with ongoing, frequent and comprehensive feedback on their learning. The formative assessment tasks such as discussion forums and a group project will be “developmental” to the learning process, whereas the summative quiz at the end of the unit is “judgmental” to the learning process (Brown, 2004, p. 84).

Personal Reflection on this Assignment

As this week has been one where I have not participated in the MET forums, I may have missed opportunities to refine my understanding of assessment. However, I have developed assessment tasks within my Moodle course, and have articulated my rationale for the design of these tasks to the best of my ability. It is interesting that not all questions that are auto-graded in the Moodle quiz tool can be pre-programmed with feedback, and a specific example is matching-type questions.

From experience in Moodle quizzes previously, the manual marking and grading can be done quickly by tutors, thus providing responsiveness of assessment results to learners.

The readings this week have enabled me to reflect on some assessment practices that I have been involved with in the past, particularly in business communication and ESL domains. I have learned from this week’s readings to be careful about combining scoring rubrics with feedback rubrics, as students may look only at the scores and disregard feedback, or if the score is not to their liking, may disregard the feedback as negative and unhelpful. I will be mindful when asking learners to do peer reviews or other types of pair or group work to encourage critiques but not scoring. However, in some situations of assessment design, feedback and scoring can go together, as “assessments can be both formative and summative” (Jenkins, 2004, p. 67). Overall, in order to facilitate learning, assessment should be designed to be as “safe” as possible for learners.

References

http://dis.shef.ac.uk/literacy/definitions.htm#ala

Brown, S. (2004). Assessment for Learning. Learning and Teaching in Higher Education, 2004-2005 (1), pp. 81-88.

Chickering, A. W., & Gamson, Z.F. (1987). Seven Principles for Good Practice in Undergraduate Education. Retrieved from http://www.aahea.org/bulletins/articles/sevenprinciples1987.htm

Gibbs, G., & Simpson, C. (2004). Conditions under which Assessment Supports Students’ Learning. Learning and Teaching in Higher Education, 2004-2005 (1), pp. 3-31.

Jenkins, M. (2004). Unfilled Promise: formative assessment using computer-aided assessment. Learning and Teaching in Higher Education, 2004-2005 (1), pp. 67-78

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.
%d bloggers like this: