Collaborative Testing: An Opportunity for Student-centered Feedback & Peer Learning

Presentation Objectives

1. Describe the SoTL Model as a process for conducting educational research in anatomy
2. Demonstrate the implementation of the SoTL model through a case study: Collaborative Assessment

The Concept of SoTL

1. Scholarship of Teaching and Learning
2. Investigate teaching practices by engaging deeply into an evidence-based analysis of how students learn effectively

Characteristics of all forms of Scholarship:   

1. Clear goals & objectives
2. Adequate preparation
3. Appropriate methods
4. Significant results
5. Dissemination of results
6. Critique & evaluation

SoTL as a bridge between research & teaching
Three levels of the teaching endeavor:

Basic Teaching – routine teaching & testing of students  

• Often stagnant and repetitive- “Faculty teach the way they were taught”

Scholarly Teaching – a subset of teachers who continually improve their own teaching by being informed:

• Pedagogical knowledge of Education Literature
• Feedback
• So, when theory & practice come together, teaching becomes………. scholarly!         

Scholarship of Teaching & Learning (SoTL) –

• A subset of scholarly teachers who engage in educational research   
• Becomes a bridge    

The SoTL Process Model
1.Innovation Phase

• To address a teaching problem or issue
• Grounded in the educational literature
• Implementation

2.Evaluation Phase

• Study design
• Statistical Analysis

3.Dissemination Phase

• Key product to peer-review
• To impact teaching beyond one’s own classroom

The Process: The Scientific-Method Approach:

1. Observation
2. Question
3. Hypothesis
4. Study Design & Experimentation
5. Analysis & Conclusion

Case Study: Collaborative Testing
Observation: the Anatomy Classroom
“ What is the most common approach a student uses to learn human anatomy?”

• Memorizing – passive, lowest level of processing; study in isolation

Assessment: Lecture tests are summative - traditionally measure how much students have learned
Research Question:
Note: Question must be well framed:

• Examine the pertinent literature in education
• Sharpen & refine broad, vague questions into specific, well-defined, answerable questions

a.Memorization:     How can we promote deeper, long-lasting learning?
b.Isolation:  How can we promote interaction & positive collaborative relationships among students?

• PEW Health Professions Commission (2000) -  developed 21 competencies needed by future health-care professionals: #15 = work in interdisciplinary teams
• Institute of Medicine (2001) – developed a 5-step plan for creating a strong health-care system; must collaborate in order to care for the patient; collaborative working relationships among health professionals are vital in ensuring quality patient care.
• Level of Educational Interactivity between the student and: a) Professor b) Course content c) Peers

Usability of Collaborative Learning Practices

• Must actively engage learners around central issues
• Improves student knowledge, enhances critical thinking, & promotes teamwork skills

Usage of cooperative learning?

• frequently used in labs
• limited in large lectures
• rare in assessment practices

c.Assessment: How can we transform a summative evaluative measure of student performance into a formative learning tool?
The most powerful single modification that enhances achievement is feedback  (Hattie, 2009):

• Feedback should be “corrective” in nature.
• Feedback should be timely.
• Feedback should be specific.
• Students can effectively provide some of their own feedback

Refined Research Question:
•Can student learning be improved by adding a collaborative feedback component to a traditional lecture exam in an anatomy course?

Hypotheses
By discussing exam questions in small groups, students will:
a) learn from each other, transforming the exam into a formative learning tool.
b) provide immediate feedback on questions they missed and the reasons for their errors.
c) perform higher on comprehensive final exam.
d) Increase interactivity & decrease isolation

Experimentation
Course:  Undergraduate Human Anatomy; large lecture hall; 400 students / semester
Examinations: 5 objective-type lecture exams

• Exam 1 = Individual effort (control)
• Exam 2 = Individual + Group (combined) effort
• Exam 3 = Individual + Group (combined) effort
• Exam 4 = Group effort
• Final exam = Individual effort (control)

Combined format: 80 question were reduced to 40 questions to be taken twice
Duration of study: 4 semesters = 1,600 students
Data analysis: Quantitative
Compared scores between:

• individual and group scores on the same exam
• final exams for semesters using and not using collaborative testing
• sections of the final exam corresponding to content tested & not tested collaboratively
• correlation analyses between similar questions missed on lecture exams and final exam

Qualitative Data Analysis: Student-completed questionnaires containing to measure perceptions of collaboration:

• Likert scale
• Open-ended questions
•  
• Results
• A. Empirical Analyses
• Combined format - Individual vs. Group scores:
• Mean individual test score = 76%
• Mean group test score = 95% (P < 0.001)

Final exam - Long-term knowledge retention:

• Mean final exam scores were 4.6% higher during semesters that used collaborative assessment (P < 0.01).
• Differential sections on the final corresponding to collaborative assessment were 7.1% higher than     sections using non-collaborative assessment (P < 0.01).
• Correlation between similar missed questions on unit exams and final is significantly lower (r = 0.58) when collaborative lecture exams are used; without collaborative assessment (r = 0.95)

B. Qualitative Analyses
1. “Discussing test questions with my group members increased my understanding of anatomy”
            80% = SA;        15%= A
2. “I studied more for the combined-format tests than if the tests were individual effort only”
            39% = SA;        28% = A
3. “Group testing increased my satisfaction of the    course”
             59% = SA;        29% = A
4. “Group testing increased my testing confidence”
            38% = SA;        45% = A
   
5. “How effectively did your group collaborate?”
            Very well       Well     Adequate    Poorly
                 55%              36%              7%           1%
6. “My preferred assessment technique was:”
         Group tests    Individual tests        Combined
            31%        4%        54%
7. “Group testing increased the likelihood that I would study in a group, rather than by myself”
        very likely        likely        no effect
             70%        18%             8%

Discussion
Improved long-term knowledge retention – higher final exam scores suggest that students:

• Collaboratively engaged in reasoning and problem solving skills to defend their answers among their peers
  Increased the depth cognitive processing & learning

The combined format:

• ensured individual preparatory accountability
• increased motivation & sense of community

Increased the formation of study groups including metacogntive skills of self-monitoring & regulating
Immediate feedback - increased:  

• student satisfaction
• learning & reduced concept misconceptions

Conclusion
SoTL provides a vehicle by which to conduct evidence-based, hypothesis-drive research
Student Interactivity & feedback are the most powerful modifications that enhances achievement
Collaborative Testing - assessment approach that applies cooperative learning in order to provide peer-led feedback:

• Individual exams provided accountability
• Students learn from peers & the exam
• Promotes growth & development of the learner both cognitively & collaboration skills
• Reflects the true collaborative nature