Collaborative Testing: An Opportunity for Student-centered Feedback & Peer Learning

Presenter First Name: 
Mark
Presenter Last Name: 
Terrell, EdD
Year of Presentation: 
2011

 

Presentation Objectives

  1. Describe the SoTL Model as a process for conducting educational research in anatomy
  2. Demonstrate the implementation of the SoTL model through a case study: Collaborative Assessment

The Concept of SoTL

  1. Scholarship of Teaching and Learning
  2. Investigate teaching practices by engaging deeply into an evidence-based analysis of how students learn effectively

Characteristics of all forms of Scholarship:   

  1. Clear goals & objectives
  2. Adequate preparation
  3. Appropriate methods
  4. Significant results
  5. Dissemination of results
  6. Critique & evaluation

SoTL as a bridge between research & teaching
Three levels of the teaching endeavor:

Basic Teaching – routine teaching & testing of students  

  • Often stagnant and repetitive- “Faculty teach the way they were taught”

Scholarly Teaching – a subset of teachers who continually improve their own teaching by being informed:

  • Pedagogical knowledge of Education Literature
  • Feedback
  • So, when theory & practice come together, teaching becomes………. scholarly!         

Scholarship of Teaching & Learning (SoTL) –

  • A subset of scholarly teachers who engage in educational research   
  • Becomes a bridge    


The SoTL Process Model
1.Innovation Phase

  • To address a teaching problem or issue
  • Grounded in the educational literature
  • Implementation

2.Evaluation Phase

  • Study design
  • Statistical Analysis

3.Dissemination Phase

  • Key product to peer-review
  • To impact teaching beyond one’s own classroom

The Process: The Scientific-Method Approach:

  1. Observation
  2. Question
  3. Hypothesis
  4. Study Design & Experimentation
  5. Analysis & Conclusion

Case Study: Collaborative Testing
Observation: the Anatomy Classroom
“ What is the most common approach a student uses to learn human anatomy?”

  • Memorizing – passive, lowest level of processing; study in isolation

Assessment: Lecture tests are summative - traditionally measure how much students have learned
Research Question:
Note: Question must be well framed:

  • Examine the pertinent literature in education
  • Sharpen & refine broad, vague questions into specific, well-defined, answerable questions

a.Memorization:     How can we promote deeper, long-lasting learning?
b.Isolation:  How can we promote interaction & positive collaborative relationships among students?

  • PEW Health Professions Commission (2000) -  developed 21 competencies needed by future health-care professionals: #15 = work in interdisciplinary teams
  • Institute of Medicine (2001) – developed a 5-step plan for creating a strong health-care system; must collaborate in order to care for the patient; collaborative working relationships among health professionals are vital in ensuring quality patient care.
  • Level of Educational Interactivity between the student and: a) Professor b) Course content c) Peers

Usability of Collaborative Learning Practices

  • Must actively engage learners around central issues
  • Improves student knowledge, enhances critical thinking, & promotes teamwork skills

Usage of cooperative learning?

  • frequently used in labs
  • limited in large lectures
  • rare in assessment practices

c.Assessment: How can we transform a summative evaluative measure of student performance into a formative learning tool?
The most powerful single modification that enhances achievement is feedback  (Hattie, 2009):

  • Feedback should be “corrective” in nature.
  • Feedback should be timely.
  • Feedback should be specific.
  • Students can effectively provide some of their own feedback


Refined Research Question:
•Can student learning be improved by adding a collaborative feedback component to a traditional lecture exam in an anatomy course?

Hypotheses
By discussing exam questions in small groups, students will:
a) learn from each other, transforming the exam into a formative learning tool.
b) provide immediate feedback on questions they missed and the reasons for their errors.
c) perform higher on comprehensive final exam.
d) Increase interactivity & decrease isolation

Experimentation
Course:  Undergraduate Human Anatomy; large lecture hall; 400 students / semester
Examinations: 5 objective-type lecture exams

  • Exam 1 = Individual effort (control)
  • Exam 2 = Individual + Group (combined) effort
  • Exam 3 = Individual + Group (combined) effort
  • Exam 4 = Group effort
  • Final exam = Individual effort (control)

Combined format: 80 question were reduced to 40 questions to be taken twice
Duration of study: 4 semesters = 1,600 students
Data analysis: Quantitative
Compared scores between:

  • individual and group scores on the same exam
  • final exams for semesters using and not using collaborative testing
  • sections of the final exam corresponding to content tested & not tested collaboratively
  • correlation analyses between similar questions missed on lecture exams and final exam

Qualitative Data Analysis: Student-completed questionnaires containing to measure perceptions of collaboration:

  • Likert scale
  • Open-ended questions
  •  
  • Results
  • A. Empirical Analyses
  • Combined format - Individual vs. Group scores:
  • Mean individual test score = 76%
  • Mean group test score = 95% (P < 0.001)

Final exam - Long-term knowledge retention:

  • Mean final exam scores were 4.6% higher during semesters that used collaborative assessment (P < 0.01).
  • Differential sections on the final corresponding to collaborative assessment were 7.1% higher than     sections using non-collaborative assessment (P < 0.01).
  • Correlation between similar missed questions on unit exams and final is significantly lower (r = 0.58) when collaborative lecture exams are used; without collaborative assessment (r = 0.95)

B. Qualitative Analyses
1. “Discussing test questions with my group members increased my understanding of anatomy”
            80% = SA;        15%= A
2. “I studied more for the combined-format tests than if the tests were individual effort only”
            39% = SA;        28% = A
3. “Group testing increased my satisfaction of the    course”
             59% = SA;        29% = A
4. “Group testing increased my testing confidence”
            38% = SA;        45% = A
   
5. “How effectively did your group collaborate?”
            Very well       Well     Adequate    Poorly
                 55%              36%              7%           1%
6. “My preferred assessment technique was:”
         Group tests    Individual tests        Combined
            31%        4%        54%
7. “Group testing increased the likelihood that I would study in a group, rather than by myself”
        very likely        likely        no effect
             70%        18%             8%

Discussion
Improved long-term knowledge retention – higher final exam scores suggest that students:

  • Collaboratively engaged in reasoning and problem solving skills to defend their answers among their peers
    Increased the depth cognitive processing & learning

The combined format:

  • ensured individual preparatory accountability
  • increased motivation & sense of community

Increased the formation of study groups including metacogntive skills of self-monitoring & regulating
Immediate feedback - increased:  

  • student satisfaction
  • learning & reduced concept misconceptions


Conclusion
SoTL provides a vehicle by which to conduct evidence-based, hypothesis-drive research
Student Interactivity & feedback are the most powerful modifications that enhances achievement
Collaborative Testing - assessment approach that applies cooperative learning in order to provide peer-led feedback:

  • Individual exams provided accountability
  • Students learn from peers & the exam
  • Promotes growth & development of the learner both cognitively & collaboration skills
  • Reflects the true collaborative nature
American Association of Anatomists

9650 Rockville Pike Bethesda, Maryland 20814-3998
Tel: 301-634-7910 | Fax: 301-634-7965

 

Copyright © 2014. All rights reserved.           
home ~ contact us