In an effort to find the best use of limited teaching assistant’s time in today’s economy, a study reported at last year’s ASEE 2010 conference by two of the authors of this paper indicated that there is no statistically significant difference in student performance when homework is collected and graded versus assigned and not graded. Student performance was measured via a multiple-choice final examination. The study was conducted in a Numerical Methods course at a large university in the southeast of USA over a period of three years encompassing data from over 300 hundred Mechanical Engineering students.
Because of the use of course management systems such as Blackboard (http://blackboard.com) in most universities and the availability of quiz makers such as Respondus (http://respondus.com), we wanted to determine if using graded online quizzes would improve the student performance, while minimizing grading time. Although other online homework and grading systems such as WebAssign (http://www.webassign.net/) are available, there are substantial recurring costs associated with the use of such systems both for students and faculty, and at this time availability of engineering topics in these systems is very limited.
As a pilot study, using Respondus, we developed endless online quizzes (EOQ) for the three subtopics of Simultaneous Linear Equations topic taught in the class – Background of Matrix Algebra, Gaussian Elimination and LU Decomposition. All quizzes had 6-7 questions and were of algorithmic form. This form allows the instructor to choose some or all variables to take values within a range, and develop a formula for the correct answer. These formulas were developed by writing scripts with symbolic computation in MATLAB and Maple.
When a student takes the quiz, the system randomly chooses the values of the selected variables, and he/she answers the question by filling in the answer field. The student’s answer is checked against the correct value. Feedback, including the answer and the correctness of the answer, is given immediately.
The students were given a limited amount of time (10 days from the start of the first subtopic) but unlimited attempts to complete all the three quizzes. The number of attempts, and the time taken and score for each attempt was automatically recorded by Blackboard.
Student performance was evaluated by the number of correct answers in the Simultaneous Linear Equations portion of the multiple-choice final examination (total of 4 questions out of a 32-question test. One-tailed t-test comparing the performance of students in the 2009 spring/summer terms (mean=2.72, variance=0.87) with those in the 2010 spring/summer terms (mean=2.89, variance=0.88) showed a significant improvement; t(232)=1.428 and p=0.08. In 2009, homework was assigned from the book but not picked for a grade, while in 2010, we assigned and graded (3% of overall grade) the online quizzes for the topic of Simultaneous Linear Equations.
Focus groups were also conducted about the experience with the EOQ. Overall, the students indicated that they really enjoyed working with EOQ for several reasons. First, there was no time limit and hence there was no stress or pressure to complete the problems. This allowed the students to stop and take a break or think through what they needed to do to complete the problem. Second, the students indicated that EOQ were helpful in preparing them for their exams, allowed them to study more than they would have without EOQ, and enjoyed getting the immediate feedback to help with “the little things”. Students also responded to a set of questions that asked how (or if) EOQ had helped them with understanding their coursework. The students indicated that the questions in EOQ helped them identify what they knew and what they did not know.