Machine-gradable quizzes in emacs+org-modex

Posted November 23, 2014 at 04:49 PM | categories: emacs, org, education | tags:

In a previous post , we considered interactive quizzes in emacs and org-mode. Here we consider a variation of that idea with the aim of creating a machine-gradable quiz, i.e. a modern version of a scantron.

The idea is simple, we will let each question be represented by an org headline, and the choices will be clickable links that store the answer as a property in the headline. Each headline will have a unique id. The grading key will contain these ids and the correct answer, and a function will determine if the right answer was selected for each question.

Here is a simple link that will store the value of the link as a property. Note that the path must be at least two characters long to be recognized as a link, unless you wrap the link in double brackets. We will have the link insert a comment to indicate to the user what they chose. We do that because the PROPERTIES drawer is usually hidden, and it is not obvious it was changed. 

(org-add-link-type 
 "mc" 
 (lambda (link)
   (save-restriction
     (org-narrow-to-subtree)
     (goto-char (point-max))
     (insert (concat
              (unless (bolp) "\n")
              (format "# you chose %s" link))))
       
   (org-entry-put (point) "ANSWER" link)))

Next, we add a subheading with some questions to test the link.

1 Some questions

1.1 question 1

What is 2 + 2?

a 1

b 2

c 4

1.2 question 2

What is 2 - 2?

a 0

b 2

c 4

2 Grading

We will store an alist of id and answer for each problem. To grade, we simple map over the alist, go to the section with the id, and compare the answers. When the answer is correct, we save a point, and when not, no point. We can use the org-mode machinery to jump to the problems and get the stored answer. We put some feedback at the end of the file to see what was right, and what was wrong. 

(let* ((key '(("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c")
              ("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a")))
       (MAX (length key))
       (points 0)
       (answer))
  
  (dolist (tup key)
    (save-excursion
      (org-open-link-from-string
       (format "id:%s" (car tup)))
      (setq answer (org-entry-get (point) "ANSWER"))
      (if (string= answer (cdr tup))
          (progn
            (setq points (+ 1 points))
            (goto-char (point-max))
            (insert (format "# id:%s: %s correct\n" (car tup) answer)))
        (goto-char (point-max))
        (insert (format "# id:%s: %s wrong (%s is correct)\n"
                        (car tup)
                        answer
                        (cdr tup))))))
  (goto-char (point-max))
  (insert (format
           "#+GRADE: %s" (/ (float points) (float MAX)))))

That works pretty well. I need to think about how to codify the key, since this would usually be stored in some file. We would also need to wrap the code block in a function that we could call easily. The org-id key is easy, but not very readable. It would make it easy to keep a database of these problems though.

Just for completeness, I want to save the key to a file, and use it. We simply write the alist in a file. Here are the contents, which are tangled to key.el. One alternative might be to have a solution copy of the quiz which has the answers in it, and we read the answers from the file. 

(("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c")
 ("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a"))

Now, we read it in like this. The rest of the code is basically the same. 

(let* ((key (with-temp-buffer 
              (insert-file-contents "key.el")
              (read (current-buffer))))
       (MAX (length key))
       (points 0)
       (answer))
  
  (dolist (tup key)
    (save-excursion
      (org-open-link-from-string
       (format "id:%s" (car tup)))
      (setq answer (org-entry-get (point) "ANSWER"))
      (if (string= answer (cdr tup))
          (progn
            (setq points (+ 1 points))
            (goto-char (point-max))
            (insert (format "# id:%s: %s correct\n" (car tup) answer)))
        (goto-char (point-max))
        (insert (format "# id:%s: %s wrong (%s is correct)\n"
                        (car tup)
                        answer
                        (cdr tup))))))
  (goto-char (point-max))
  (insert (format
           "#+GRADE: %s" (/ (float points) (float MAX)))))

It is probably much easier to have a solution version of the quiz, and generate the key from it. For example, we can collect the ID and ANSWER from the problems in this file like this. 

(let ((key '()))
  (org-map-entries
   (lambda ()
     (let ((id) (ans))
       (when (and
              (setq id (org-entry-get (point) "ID"))
              (setq ans (org-entry-get (point) "ANSWER")))
         (add-to-list 'key (cons id ans))))))
key)

(("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a")
 ("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c"))

So, if we had a master solution file, we could read the key from there. That is the way to do this.

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.7c

Discuss on Twitter