The Kitchin Research Group

Commenting in org-files

Fri, 24 Apr 2015 12:41:39 EDT

There was an interesting discussion on the org-mode mail list about putting comments in org files. Eric Fraga suggested using inline tasks, and customizing the export of them so they make a footnote, or use the todonotes package (suggested by Marcin Borkowski). Here is Eric's export. A big advantage of this is integration with the Agenda, so you can see what there is todo in your document.

  (setq org-inlinetask-export-templates
        '((latex "%s\\footnote{%s\\\\ %s}\\marginpar{\\fbox{\\thefootnote}}"
                 '((unless
                       (eq todo "")
                     (format "\\fbox{\\textsc{%s%s}}" todo priority))
                   heading content))))

Eric Abrahamsen suggested an idea to use a link syntax. I like the idea a lot, so here we develop some ideas. A link has two parts, the path, and description. A simple comment would just be a simple link, probably in double square brackets so you can have spaces in your comment. COMMENT It might be feasible to use COMMENT the description to "mark text" that the comment refers to. The remaining question is what functionality should our link have when you click on it, and how to export it. For functionality, a click will show the comment in the minibuffer and offer to delete it. For export, for now we will make it export with todonotes in LaTeX, and as a red COMMENT with a tooltip in html. To use this, you need to have the LaTeX package todonotes included in your org file.

Here is our comment link.

(org-add-link-type
 "comment"
 (lambda (linkstring)
   (let ((elm (org-element-context))
         (use-dialog-box nil))
     (when (y-or-n-p "Delete comment? ")
       (setf (buffer-substring
              (org-element-property :begin elm)
              (org-element-property :end elm))
             (cond
              ((org-element-property :contents-begin elm)
               (buffer-substring
                (org-element-property :contents-begin elm)
                (org-element-property :contents-end elm)))
              (t
               ""))))))
 (lambda (keyword desc format)
   (cond
    ((eq format 'html)
     (format "<font color=\"red\"><abbr title=\"%s\" color=\"red\">COMMENT</abbr></font> %s" keyword (or desc "")))
    ((eq format 'latex)
     (format "\\todo{%s}{%s}" keyword (or desc ""))))))

It would be convenient to have a quick function for adding a comment to some highlighted text.

(defun add-comment (begin end)
  (interactive "r")
  (if (region-active-p)
      (let ((selected-text (buffer-substring begin end)))
        (setf (buffer-substring begin end)
              (format "[[comment:%s][%s]]"
                      (read-input "Comment: ") selected-text)))
  (insert (format  "[[comment:%s]]" (read-input "Comment: ")))))

Test 1: COMMENT

COMMENT Test 2

That is it. I could see a few other enhancements that might be very useful, e.g. a command to list all the comments, remove all the comments, etc… I am pretty satisfied with this for now though.

1 An updated approach to comments.

Rainer asked about making some comments inline. It would be nice if a single link syntax could accommodate both styles of comments. I previously developed an approach to extend links with attributes (http://kitchingroup.cheme.cmu.edu/blog/2015/02/05/Extending-the-org-mode-link-syntax-with-attributes/ ), which I will reuse here for that purpose. The idea is to add an ":inline" attribute to change the export behavior. We only modify the LaTeX export here.

(org-add-link-type
 "comment"
 ;;  follow function
(lambda (linkstring)
   (let ((elm (org-element-context))
         (use-dialog-box nil))
     (when (y-or-n-p "Delete comment? ")
       (setf (buffer-substring
              (org-element-property :begin elm)
              (org-element-property :end elm))
             (cond
              ((org-element-property :contents-begin elm)
               (buffer-substring
                (org-element-property :contents-begin elm)
                (org-element-property :contents-end elm)))
              (t
               ""))))))
 ;; format function
 (lambda (path description format)
   (let* ((data (read (concat "(" path ")")))
          (head (car data))
          (plist (cdr data)))
     (cond
      ((eq format 'html)
       (format "<font color=\"red\"><abbr title=\"%s\" color=\"red\">COMMENT</abbr></font> %s" path (or description "")))
      ((eq format 'latex)
       (format "\\todo%s{%s}%s"
               (if (-contains? data :inline) "[inline]" "")
               (mapconcat (lambda (s)
                            (format "%s" s))
                          (-remove-item :inline data) " ")
               (if description (format "{%s}" description) "")))))))

Here are some examples of the syntax:

[[comment: :inline the rest of your text]]

[[comment:Some text you want to highlight]]

[[comment:Some text you want to highlight :inline]]

It doesn't matter where the :inline attribute is added. This seems to work pretty well.

We can modify our convenience function to allow us to use a prefix arg to make the comment inline. Here is one way to do it.

(defun add-comment (begin end &optional arg)
  "Comment the region. With a prefix ARG, make the comment inline."
  (interactive (list (region-beginning)
                     (region-end)
                     current-prefix-arg))
  (let ((inline (if arg ":inline " "")))
        (if (region-active-p)
            (let ((selected-text (buffer-substring begin end)))
              (setf (buffer-substring begin end)
                    (format
                     "[[comment:%s%s][%s]]"
                     inline
                     (read-input "Comment: ") selected-text)))
          (insert (format
                   "[[comment:%s%s]]"
                   inline
                   (read-input "Comment: "))))))

add-comment

Test COMMENT text to COMMENT comment on.

org-mode source

Org-mode version = 8.2.10

Machine-gradable quizzes in emacs+org-modex

Sun, 23 Nov 2014 16:49:52 EST

In a previous post , we considered interactive quizzes in emacs and org-mode. Here we consider a variation of that idea with the aim of creating a machine-gradable quiz, i.e. a modern version of a scantron.

The idea is simple, we will let each question be represented by an org headline, and the choices will be clickable links that store the answer as a property in the headline. Each headline will have a unique id. The grading key will contain these ids and the correct answer, and a function will determine if the right answer was selected for each question.

Here is a simple link that will store the value of the link as a property. Note that the path must be at least two characters long to be recognized as a link, unless you wrap the link in double brackets. We will have the link insert a comment to indicate to the user what they chose. We do that because the PROPERTIES drawer is usually hidden, and it is not obvious it was changed.

(org-add-link-type 
 "mc" 
 (lambda (link)
   (save-restriction
     (org-narrow-to-subtree)
     (goto-char (point-max))
     (insert (concat
              (unless (bolp) "\n")
              (format "# you chose %s" link))))
       
   (org-entry-put (point) "ANSWER" link)))

Next, we add a subheading with some questions to test the link.

1 Some questions

1.1 question 1

What is 2 + 2?

a 1

b 2

c 4

1.2 question 2

What is 2 - 2?

a 0

b 2

c 4

2 Grading

We will store an alist of id and answer for each problem. To grade, we simple map over the alist, go to the section with the id, and compare the answers. When the answer is correct, we save a point, and when not, no point. We can use the org-mode machinery to jump to the problems and get the stored answer. We put some feedback at the end of the file to see what was right, and what was wrong.

(let* ((key '(("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c")
              ("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a")))
       (MAX (length key))
       (points 0)
       (answer))
  
  (dolist (tup key)
    (save-excursion
      (org-open-link-from-string
       (format "id:%s" (car tup)))
      (setq answer (org-entry-get (point) "ANSWER"))
      (if (string= answer (cdr tup))
          (progn
            (setq points (+ 1 points))
            (goto-char (point-max))
            (insert (format "# id:%s: %s correct\n" (car tup) answer)))
        (goto-char (point-max))
        (insert (format "# id:%s: %s wrong (%s is correct)\n"
                        (car tup)
                        answer
                        (cdr tup))))))
  (goto-char (point-max))
  (insert (format
           "#+GRADE: %s" (/ (float points) (float MAX)))))

That works pretty well. I need to think about how to codify the key, since this would usually be stored in some file. We would also need to wrap the code block in a function that we could call easily. The org-id key is easy, but not very readable. It would make it easy to keep a database of these problems though.

Just for completeness, I want to save the key to a file, and use it. We simply write the alist in a file. Here are the contents, which are tangled to key.el. One alternative might be to have a solution copy of the quiz which has the answers in it, and we read the answers from the file.

(("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c")
 ("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a"))

Now, we read it in like this. The rest of the code is basically the same.

(let* ((key (with-temp-buffer 
              (insert-file-contents "key.el")
              (read (current-buffer))))
       (MAX (length key))
       (points 0)
       (answer))
  
  (dolist (tup key)
    (save-excursion
      (org-open-link-from-string
       (format "id:%s" (car tup)))
      (setq answer (org-entry-get (point) "ANSWER"))
      (if (string= answer (cdr tup))
          (progn
            (setq points (+ 1 points))
            (goto-char (point-max))
            (insert (format "# id:%s: %s correct\n" (car tup) answer)))
        (goto-char (point-max))
        (insert (format "# id:%s: %s wrong (%s is correct)\n"
                        (car tup)
                        answer
                        (cdr tup))))))
  (goto-char (point-max))
  (insert (format
           "#+GRADE: %s" (/ (float points) (float MAX)))))

It is probably much easier to have a solution version of the quiz, and generate the key from it. For example, we can collect the ID and ANSWER from the problems in this file like this.

(let ((key '()))
  (org-map-entries
   (lambda ()
     (let ((id) (ans))
       (when (and
              (setq id (org-entry-get (point) "ID"))
              (setq ans (org-entry-get (point) "ANSWER")))
         (add-to-list 'key (cons id ans))))))
key)

(("38FCCF3D-7FC5-49BF-BB77-486BBAA17CD9" . "a")
 ("19C7BA30-A761-4C94-9F3B-E6010E263949" . "c"))

So, if we had a master solution file, we could read the key from there. That is the way to do this.

org-mode source

Org-mode version = 8.2.7c

A hint system for problems in org-mode

Sun, 09 Nov 2014 09:55:17 EST

I use org-mode to write problems for classes that I teach. Sometimes it is helpful to be able to provide hints about aspects of the problem. I have used drawers for that before. Here I will look at another approach. The idea is to store a unique id (org-id) in the problem headline. We will make hints somewhere else, and each hint will store the id they refer to in some property. We will run a command in the problem that finds the hints, and offers a menu you can select from.

In the next sections, we define a Problem statement, a section with hints in it, and finally code that defines a hint function.

1 Problem statement

What is the seventh number in the Fibonacci series?

2 Hints

Hints will be stored in headings somewhere. Here we put them in this file, but they could be stored in another file too. We give each hint a HINT property, with the problem id as the value. Here are three hints. In real example, the hints would not be this easy to find in the document. You might store them somewhere else, in another file for example.

2.1 What is the Fibonacci series?

The $i^th number in the Fibonacci series is equal to the sum of the previous two numbers in the series.

2.2 What does the series start with?

The Fibonacci series starts with 1.

2.3 Example of the series.

The Fibonacci series goes as 1, 1, 2, 3, 5, 8, …

3 The hint code

We want to get the id from the problem the point is in, and then find hints for the problem. Then, we construct a menu and prompt the user to select a hint. I use a number to select the hint because it was easier to generate the menu that way. I like characters better, because you just have to press a key. With numbers you type the number and press enter. We open a new buffer with the contents of the hint in it. You can close the buffer by pressing q.

(defun hint ()
  "Present a menu of hints for the problem at point"
  (interactive)
  (let ((id (org-entry-get (point) "ID"))
        (entries '())
        (menu "")
        choice)

    (unless id
      (error "No problem ID found"))

    (org-map-entries
     (lambda ()
       (save-restriction
         (org-narrow-to-subtree)
         (add-to-list 'entries
                      (cons
                       (elt (org-heading-components) 4)
                       (buffer-string))
                      t)))
     (format "HINT=\"%s\"" id))

    ;; generate menu string
    (dolist (i (number-sequence 1 (length entries)))
      (setq menu (concat menu (format "[%s] %s\n" (- i 1)
                                      (car (elt entries (- i 1)))))))

    (setq choice (elt entries (read-number (concat menu "Your choice: ") 0)))
    ;; this feels a little clunky. Maybe I could just save a marker to
    ;; the headline, and open it in a narrowed indirect buffer.
    (when choice
      (switch-to-buffer "*hint*")
      (erase-buffer)
      (insert (cdr choice))
      
      (org-mode)
      (show-subtree)
      (setq buffer-read-only t)
      (use-local-map (copy-keymap org-mode-map))
      (local-set-key "q" #'(lambda () (interactive) (kill-buffer)))
      )))

hint

4 Summary

This seems like an interesting way to provide hints, or related information in org-mode. You could also consider using tags, or more sophisticated code to determine what else is relevant. For example, you might keep track of some performance metric, and use some heuristic algorithm that ranks the related information. Or perhaps fuzzy text searching, or combinations of criteria. If the number of hits got large, then the menu approach here might not be the best one. Then, something like the occur interface might be more suitable.

org-mode source

Org-mode version = 8.2.7c

A context-sensitive file link menu in org-mode

Sat, 08 Nov 2014 10:24:14 EST

I am still interested in various ways to get more functionality of org-links. For example, we looked at:

enabling right clicks on links
new links with menus

When you click on a link, the function org-open-at-point runs, which is a large function that does a lot of things. One of them is to check if the link is defined in org-link-protocols, and to run the function definition there if it is. Here is a list of links defined for me. I defined a lot of these in org-ref, and my own init files, so you may not see these on your system.

(mapcar 'car org-link-protocols)

google

ResearcherID

orcid

message

mac-outlook

skim

addressbook

x-together-item

rmail

mhe

irc

info

gnus

docview

bibtex

bbdb

ans

exercise

solution

assignment

doi

bibentry

Autocites

autocites

supercites

Textcites

textcites

Smartcites

smartcites

footcitetexts

footcites

Parencites

parencites

Cites

cites

fnotecite

Pnotecite

pnotecite

Notecite

notecite

footfullcite

fullcite

citeurl

citedate*

citedate

citetitle*

citetitle

Citeauthor*

Autocite*

autocite*

Autocite

autocite

supercite

parencite*

cite*

Smartcite

smartcite

Textcite

textcite

footcitetext

footcite

Parencite

parencite

Cite

Citeauthor

Citealp

Citealt

Citep

Citet

citeyear*

citeyear

citeauthor*

citeauthor

citetext

citenum

citealp*

citealp

citealt*

citealt

citep*

citep

citet*

citet

nocite

cite

eqref

nameref

pageref

ref

label

list-of-tables

list-of-figures

addbibresource

bibliographystyle

printbibliography

nobibliography

bibliography

pydoc

index

attachfile

msx

file+emacs

file+sys

Interestingly, file links are not defined in org-link-protocols, they are handled separately. I would like to change the behavior of file+emacs links. Instead of just opening the file, I want a menu to give me the option to create the file if it does not exist, and to open it in emacs, or with a system program if the file does exist. Let us see what this link does.

(assoc "file+emacs" org-link-protocols)

file+emacs

org-open-file-with-emacs

nil

When you click on the link, it runs org-open-file-with-emacs, and there is no formatting function defined.

So, let us define a list of functions that could make a menu. A new variation we use in this post is that each element of the list will be a (key menu-name action-func visible-p) list. visible-p will be a function that determines if the function is listed in the menu. That way, our menu will be context specific.

We want an option to create a file if it does not exist, and if it does exist, a choice to open in emacs, or a system program. So the idea here is to create the menu in a variable (so it easy to add to later), then when you click on the link it will run a menu function that filters the functions to run, and then prompt you for a selection.

(defvar file+emacs-menu '()
  "list of menu entries. (key name action visible).
key is a character to select.
name is what shows in the menu as [key]name
action is a function that takes a path
visible is a function that determines if the entry is in the menu.")

(setq file+emacs-menu
      '(("c" "reate"
         find-file ; action function
         (lambda (x) (not (file-exists-p x)))) ; visible-p
        ("o" "pen"
         org-open-file-with-emacs
         (lambda (x) (file-exists-p x)))
        ("e" "xternal open"
         (lambda (x) (org-open-file path '(16)))
         (lambda (x) (file-exists-p x)))))


(defun file+emacs-menu (path)
  "menu command for file+emacs links"
  (interactive)
  (let* ((filtered-menu-list (-filter
                              (lambda (x) (funcall (car (last x)) path))
                              file+emacs-menu))
         (menu-string (concat
                       (mapconcat
                        (lambda (tup)
                          (concat "[" (elt tup 0) "]"
                                  (elt tup 1) " "))
                        filtered-menu-list
                        "") ": "))
         (input (read-char-exclusive menu-string nil 1))
         (selected-func (and
                         input
                         (elt
                          (assoc
                           (char-to-string input) filtered-menu-list)
                          2))))
    (when selected-func
      (funcall selected-func path))))

file+emacs-menu

Now we need to change the link definition in org-link-protocols. setf comes to the rescue. We just get the whole entry, and then setf the second position in it like this.

(setf (elt (assoc "file+emacs" org-link-protocols) 1)
  'file+emacs-menu)

file+emacs-menu

Here we just confirm we set it.

(assoc "file+emacs" org-link-protocols)

file+emacs

file+emacs-menu

nil

Now, when we click on these links, we get our context specific menu. When

This file exists: ase-db.org so we see this menu:

This file does not exist: test.noext So we see:

For these, we can select to open them in a pdf reader or MS Word from our new menu. attaching-code-blocks-to-a-pdf.pdf

org-to-word.docx

I admit this example was a little contrived. You can do most of these things with prefix commands, or more specific commands in emacs. But, I rarely remember those. I would have preferred to use the file link in this example, but it is not defined in org-link-protocols, so this style of modification would not work, and I did not want to add it to org-link-protocols just to show how to change it this way.

This general approach would be very useful for links where there may be multiple contexts or actions that make sense. For file links, you may want do different things if the file already exists, or if it does not exist. As another example, my doi link gives me a menu to:

open in https://doi.org
open the doi in Web of Science
find citing articles in Web of Science
search the doi in Google Scholar
open the doi in CrossRef
open the doi in Pubmed
find the doi in my bibtex file
get a bibtex entry for the doi

I get all that from a click! org-ref offers similar functionality for cite links, where you might want to do different things from a click:

See preview of the citation
open the bibtex entry
open the pdf if you have it
open the url for the entry
any of the things I listed for the doi example above.

I am sure there are many other things that might be useful to do!

org-mode source

Org-mode version = 8.2.7c

A generalized org-link with an extendable menu of actions

Sun, 02 Nov 2014 19:33:14 EST

In org-ref the cite links are actually a menu of options. Here I want to explore generalizing the concept, and enable users to add new menu items. When you click on a cite link, it calls a function that constructs the menu as a string. Then, we read something from the minibuffer to say what should be done. We could store these things in variables, so that a user could add their own things. We need a list of characters, menu strings, and functions to run when to create the menu. Here we put together a little example code that shows the idea.

(defun option-a (link)
  (message "You picked option A for %s" link))

(defun option-b (link)
  (message "You picked option B for %s" link))

(defvar menu-funcs
 '(("a" "Option 1" option-a)
   ("b" "Option 2" option-b)))

(defun menu-click (path)
  (message
   (concat
    (mapconcat
     (lambda (tup)
       (concat "[" (elt tup 0) "]"
               (elt tup 1) " "))
     menu-funcs "") ": "))
  (setq input (read-char-exclusive))
  (funcall
   (elt 
    (assoc
     (char-to-string input) menu-funcs)
    2)
   path))

(org-add-link-type
 "menu"
 'menu-click)

element Clicking on that link in Emacs gives me the desired menu!

and

Now, we test adding a new function.

(defun option-c (link)
  (message "You picked option C for %s" link))

(add-to-list 'menu-funcs
 '("c" "Option C" option-c))

c	Option C	option-c
a	Option 1	option-a
b	Option 2	option-b

Well, that worked fine! You might want to append to the list instead of put it at the beginning, but that is really your choice.

(defun option-4 (link)
  (message "You picked option 4 for %s" link))

(add-to-list 'menu-funcs
 '("4" "Option 4" option-4) t)

c	Option C	option-c
a	Option 1	option-a
b	Option 2	option-b
4	Option 4	option-4

I think this example more or less shows the basic idea here. These external functions may do a variety of things, like look up something on google, or some other web service, search for something on your hard drive, etc… For example in org-ref, clicking on a cite link gives you options to open the bibtex file, a pdf, a url or notes. This would allow you to open other things too, if you felt like it. For better or worse, you can modify the behavior of the link after it is defined.

org-mode source

Org-mode version = 8.2.7c

Org-mode is awesome

Sat, 12 Jul 2014 16:12:48 EDT

I made a video on Youtube (https://www.youtube.com/watch?v=fgizHHd7nOo ) demonstrating how we use Emacs and org-mode. The source for that file can be found here .

There are some other good org-mode videos out there. Here are few:

org-mode source

Org-mode version = 8.2.6