Org-ref uses timers to give you messages about the cite link at point. I am not so crazy about the timer, there is always a (short) delay, and I have had trouble debugging timers in the past, and you have to put the point on the link. Since I wrote that code, I have learned some new things about Emacs, including dynamic tooltips. This will allow me to use the mouse to see what a cite link refers to. While reading documents, I am more likely to use a mouse than when typing a document, and getting a tooltip by hovering sounds like a good idea.

Here, we explore using dynamic tooltips on cite links. The idea is pretty simple, we tie into font-lock to add a function to the :help-echo property of a cite link. The function will go to point, and compute the citation string at point, which will be displayed as a tooltip when the mouse hovers over the citation.

Font-lock allows you to specify a function that sets match-data and that can have other side-effects, e.g. setting text properties. Org-ref has a regexp that defines cite links, which we use here, and a function that gets the citation string at point. We just go to the mouse position, and get that string, wrapped in a save-excursion macro so that point does not actually move. Then, we add the function to font-lock keywords, and we are done!

Here are some papers we wrote on using org-mode kitchin-2015-examp,kitchin-2015-data-surfac-scien and some other references in my bibliography zou-2014-cobal-embed,zlotea-2014-nanoal and one final example zhu-2015.

Here is the short code required to do this. You can see the tooltips in action here: https://www.youtube.com/watch?v=ifSmlId2rk0

(defun org-ref-match-next-cite-link (&optional limit)
  (when (re-search-forward org-ref-cite-re limit t)
    (add-text-properties
     (match-beginning 0) (match-end 0)
     (list
      'help-echo (lambda (window object position)
                   (save-excursion
                     (goto-char position)
                     (let ((s (org-ref-get-citation-string-at-point)))
                       (with-temp-buffer
                         (insert s)
                         (fill-paragraph)
                         (buffer-string)))))))))

; do this for this buffer
(font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)
(font-lock-fontify-buffer)

;; do this for every org file
(add-hook
 'org-mode-hook
 (lambda ()
   (font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)))

Bibliography

• [kitchin-2015-data-surfac-scien] "John Kitchin", Data Sharing in Surface Science, "Surface Science ", (0), - (2015). link. doi.
• [kitchin-2015-examp] Kitchin, Examples of Effective Data Sharing in Scientific Publishing, ACS Catalysis, 5(6), 3894-3899 (2015). link. doi.
• [zhu-2015] Yinlong Zhu, Wei Zhou, Zhi-Gang Chen, Yubo Chen, , Chao Su, Moses Tad\'e & Zongping Shao, \ceSrNb_0.1Co_0.7Fe_0.2 O_3-$\delta$ Perovskite As a Next-Generation Electrocatalyst for Oxygen Evolution in Alkaline Solution, Angew. Chem. Int. Ed., 54(13), 3897-3901 (2015). link. doi.
• [zlotea-2014-nanoal] Zlotea, Morfin, Nguyen, Nguyen, , Nelayah, Ricolleau, Latroche & Piccolo, Nanoalloying Bulk-Immiscible Iridium and Palladium Inhibits Hydride Formation and Promotes Catalytic Performances, Nanoscale, 6(17), 9955 (2014). link. doi.
• [zou-2014-cobal-embed] Zou, Huang, Goswami, , Silva, Sathe, Mikmekov\'a, ska & Asefa, Cobalt-Embedded Nitrogen-Rich Carbon Nanotubes Efficiently Catalyze Hydrogen Evolution Reaction At All pH Values, Angewandte Chemie, , 4461-4465 (2014). link. doi.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

The documentation of functions in emacs allows you to put some light markup into function doc strings that will render as the key sequence required to run the command when you look up the help on the function. I would like to have something like that in org-mode. You can look up the key-binding to a command like this:

(substitute-command-keys "\\[org-agenda]")

C-c a

We are going to explore a way to recognize the syntax shown above, change its appearance to alert us that we are looking at an emacs command, add a tooltip, and make it clickable to open the documentation, and s (super) clickable to find the function code. Font lock is the tool we will use for this. Basically, we need a regular expression to match the syntax, and a function to find the next instance, and put some properties on the matched text.

I made a video (https://www.youtube.com/watch?v=VLUMW0sR4Vk ) showing what this post is all about. It isn't easy to see in the post ☺.

Here we use the `rx' library to build up a regular expression for this. It is a bit easier to document than a raw regexp. Since we are matching \ in the pattern, there are some obligatory escaping \ characters in there too. All we need is to integrate this into font-lock. We define a function that will move the point to the end of the next match, and put properties on the match. We will go ahead and make the text clickable so we can access documentation and code easily. The tooltip will show the key-binding to run the command.

(require 'rx)

(defvar elisp-symbol-keybinding-re
  (rx
   ;; opening \\[
   (eval "\\[")
   ;; one or more characters that are not ]
   (group (one-or-more (not (any "]"))))
   ;; The closing ]
   "]")
"Regexp for an elisp command keybinding syntax. \\[some-command]
Regexp group 1 matches `some-command'.")

(defun match-next-keybinding (&optional limit)
  "Move point to the end of the next expression matching
`elisp-symbol-keybinding-re', and put properties on the match
that shows the key sequence. Non-bound commands are not
fontified."
  (when (and (re-search-forward
              elisp-symbol-keybinding-re
              limit t)
             (fboundp (intern (match-string 1))))
    (let* ((beg (match-beginning 0))
           (end (match-end 0))
           (s (match-string 0))
           (command (match-string 1))
           (describe-func `(lambda ()
                    "Run `describe-function' on the command."
                    (interactive)
                    (describe-function (intern ,command))))
           (find-func `(lambda ()
                     "Run `find-function' on the command."
                     (interactive)
                     (find-function (intern ,command))))
           (map (make-sparse-keymap)))

      ;; this is what gets run when you click on it.
      (define-key map [mouse-1] describe-func)
      (define-key map [s-mouse-1] find-func)
      ;; Here we define the text properties
      (add-text-properties
       beg end
       `(local-map ,map
         mouse-face highlight
         help-echo ,(format
                     "%s\n\nClick for documentation.\ns-mouse-1 to find function."
                     (substitute-command-keys s))
         keybinding t)))))

Let's go ahead and make syntax for `some-command' too. This one seems simple enough we just write a regexp for it.

(defun match-next-emacs-command (&optional limit)
  "Move point to the end of the next expression matching
`this-syntax', and put a tooltip on the match
that shows the key sequence. Works on commands and variables."
  (when (and (re-search-forward
              "`\\([^']+\\)'"
              limit t)
             (or (boundp (intern (match-string 1)))
                 (fboundp (intern (match-string 1)))))
    (let* ((beg (match-beginning 0))
           (end (match-end 0))
           (s (match-string 0))
           (command (match-string 1))
           (describe-func
            `(lambda ()
               "Run `describe-function/variable' on the command."
               (interactive)
               (cond ((fboundp (intern ,command))
                      (describe-function (intern ,command)))
                     ((boundp (intern ,command))
                      (describe-variable (intern ,command))))))
           (find-func `(lambda ()
                     "Run `find-function' on the command."
                     (interactive)
                     (find-function (intern ,command))))
           (map (make-sparse-keymap)))

      ;; this is what gets run when you click on it.
      (define-key map [mouse-1] describe-func)
      (define-key map [s-mouse-1] find-func)
      ;; Here we define the text properties
      (add-text-properties
       beg end
       `(local-map ,map
         mouse-face highlight
         help-echo ,(format
                     "%s\n\nClick for documentation.%s"
                     (if (fboundp (intern command))
                         (substitute-command-keys (format "\\[%s]" command))
                       "Variable")
                     (if (fboundp (intern command))
                         "\ns-mouse-1 to find function."
                       ""))
         keybinding t)))))

Now we need a way to turn them on and off. We do that here with a minor mode.

(define-minor-mode emacs-keybinding-command-tooltip-mode
  "Fontify on emacs keybinding syntax. Adds a tooltip for
keybinding, and make the command clickable to get to the
documentation."
  :lighter " KB"
  (if emacs-keybinding-command-tooltip-mode
      ;; turn them on
      (font-lock-add-keywords
       nil
       '((match-next-keybinding 1 font-lock-constant-face)
         (match-next-emacs-command 1 font-lock-constant-face)))
    ;; turn them off
    (font-lock-remove-keywords
     nil
     '((match-next-keybinding 1 font-lock-constant-face)
       (match-next-emacs-command 1 font-lock-constant-face))))
  (font-lock-fontify-buffer))

Here we turn it on:

(emacs-keybinding-command-tooltip-mode -1)

Here are some sample uses. You can use \\[org-toggle-latex-overlays] to toggle latex overlays.

You can use \\[org-ref-helm-insert-cite-link] to insert citations.

That more or less does it! I don't know if this is the canonical way to do this, but it works nicely here. You can also use overlays, but I found them a little confusing because they are not editable, and you have to toggle the minor mode to see them. Here we have unobtrusive tooltips. One downside is these won't export in any fashion in org-mode since it is not part of the syntax. It might be a good idea to adjust `font-lock-extra-managed-props' for this

It works for this syntax too: `helm', which is also commonly used in doc strings. This should be pretty handy in org-mode documents about Emacs!

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

org-mode has a lot of pre-defined entities (see http://kitchingroup.cheme.cmu.edu/blog/2013/10/03/Exporting-accented-characters-to-latex-from-org-mode/ ), otherwise known to me as non-ascii characters. I rarely remember what these are, and occasionally want to insert the LaTeX or HTML code, so here we build a helm command to show them to me, and allow me to select one for insertion. We generate the helm sources from org-entities below. It works pretty well!

(defun helm-insert-org-entity ()
  "Helm interface to insert an entity from `org-entities'.
F1 inserts utf-8 character
F2 inserts entity code
F3 inserts LaTeX code (does not wrap in math-mode)
F4 inserts HTML code"
  (interactive)
  (helm :sources (reverse
                  (let ((sources '())
                        toplevel
                        secondlevel)
                    (dolist (element (append
                                      '("* User" "** User entities")
                                      org-entities-user org-entities))
                      (when (and (stringp element)
                                 (s-starts-with? "* " element))
                        (setq toplevel element))
                      (when (and (stringp element)
                                 (s-starts-with? "** " element))
                        (setq secondlevel element)
                        (add-to-list
                         'sources
                         `((name . ,(concat
                                     toplevel
                                     (replace-regexp-in-string
                                      "\\*\\*" " - " secondlevel)))
                           (candidates . nil)
                           (action . (("insert utf-8 char" . (lambda (candidate)
                                                               (insert (nth 6 candidate))))
                                      ("insert org entity" . (lambda (candidate)
                                                           (insert (concat "\\" (car candidate)))))
                                      ("insert latex" . (lambda (candidate)
                                                          (insert (nth 1 candidate))))
                                      ("insert html" . (lambda (candidate)
                                                         (insert (nth 3 candidate)))))))))
                      (when (and element (listp element))
                        (setf (cdr (assoc 'candidates (car sources)))
                              (append
                               (cdr (assoc 'candidates (car sources)))
                               (list (cons
                                      (format "%10s %s" (nth 6 element) element)
                                      element))))))
                    sources))))

helm-insert-org-entity

Now I can write things like the particle was 60 Å in diameter at a temperature of 600°C, leading to an expansion coefficient of α=0.2 ± 0.01. It isn't quite as fast as knowing the keyboard shortcuts for those symbols, but a lot faster than looking them up then copy and pasting them. So far it seems like these export to HTML and LaTeX just fine, and they are more convenient and better looking than using the org-entities codes. This will make its way into jmax soon.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

If you run long Python blocks from org-mode, you might want to keep working while it runs. Currently Emacs gets blocked and you have to wait patiently. In this post we consider some ways to avoid this that run our code asynchronously, but still put results where they belong in the org-buffer.

This is a long post. You may want to see the video: https://www.youtube.com/watch?v=VDyoN8yipSE , or skip to the end where the best and final version is shown.

(require 'async)

(defun org-babel-async-execute ()
  "Run a python block at point asynchrously."
  (interactive)

  (let ((current-file (buffer-file-name))
        (uuid (org-id-uuid))
        (temporary-file-directory "./")
        (tempfile (make-temp-file "py-")))

    (org-babel-tangle '(4) tempfile)
    (org-babel-remove-result)
    (save-excursion
      (re-search-forward "#\\+END_SRC")
      (insert (format
               "\n\n#+RESULTS: %s\n: %s"
               (or (org-element-property :name (org-element-context))
                   "")
               uuid)))

    (async-start
     ;; what to start
     `(lambda ()
        ;; now we run the command then cleanup
        (prog1
            (shell-command-to-string (format "python %s" ,tempfile))
          (delete-file ,tempfile)))

     `(lambda (result)
        "Code that runs when the async function finishes."
        (save-window-excursion
          (save-excursion
            (save-restriction
              (with-current-buffer (find-file-noselect ,current-file)
                (goto-char (point-min))
                (re-search-forward ,uuid)
                (beginning-of-line)
                (kill-line)
                (insert (mapconcat
                         (lambda (x)
                           (format ": %s" x))
                         (butlast (s-split "\n" result))
                         "\n"))))))))))

org-babel-async-execute

print 'hello world'
import time
time.sleep(5)

import os
print os.getcwd()
print time.asctime()

hello world
/Users/jkitchin/blogofile-jkitchin.github.com/_blog
Fri Nov 20 10:17:53 2015

(defun org-babel-async-execute ()
  (interactive)
  (let* ((current-file (buffer-file-name))
        (uuid (org-id-uuid))
        (temporary-file-directory "./")
        (tempfile (make-temp-file "py-"))
        (pbuffer (format "*%s*" uuid))
        process)

    (org-babel-tangle '(4) tempfile)
    (org-babel-remove-result)

    (save-excursion
      (re-search-forward "#\\+END_SRC")
      (insert (format
               "\n\n#+RESULTS: %s\n: %s"
               (or (org-element-property :name (org-element-context))
                   "")
               uuid)))

    (setq process (start-process
                   uuid
                   pbuffer
                   "python"
                   tempfile))

    (set-process-sentinel
     process
     `(lambda (process event)
        (when (string= "finished\n" event)
          (delete-file ,tempfile)
          (save-window-excursion
            (save-excursion
              (save-restriction
                (with-current-buffer (find-file-noselect ,current-file)
                  (goto-char (point-min))
                  (re-search-forward ,uuid)
                  (beginning-of-line)
                  (kill-line)
                  (insert (mapconcat
                           (lambda (x)
                             (format ": %s" x))
                           (split-string
                            (with-current-buffer ,pbuffer (buffer-string))
                            "\n")
                           "\n")))))))
        (kill-buffer ,pbuffer)))))

org-babel-async-execute

print 'hello world'
import time
time.sleep(10)

import os
print os.getcwd()
print time.asctime()

hello world
/Users/jkitchin/blogofile-jkitchin.github.com/_blog
Fri Nov 20 10:20:01 2015

(defun org-babel-async-execute:python ()
  "Execute the python src-block at point asynchronously.
:var headers are supported.
:results output is all that is supported for output.

A new window will pop up showing you the output as it appears,
and the output in that window will be put in the RESULTS section
of the code block."
  (interactive)
  (let* ((current-file (buffer-file-name))
         (uuid (org-id-uuid))
         (code (org-element-property :value (org-element-context)))
         (temporary-file-directory ".")
         (tempfile (make-temp-file "py-"))
         (pbuffer (format "*%s*" uuid))
         (varcmds (org-babel-variable-assignments:python
                   (nth 2 (org-babel-get-src-block-info))))
         process)

    ;; get rid of old results, and put a place-holder for the new results to
    ;; come.
    (org-babel-remove-result)

    (save-excursion
      (re-search-forward "#\\+END_SRC")
      (insert (format
               "\n\n#+RESULTS: %s\n: %s"
               (or (org-element-property :name (org-element-context))
                   "")
               uuid)))

    ;; open the results buffer to see the results in.
    (switch-to-buffer-other-window pbuffer)

    ;; Create temp file containing the code.
    (with-temp-file tempfile
      ;; if there are :var headers insert them.
      (dolist (cmd varcmds)
        (insert cmd)
        (insert "\n"))
      (insert code))

    ;; run the code
    (setq process (start-process
                   uuid
                   pbuffer
                   "python"
                   tempfile))

    ;; when the process is done, run this code to put the results in the
    ;; org-mode buffer.
    (set-process-sentinel
     process
     `(lambda (process event)
        (save-window-excursion
          (save-excursion
            (save-restriction
              (with-current-buffer (find-file-noselect ,current-file)
                (goto-char (point-min))
                (re-search-forward ,uuid)
                (beginning-of-line)
                (kill-line)
                (insert
                 (mapconcat
                  (lambda (x)
                    (format ": %s" x))
                  (butlast (split-string
                            (with-current-buffer
                                ,pbuffer
                              (buffer-string))
                            "\n"))
                  "\n"))))))
        ;; delete the results buffer then delete the tempfile.
        ;; finally, delete the process.
        (when (get-buffer ,pbuffer)
          (kill-buffer ,pbuffer)
          (delete-window))
        (delete-file ,tempfile)
        (delete-process process)))))

org-babel-async-execute:python

print 'hello world'
import time
time.sleep(1)

for i in range(5):
    print i

    time.sleep(0.5)


import os
print os.getcwd()
print time.asctime()

print data

raise IOError('No file!')

hello world
0
1
2
3
4
/Users/jkitchin/blogofile-jkitchin.github.com/_blog
Fri Nov 20 19:30:16 2015
[1, 3]
Traceback (most recent call last):
  File "/Users/jkitchin/blogofile-jkitchin.github.com/_blog/py-84344aa1", line 18, in <module>
    raise IOError('No file!')
IOError: No file!

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

I have been thinking about a way to have functional and readable mathematics in technical documents. It has always bothered me that I have to write a LaTeX version of an equation, and then a separate implementation of the equation in code somewhere. At least twice in my life these separate representations have not agreed!

One solution might be if my functional code could be converted to LaTeX easily. I explore one simple approach to this here. It is somewhat inspired by this work here http://oremacs.com/2015/01/23/eltex/ on writing LaTeX in emacs-lisp, and from my work with org-mode in mixing narrative text, LaTeX and code.

The idea is to use emacs-lisp for the code, so it is functional, but provide an alternative output for the same code for a document conversion. In other words, we accept there is more than one version we need: a functional version for working, and a consumption version for presentation. We will generate the consumption version from the functional version.

I know emacs-lisp is not ideal for mathematics the way we are accustomed to seeing it, but it enables the idea I want to explore here so we will try it.

Here is the simplest example I could come up with for functional math. We can run it ourselves, and verify it is correct.

(+ 1 2 3)

6

Now, I can change the meaning of this code temporarily, so that it not only evaluates the form, but also represents the equation and result in LaTeX code. If this was incorporated into a preprocessor of the document, we could have a functional version representing our equations, in code form, and a presentation version generated from this version. The code that follows isn't how I would do this is in some production setting; it is only to show that you can temporarily change the meaning of "+". In a production setting, there would just be (+ 1 2 3) in the text, and a preprocessor would find all the sexps in the text, and replace them with the export format using code like this. At least, that is what I am imagining. It might be feasible to do this already with inline org-babel calls and an org-mode export filter, but I didn't try it here. So, here is the proof of concept code.

(cl-flet ((+ (lambda (&rest args)
               (format
                "$%s = %s$"
                (mapconcat #'number-to-string args " + ")
                (eval `(+ ,@args))))))
  (+ 1 2 3))

\(1 + 2 + 3 = 6\)

Here is an example that generates a fraction from a division.

(cl-flet ((/ (lambda (&rest args)
               (format
                "$\\frac{%s}{%s} = %s$"
                (car args)
                (mapconcat 'number-to-string (cdr args) " \\cdot ")
                (eval `(/ ,@args))))))
  (/ 1.0 2.0 3.0))

\(\frac{1.0}{2.0 \cdot 3.0} = 0.16666666666666666\)

As a proof of concept, this idea looks feasible, but this implementation has some limitations. Getting this to a complete workable approach would require a lot of work, basically creating transformation functions for many, many kinds of mathematical functions, and a lot of other kinds of logic. For example, (+ 1 2 (+ 3 4)) would not render correctly with the codes above. It isn't even clear what it should render to. I think you want 1 + 2 + (3 + 4) as the rendered output.

Anyway, it is an interesting idea, one that blurs the lines between code and mathematics. We are so used to the equation representation of mathematics, rather than the code representation that being able to go back and forth seems like a good idea, especially when one is derived from the other.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10