It has always bothered me a little that I have to add a file image after code blocks in org-mode to see the results. That extra work… I also don't like having to explicitly print the figure in the code, since that is the extra work, just in a different place. Today I look into two approaches to this. First, we consider something like tooltips, and second just putting overlays of image files right on the file name. The plus side of this is no extra work. The downside is they won't export; that will still take the extra work, but you needed that for the caption anyway for now.

Here is a video illustrating the code in this post: https://www.youtube.com/watch?v=VuAnwCERM0U

Here is a test.

import matplotlib.pyplot as plt
plt.plot([0, 1, 2, 4, 16])
plt.savefig("test-fig.png")

(defvar image-tooltip-re (concat  "\\(?3:'\\|\"\\)\\(?1:.*\\."
                                  (regexp-opt '("png" "PNG" "JPG" "jpeg"
                                                "jpg" "JPEG" "eps" "EPS"))
                                  "\\)\\(?:\\3\\)")
  "Regexp to match image filenames in quotes")

(defun image-tooltip (window object position)
  (save-excursion
    (goto-char position)
    (let (beg end imgfile img s)
      (while (not (looking-at image-tooltip-re))
        (forward-char -1))
      (setq imgfile (match-string-no-properties 1))
      (when (file-exists-p imgfile)
        (setq img (create-image (expand-file-name imgfile)
                                'imagemagick nil :width 200))
        (propertize "Look in the minibuffer"
                    'display img)))))

(font-lock-add-keywords
 nil
 `((,image-tooltip-re
    0 '(face font-lock-keyword-face
             help-echo image-tooltip))))

(font-lock-fontify-buffer)

(defun next-image-overlay (&optional limit)
  (when (re-search-forward image-tooltip-re limit t)
    (setq beg (match-beginning 0)
          end (match-end 0)
          imgfile (match-string 1))
    (when (file-exists-p imgfile)
      (setq img (create-image (expand-file-name imgfile)
                              'imagemagick nil :width 300))
      (setq ov (make-overlay beg end))
      (overlay-put ov 'display img)
      (overlay-put ov 'face 'default)
      (overlay-put ov 'org-image-overlay t)
      (overlay-put ov 'modification-hooks
                   (list 'org-display-inline-remove-overlay)))))

(font-lock-add-keywords
 nil
 '((next-image-overlay (0  'font-lock-keyword-face t)))
 t)

import matplotlib.pyplot as plt
plt.plot([-0, 1, 2, 4, 16])
plt.savefig("test-fig.png")

(ov-clear 'org-image-overlay)

(font-lock-remove-keywords
 nil
 '((next-image-overlay (0  'font-lock-keyword-face t))))

(ov-clear 'org-image-overlay)

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

I often want to rearrange org headlines. There are super convenient shortcuts for some things, like Alt-up/down to move up and down, and Alt-left/right to change levels, and shift variants of that when there are nested headlines. There is also refile for some use cases. The case these don't handle for me is when I have a headline at the bottom and I want to move it a distance. To cut down on key strokes, I usually kill the headline, jump to where I want, and paste it.

In lispy, there is a teleport command for this! I am going to adapt this here for reorganizing org-headlines. This version allows you to move headlines around on the visible area. You need another strategy for the candidates to jump to if you want to move a headline off the screen. Note though that ivy is really smart, you can have one file in a split view and it will jump to any headline in any window! I use ivy for the navigation here, but you could also use helm to select headlines, for example. This function puts your headline after the selected headline, unless you use a prefix arg, and then it goes before.

Check out the video to see this in action: https://www.youtube.com/watch?v=Vv-2888rpyc

Note: this code is a little more advanced than what is in the video; I added a numeric prefix to change the headline level.

(defun org-teleport (&optional arg)
  "Teleport the current heading to after a headline selected with avy.
With a prefix ARG move the headline to before the selected
headline. With a numeric prefix, set the headline level. If ARG
is positive, move after, and if negative, move before."
  (interactive "P")
  ;; Kill current headline
  (org-mark-subtree)
  (kill-region (region-beginning) (region-end))
  ;; Jump to a visible headline
  (avy-with avy-goto-line (avy--generic-jump "^\\*+" nil avy-style))
  (cond
   ;; Move before  and change headline level
   ((and (numberp arg) (> 0 arg))
    (save-excursion
      (yank))
    ;; arg is what we want, second is what we have
    ;; if n is positive, we need to demote (increase level)
    (let ((n (- (abs arg) (car (org-heading-components)))))
      (cl-loop for i from 1 to (abs n)
               do
               (if (> 0 n)
                   (org-promote-subtree)
                 (org-demote-subtree)))))
   ;; Move after and change level
   ((and (numberp arg) (< 0 arg))
    (org-mark-subtree)
    (goto-char (region-end))
    (when (eobp) (insert "\n"))
    (save-excursion
      (yank))
    ;; n is what we want and second is what we have
    ;; if n is positive, we need to demote
    (let ((n (- (abs arg) (car (org-heading-components)))))
      (cl-loop for i from 1 to (abs n)
               do
               (if (> 0 n) (org-promote-subtree)
                 (org-demote-subtree)))))

   ;; move to before selection
   ((equal arg '(4))
    (save-excursion
      (yank)))
   ;; move to after selection
   (t
    (org-mark-subtree)
    (goto-char (region-end))
    (when (eobp) (insert "\n"))
    (save-excursion
      (yank))))
  (outline-hide-leaves))

org-teleport

Now we add some new speed commands to help us out. I think we should be able to mark subtrees ("@" is bound to this, but I like "m" better) and kill them with a key stroke, in addition to teleporting them. Since we figured out the nice way to jump to a headline up there, we bind that to "q" which isn't used so far, and maps to a similar concept in lispy. The lowercase "t" is already bound to changing the TODO state, so we use capital "T" for the speed key to teleport. Note it is possible to "compose" the same effect by typing "k" to kill a headline, then "q" to jump somewhere else (or navigate where you want, and then "C-y" to paste it at the new location. Or, "T".

(add-to-list 'org-speed-commands-user (cons "m" 'org-mark-subtree))
(add-to-list 'org-speed-commands-user (cons "k" (lambda ()
                                                  (org-mark-subtree)
                                                  (kill-region
                                                   (region-beginning)
                                                   (region-end)))))
(add-to-list 'org-speed-commands-user
             (cons "q" (lambda ()
                         (avy-with avy-goto-line
                           (avy--generic-jump "^\\*+" nil avy-style)))))

(add-to-list 'org-speed-commands-user (cons "T" 'org-teleport))
"done"

done

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

I have wanted for a long time to be able to add captions and attributes to figures and tables generated from code blocks. I brought this up on the mailing list (https://lists.gnu.org/archive/html/emacs-orgmode/2015-11/msg00544.html ) and finally, I have figured out a way to do it that seems ok. It is based on the :wrap feature of org-babel.

The idea is to use a function that will wrap the results in additional text. We use this function that will add a caption and attributes.

(defun src-decorate (&optional caption attributes)
  "A wrap function for src blocks."
  (concat
   "ORG\n"
   (when attributes
     (concat (mapconcat 'identity attributes "\n") "\n"))
   (when caption
     (format "#+caption: %s" caption))))

src-decorate

Now, we try it out on a figure:

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 2 * np.pi)
y = np.sin(x)

plt.plot(x, y)
plt.savefig('images/sin.png')
print('[[./images/sin.png' + ']]')

import numpy as np

x = np.linspace(0, 2 * np.pi, 5)
y = np.sin(x)

return [['x', 'y'], None] + list(zip(x, y))

We often write code in supporting information files that generates figures or tables in a scientific manuscript. Today, we explore how to call those code blocks remotely but get the output in the file we call it from. We will write code in si.org that generates an interactive figure that is presented in this file. We will use data published in hallenbeck-2015-compar-co2. You can find the data we used in the SI for that paper, or more conveniently here .

So, we make a named code block in the si.org file called "figure-1". Then we call it like this:

#+call: si.org:figure-1() :wrap html

That executes the code block in the other file, and wraps the output in an HTML block in this file! I do not like my code blocks to execute when I export because they are usually expensive calculations, so I have to manually run the line with C-c C-c, but you can override that behavior with a local setting of org-export-babel-evaluate. So, without further delay, here is the result. Now we have a nice, neat blog post file, with code in an si.org file!

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Occasionally, someone wants side by side figures with subcaptions that have individually referenceable labels. This is not too hard in LaTeX, and there is a solution here: http://www.johndcook.com/blog/2009/01/14/how-to-display-side-by-side-figurs-in-latex/ .

We can create side by side figures in raw LaTeX like this (note however, this will not show up in html export):

And in our text we can refer to the overall Figure fig12, or the subfigures Figure fig:a or Figure fig:b. This works fine if your end goal is LaTeX export. It does not work fine if you want to consider HTML or some other output.

So, here we consider how we could remove the LaTeX dependency by representing the figures in a sexp data structure, for example something like this. I change the labels and captions a bit so they are actually distinguishable.

(figure ()
 (subfigure '("Left graph from sexp." (label "fig:sa"))
            (includegraphics '((width . "3in"))
                             "images/cos-plot.png"))
 (enskip)
 (subfigure '("Right graph from sexp" (label "fig:sb"))
            (includegraphics '((width . "3in"))
                             "images/eos-uncertainty.png"))
 (caption
  "Text pertaining to both graphs from a sexp, " (ref "fig:sa")
  " and " (ref "fig:sb") "." (label "figs12")))

"emacs-lisp"

This doesn't look much worse than the LaTeX code itself. It might not seem useful right away, but imagine if this was really code that could evaluate to the format we want. Remember the sexp bibtex entry that could evaluate to bibtex, json or xml? Let's look at this here. What we consider is kind of like http://oremacs.com/2015/01/23/eltex/ , but we could include other kinds of exports if we wanted.

Here is our special block in org-mode. It should render roughly as side by side images in LaTeX and HTML.

Now, we need a function to format the sexp block for export. It is easy, we just eval the contents of the block. We do assume here there is just one sexp to evaluate. This function will handle all special blocks, but we only want to do something different for the sexp blocks.

(defun sb-format (sb contents info)
  (cond
     ((string= "SEXP" (org-element-property :type sb))
      (eval (read (buffer-substring
                   (org-element-property :contents-begin sb)
                   (org-element-property :contents-end sb)))))
     (t
      contents)))))

sb-format

All that is left is to define the functions. We do that next.

(defun label (arg)
  (format "\\label{%s}" arg))

(defun ref (arg)
  (format "\\ref{%s}" arg))

(defun caption (&rest body)
  (format "\\caption{%s}"
         (mapconcat 'eval body "")))

(caption
  "Text pertaining to both graphs, " (ref "fig:a")
  " and " (ref "fig:b") "." (label "fig12"))

\caption{Text pertaining to both graphs, \ref{fig:a} and \ref{fig:b}.\label{fig12}}

(defun includegraphics (options path)
  (format "\\includegraphics%s{%s}"
          (if options
              (format "[%s]"
                      (mapconcat (lambda (ccell)
                                   (format "%s=%s"
                                           (car ccell)
                                           (cdr ccell)))
                                 options
                                 ","))
            "")
          path))

(includegraphics '((width . "3in"))
                 "images/eos-uncertainty.png")

\includegraphics[width=3in]{images/eos-uncertainty.png}

(defun subfigure (options &rest body)
  (format "\\subfigure%s{%s}"
          (if options
              (format "[%s]"
                      (mapconcat 'eval options ""))
            "")
          (mapconcat 'eval body "")))

(subfigure '("Right graph" (label "fig:b"))
            (includegraphics '((width . "3in"))
                             "images/eos-uncertainty.png"))

\subfigure[Right graph\label{fig:b}]{\includegraphics[width=3in]{images/eos-uncertainty.png}}

(defun figure (options &rest body)
  (format "\\begin{figure}
%s
\\end{figure}"
(mapconcat 'eval body "\n")))

(defun enskip () "\\enskip")

enskip

(figure ()
 (subfigure '("Left graph from sexp." (label "fig:ssa"))
            (includegraphics '((width . "3in"))
                             "images/cos-plot.png"))
 (enskip)
 (subfigure '("Right graph from sexp" (label "fig:ssb"))
            (includegraphics '((width . "3in"))
                             "images/eos-uncertainty.png"))
 (caption
  "Text pertaining to both graphs from a sexp, " (ref "fig:ssa")
  " and " (ref "fig:ssb") "." (label "figss12")))

(org-export-define-derived-backend 'my-latex 'latex
  :translate-alist '((special-block . sb-format)))

(org-latex-compile (org-export-to-file 'my-latex "custom-sb-export.tex"))
(org-open-file "custom-sb-export.pdf")

2 HTML functions

We can use the same sexp block to get figures side-by-side in HTML. We need to redefine each element and its HTML output.

(defun label (arg)
  (format "<a name=\"%s\"></a>" arg))

(defun ref (arg)
  (format "<a href=\"#%s\">%s</a>" arg arg))

(defun caption (&rest body)
  (format "<caption>%s</caption>"
         (mapconcat 'eval body "")))

(caption
  "Text pertaining to both graphs, " (ref "fig:a")
  " and " (ref "fig:b") "." (label "fig12"))

<caption>Text pertaining to both graphs, <a href="#fig:a">fig:a</a> and <a href="#fig:b">fig:b</a>.<a name="fig12"></a></caption>

We will ignore options for the includegraphics html output. We would need to specify some way to do unit conversions for html. Here we fix the width.

(defun includegraphics (options path)
  (format "<img src=\"/media/%s\" width=\"300\">"
          path))

(includegraphics '((width . "3in"))
                 "images/eos-uncertainty.png")

<img src="/media/images/eos-uncertainty.png" width="300">

We wrap a subfigure in a table cell.

(defun subfigure (options &rest body)
  (format "<td>%s%s</td>"
          (mapconcat 'eval body "")
          (when options
            (concat "<br>"
                    (mapconcat 'eval options "")))))

(subfigure '("Right graph" (label "fig:b"))
            (includegraphics '((width . "3in"))
                             "images/eos-uncertainty.png"))

<td><img src="/media/images/eos-uncertainty.png" width="300"><br>Right graph<a name="fig:b"></a></td>

We assume we can put the images in a single row.

(defun figure (options &rest body)
  (format "<span class=\"image\"><table>
<tr>%s</tr>
</table></span>"
(mapconcat 'eval body "\n")))

(defun enskip () "")

enskip

Now, here is our specification.

(figure ()
 (subfigure '("Left graph" (label "fig:ha"))
            (includegraphics '((width . "3in"))
                             "images/cos-plot.png"))
 (enskip)
 (subfigure '("Right graph" (label "fig:hb"))
            (includegraphics '((width . "3in"))
                             "images/eos-uncertainty.png"))
 (caption
  "Text pertaining to both graphs, " (ref "fig:ha")
  " and " (ref "fig:hb") "." (label "figh12")))

Text pertaining to both graphs, fig:ha and fig:hb.

Left graph

Right graph

And our derived backend for HTML.

(org-export-define-derived-backend 'my-html 'html
  :translate-alist '((special-block . sb-format)))

(browse-url (org-export-to-file 'my-html "custom-sb-export.html"))

#<process open custom-sb-export.html>

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10