A callable plist data structure for Emacs

| categories: elisp, emacs, macro | tags: | View Comments

Table of Contents

Emacs lisp has a few data structures that store key-value pairs. Here are some canonical examples of these data structures and the way to get data out of them.

  • a-lists
(let ((data '((key1 . 4)
              (key2 . "tree"))))
  (cdr (assoc 'key2 data)))
  • p-lists
(let ((data '(:key1 4 :key2 "tree")))
  (plist-get data :key2))
  • A hash table
(let ((data #s(hash-table data (key1 4 key2 "tree"))))
  (gethash 'key2 data))

Each of these uses some function to get data out of them. I have been learning about closures today, and realized a way you can make a "callable" data structure using them. In a closure, the data is stored as part of a function. We will use a "let over lambda" with a defalias in a lexical environment to achieve this. I will wrap a p-list with this approach, but it could work with any of the examples above. We will make the function have a few behaviors that allow us to see the whole data structure with no args, to get a value with one arg that is a key, and to set a value if there are more than two args add them as key-val pairs to the data structure. This block binds the function to the symbol "d" which is then a callable function.

(let ((data '(:key1 4 :key2 "tree")))
  (defalias 'd
    (lambda (&rest key-vals)
       ;; no args, return data
       ((= 0 (length key-vals))
       ;; just a key, get val
       ((= 1 (length key-vals))
        (plist-get data (car key-vals)))
        (loop for key in (-slice key-vals 0 nil 2)
              for val in (-slice key-vals 1 nil 2)
              (plist-put data key val))

Now we can use it like to get some data out:

(d :key2)

And add new values like:

(d :key3 "oak")

You can update a value with this too:

(d :key3 "pine")

or add multiple values like this:

(d :key4 0 :key5 9)

And see the whole plist with no args:


Pretty nice! It seems like there ought to be a macro to facilitate creating those. Here is one. This macro basically expands to the same code as above, but for fun I add a default value option.

(defmacro default-dict (var &optional default &rest key-vals)
  "Bind a callable plist to VAR that contains KEY-VALS."
  (let ()
    `(let ((data ',key-vals))
       (defalias ',var
         (lambda (&rest key-vals)
           (message "%s" key-vals)
            ;; no args, return data
            ((= 0 (length key-vals))
            ;; just a key, get val
            ((= 1 (length key-vals))
             (or  (plist-get data (car key-vals)) ,default))
             (loop for key in (-slice key-vals 0 nil 2)
                   for val in (-slice key-vals 1 nil 2)
                   (plist-put data key val))

Here is an instance of it.

(default-dict d2 "None" :key1 4 :key2 "tree")

And here it is in use.

(d2 :key1)
(d2 :new-key)

Not bad. If you come from Python, you might find this style of data structure to be more similar to what you are used to seeing. It sure seems less verbose than the usual plist boilerplate I have used before.

1 An update <2017-04-21 Fri>

One (perhaps undesirable even) feature of the approach above is that it creates a function in the global namespace. This might have unintended consequences with name clashes or shadowing, and if you later use the same variable name for a plist, you would change the function behavior. Here we consider a way to limit the scope of where these functions exist and work. The labels macro provides one way to do this, we just create temporary functions that only exist within a scope. There is a lot of backticking and comma operators in this, and it took quite a few iterations to get it working!

This macro creates temporary functions for each keyword that return the value in the plist.

(defmacro with-dict (key-vals &rest body)
  "A context-manager for a plist where each key is a callable
function that returns the value."
  (declare (indent 1))
  (let* ((g (if (symbolp key-vals)
                (symbol-value key-vals)
         (keys (-slice g 0 nil 2)))
    `(labels ,(loop for key in keys
                    (list key '() `(plist-get ',g  ,key)))

Here is how we use it:

(with-dict (:a 1 :b 'some-symbol :c 3)

We can also use it with variables that hold mappings like this.

(let ((d '(:key1 1 :key2 some-other-symbol :key3 3)))
  (with-dict d
    (format "We got %s" (:key2))))

That is pretty interesting! In case that looks similar to a context manager in Python, now you know where Python got that idea ;)

Another related idea is to let-bind the values to variables withing a scope. We can't use the keywords directly here, so I use some hackery to strip off the colon so it is a regular symbol. That is not quite as nice I guess since you have to remember to remove the : from the symbols in the body of your code.

(defmacro with-plist-vals (plist &rest body)
  "Bind the values of a plist to variables with the name of the keys."
  (declare (indent 1))
  `(let ,(loop for key in (-slice plist 0 nil 2)
               for val in (-slice plist 1 nil 2)
               collect (list (intern
                              (substring (symbol-name key) 1))

Here is an example usage.

(with-plist-vals (:a 4 :b 6)
 (* 2 a))

Obviously that is just an alternate syntax for the let statement, but it lets you leverage the plist syntax for multiple purposes.

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.5

Read and Post Comments

A new org-mode exporter to Word for scimax

| categories: emacs, export, orgmode | tags: | View Comments

I am continuing to chip away to getting a reasonable export behavior for org-mode to MS Word. I have previously made some progress with Pandoc here and here, but those solutions never stuck with me. So here is another go. Here I leverage Pandoc again, but use a path through LaTeX to get citations without modifying the org-ref cite link syntax. The code for this can be found here: https://github.com/jkitchin/scimax/blob/master/ox-word.el. The gist is you use org-ref like you always do, and you specify the bibliography style for Pandoc like this:

You can download other csl files at https://www.zotero.org/styles. Then you can simply export the org-doc to a Word document with the key-binding C-c C-e w p.

Here is an example document to illustrate the exporter. I have written about data sharing in catalysis kitchin-2015-examp and surface science kitchin-2015-data-surfac-scien.

Here is an example source block.

%matplotlib inline
import matplotlib.pyplot as plt

plt.plot([1, 2, 3, 4, 5, 6])

See Ref. fig:line for example. These do not work. That might require additional pre-processing to replace them with numbers.

Here is the Word document that is generated: 2017-04-15.docx

As a penultimate result it might be ok. The references are reasonably formatted, but not compatible with Endnote, or other bibliography manager software. There are still some issues with Figure numbering and cross-references, but it is not too bad. The main benefit of this seems to be that one source generates HTML and the Word document.


Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.5

Read and Post Comments

Autoformatting ordinal numbers and fractions in orgmode

| categories: emacs, orgmode | tags: | View Comments

MS Word has a few things I like. One of them is the ability to autoformat things to make an ordinal number string like 1st to the superscripted version 1st while you type or a 1/2 to ½. I thought it would be pretty easy to implement that for org-mode. It turns out it was not so easy!

There does not appear to be a way to specify a regexp pattern as an abbreviation, or an abbrev that starts with a number. What we need for ordinal numbers is to recognize a sequence of numbers followed by "st", "nd", "rd" or "th" followed by a space or punctuation, and then superscript the letters. In case you didn't want the replacement to occur, you should be able to undo it and get back the original string. This addition was a little hard won, so I am sharing the lessons here.

The logic I used is to put a function in the post-self-insert-hook. The function only works in org-mode, when not in a codeblock and when looking back at a regexp that matches a pattern to be replaced. Getting it to undo was trickier than expected. Eventually I worked out that you put an undo boundary in place before the change, and then it seems like you can undo the changes. I created a minor mode so it is easy to toggle this on and off.

Here is the implementation:

(defcustom scimax-autoformat-ordinals t
  "Determines if scimax autoformats ordinal numbers."
  :group 'scimax)

(defun scimax-org-autoformat-ordinals ()
  "Expand ordinal words to superscripted versions in org-mode.
1st to 1^{st}.
2nd to 2^{nd}
3rd to 3^{rd}
4th to 4^{th}"
  (when (and scimax-autoformat-ordinals
             (eq major-mode 'org-mode)
             (not (org-in-src-block-p))
             (looking-back "\\(?3:\\<\\(?1:[0-9]+\\)\\(?2:st\\|nd\\|rd\\|th\\)\\>\\)\\(?:[[:punct:]]\\|[[:space:]]\\)"
      (replace-match "\\1^{\\2}" nil nil nil 3))))

(defcustom scimax-autoformat-fractions t
  "Determines if scimax autoformats fractions."
  :group 'scimax)

(defun scimax-org-autoformat-fractions ()
  "Expand fractions to take up space."
  (when (and scimax-autoformat-fractions
             (eq major-mode 'org-mode)
             (not (org-in-src-block-p))
             (looking-back "\\(?3:\\<\\(1/4\\|1/2\\|3/4\\)\\>\\)\\(?:[[:punct:]]\\|[[:space:]]\\)"
      (replace-match (cdr (assoc (match-string 3) '(("1/4" . "¼")
                                                    ("1/2" . "½")
                                                    ("3/4" . "¾"))))
                     nil nil nil 3))))

(defun scimax-org-autoformat ()
  "Autoformat functions."

(define-minor-mode scimax-autoformat-mode
  "Toggle `scimax-autoformat-mode'.  Converts 1st to 1^{st} as you type."
  :init-value nil
  :lighter (" om")
  (if scimax-ordinal-mode
      (add-hook 'post-self-insert-hook #'scimax-org-autoformat nil 'local)
    (remove-hook 'post-self-insert-hook #'scimax-org-autoformat 'local)))

This is now a feature in scimax. This marks the 500th blog post! That is ½ way to 1000. At the current rate of posting, it will be at least 5 years until I hit that!

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.5

Read and Post Comments

A better return in org-mode

| categories: emacs, orgmode | tags: | View Comments

Over on Stackoverflow someone wanted a better return in org-mode. They wanted return to add items in a list (instead of M-Ret). Someone posted a partial solution, and here I improve on it to add new items to lists, new headings after a heading, and new rows to tables. In each case, a double return on an empty item, headline or table row will delete that line, and terminate the list, headlines or table. You can still use M-Ret, and this function falls through to org-return like it did before. You can use a prefix arg to get a regular return if you want one (e.g. you want to press enter on a headline to push it down).

Here is the function. Give it a try. It is a small but helpful addition I think. I have not used it for long, so if you come across issues leave a comment!

(require 'org-inlinetask)

(defun scimax/org-return (&optional ignore)
  "Add new list item, heading or table row with RET.
A double return on an empty element deletes it.
Use a prefix arg to get regular RET. "
  (interactive "P")
  (if ignore

     ((eq 'line-break (car (org-element-context)))

     ;; Open links like usual, unless point is at the end of a line.
     ;; and if at beginning of line, just press enter.
     ((or (and (eq 'link (car (org-element-context))) (not (eolp)))

     ;; It doesn't make sense to add headings in inline tasks. Thanks Anders
     ;; Johansson!

     ;; checkboxes too
      (org-insert-todo-heading nil))

     ;; lists end with two blank lines, so we need to make sure we are also not
     ;; at the beginning of a line to avoid a loop where a new entry gets
     ;; created with only one blank line.
      (if (save-excursion (beginning-of-line) (org-element-property :contents-begin (org-element-context)))
        (delete-region (line-beginning-position) (line-end-position))

     ;; org-heading
      (if (not (string= "" (org-element-property :title (org-element-context))))
          (progn (org-end-of-meta-data)
        (setf (buffer-substring
               (line-beginning-position) (line-end-position)) "")))

     ;; tables
      (if (-any?
           (lambda (x) (not (string= "" x)))
            (- (org-table-current-dline) 1)
        ;; empty row
        (setf (buffer-substring
               (line-beginning-position) (line-end-position)) "")

     ;; fall-through case

(define-key org-mode-map (kbd "RET")

Here are a few tests:

  1. numbered item
  2. second item
    1. nested number
    2. second number
  • [ ] check 1
  • [ ] check 2
  • [ ] check 3
an inline task

With some content

1 a subheading

2 another Subheading

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.5

Read and Post Comments

Object-oriented font-locking in emacs-lisp

| categories: emacs, fontlock | tags: | View Comments

I have been interested in functional text for a long time. With functional text you can read it, but also interact with it kitchin-2015-examp. Lately I have been thinking about how to use some features of object-oriented programming to functional text. The premise is to use an object hierarchy to encapsulate some knowledge, and provide functionality on the objects. We can use inheritance to customize some of this knowledge and functionality.

The example I will work out here is to provide functional text for chemical elements. The goal is to define some objects that represent elements, and construct font-lock rules from the objects to make the text functional in Emacs. Functional here means it stands out so we know there is something special about it, it has a tooltip to get some information (like what type of element it is, and its atomic mass), and it is clickable to get more functionality.

This post will make a lot more sense in this video: https://www.youtube.com/watch?v=IWxCj5cr8rY

First, we create a base class of an Element. I use an instance tracker for this to make book keeping easy later. The base class will have a name, synonyms for the name, and a default face to color it. We define a few methods to get an atomic mass and search google for the element. Finally, we provide a function to generate a tooltip, and a font-lock rule.

(defvar atomic-masses '(("Hydrogen" . 1.008)
                        ("Argon" . 39.948)
                        ("Sodium" . 22.989)
                        ("Palladium" . 106.42))
  "a-list of atomic masses.")

(defvar elements '() "List of known elements")
(setq elements '()) ;; this is to start over

(defclass element (eieio-instance-tracker)
  ((tracking-symbol :initform elements
                    :documentation "Variable that holds all class instances.")
   (name :initarg :name
         :documentation "The name of the element")
   (synonyms :initarg :synonyms :initform '()
             :documentation "List of regular expressions that match the element.")
   (face :initarg :face :initform 'font-lock-type-face
         :documentation "The face to use with font-lock."))
  "Base class for a chemical element.")

(defmethod element-atomic-mass ((x element))
  "Return atomic mass from `atomic-masses'."
  (cdr (assoc (oref x :name) atomic-masses)))

(defmethod element-help-echo ((x element))
  "A tooltip for the element.
It will look like class (inherited classes) mass=atomic-mass"
  (format "%s %s: mass=%s"
          (eieio-object-class x)
          (mapcar 'eieio-class-name (eieio-class-parents (eieio-object-class x)))
          (or (element-atomic-mass x) "unknown")))

(defmethod element-search ((x element))
  "Search google for the element"
  (google-this-string nil (oref x :name) t))

(defmethod element-font-lock-rule ((x element))
  "Return font-lock rule for the element."
  (let ((map (make-sparse-keymap)))
    (define-key map [mouse-1]
      (lambda ()
        "Construct the object and run `element-search' on it."
          (get-text-property (point) 'element-name)
          :name 'elements))))

     ;; Construct the pattern to match
     (rx-to-string `(: bow
                       (or  ,(oref x :name)
                            ,@(loop for sy in (oref x :synonyms)
                                    collect `(regexp ,sy)))
     0  ;; font-lock the whole match
     ;; These are the properties to put on the matches
     `(quote (face ,(oref x :face)
                   element-name ,(oref x :name)
                   local-map ,map
                   mouse-face 'highlight
                   help-echo ,(element-help-echo x))))))

Now, we can define some sub-classes. These are families of elements. For a metal, we change the face. For noble gases, we override the help-echo function, and for alkali metals we override the search function. The point is that we can customize the behavior for different classes.

(defclass metal (element)
  ((face :initform '(:foreground "orange" :underline t)))

(defclass noble-gas (element)
  "A noble gas")

(defmethod element-help-echo ((x noble-gas))
  "I am not a common element.")

(defclass alkali (element metal)
  "Alkali metal")

(defmethod element-search ((x alkali))
  (let ((visible-bell t))
    (message "You clicked on an alkali metal: %s." (oref x :name))))

Now we can define some elements. These are all instances of each class. For some, we define synonyms, and alternate appearances. Note the synonyms are regular expressions.

(element :name "Hydrogen" :synonyms '("H" "[hH]ydrogen"))

(noble-gas :name "Argon" :synonyms '("Ar"))

(alkali :name "Sodium" :synonyms '("Na" "[nN]atrium"))
(alkali :name "Potassium" :synonyms '("K") :face '(:foreground "red"))

(metal :name "Palladium")

The instance tracker shows us the defined objects.


1 Font-locking the elements

Here we generate font-lock rules from the set of objects. Each object will return its font-lock rule, so we just map over each object to get the list of rules.

 (mapcar 'element-font-lock-rule elements))


Now any time we have Palladium or Hydrogen it will be highlighted. And Sodium and Argon.

Here are some synonyms: hydrogen H Natrium natrium.

Potassium has a different color than Na.

2 Summary

This seems like a pretty useful way to encapsulate functionality for functional text. Clearly most of the work should go in the base class, and the inheritance model, so you do not have to repeat things unnecessarily. Some features are missing, like conveniently adding synonyms and regenerating the font-lock rules. It is also the case that we do not persist these objects. They could be written to disk so that they can be reloaded later.

The actions you can use on a highlighted word are pretty limited in this implementation. It would be nice if you got a menu of options that was user extendable and dynamic. Either a popup menu, or a hydra would be fine.

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.5

Read and Post Comments

Next Page »