The Kitchin Research Group

Emacs lisp has a few data structures that store key-value pairs. Here are some canonical examples of these data structures and the way to get data out of them.

• a-lists

(let ((data '((key1 . 4)
              (key2 . "tree"))))
  (cdr (assoc 'key2 data)))

tree

• p-lists

(let ((data '(:key1 4 :key2 "tree")))
  (plist-get data :key2))

tree

• A hash table

(let ((data #s(hash-table data (key1 4 key2 "tree"))))
  (gethash 'key2 data))

tree

Each of these uses some function to get data out of them. I have been learning about closures today, and realized a way you can make a "callable" data structure using them. In a closure, the data is stored as part of a function. We will use a "let over lambda" with a defalias in a lexical environment to achieve this. I will wrap a p-list with this approach, but it could work with any of the examples above. We will make the function have a few behaviors that allow us to see the whole data structure with no args, to get a value with one arg that is a key, and to set a value if there are more than two args add them as key-val pairs to the data structure. This block binds the function to the symbol "d" which is then a callable function.

(let ((data '(:key1 4 :key2 "tree")))
  (defalias 'd
    (lambda (&rest key-vals)
      (cond
       ;; no args, return data
       ((= 0 (length key-vals))
        data)
       ;; just a key, get val
       ((= 1 (length key-vals))
        (plist-get data (car key-vals)))
       (t
        (loop for key in (-slice key-vals 0 nil 2)
              for val in (-slice key-vals 1 nil 2)
              do
              (plist-put data key val))
        data)))))

d

Now we can use it like to get some data out:

(d :key2)

tree

And add new values like:

(d :key3 "oak")

You can update a value with this too:

(d :key3 "pine")

or add multiple values like this:

(d :key4 0 :key5 9)

And see the whole plist with no args:

(d)

Pretty nice! It seems like there ought to be a macro to facilitate creating those. Here is one. This macro basically expands to the same code as above, but for fun I add a default value option.

(defmacro default-dict (var &optional default &rest key-vals)
  "Bind a callable plist to VAR that contains KEY-VALS."
  (let ()
    `(let ((data ',key-vals))
       (defalias ',var
         (lambda (&rest key-vals)
           (message "%s" key-vals)
           (cond
            ;; no args, return data
            ((= 0 (length key-vals))
             data)
            ;; just a key, get val
            ((= 1 (length key-vals))
             (or  (plist-get data (car key-vals)) ,default))
            (t
             (loop for key in (-slice key-vals 0 nil 2)
                   for val in (-slice key-vals 1 nil 2)
                   do
                   (plist-put data key val))
             data)))))))

Here is an instance of it.

(default-dict d2 "None" :key1 4 :key2 "tree")

d2

And here it is in use.

(d2 :key1)

4

(d2 :new-key)

None

Not bad. If you come from Python, you might find this style of data structure to be more similar to what you are used to seeing. It sure seems less verbose than the usual plist boilerplate I have used before.

(defmacro with-dict (key-vals &rest body)
  "A context-manager for a plist where each key is a callable
function that returns the value."
  (declare (indent 1))
  (let* ((g (if (symbolp key-vals)
                (symbol-value key-vals)
              key-vals))
         (keys (-slice g 0 nil 2)))
    `(labels ,(loop for key in keys
                    collect
                    (list key '() `(plist-get ',g  ,key)))
       ,@body)))

with-dict

(with-dict (:a 1 :b 'some-symbol :c 3)
  (:b))

(let ((d '(:key1 1 :key2 some-other-symbol :key3 3)))
  (with-dict d
    (format "We got %s" (:key2))))

We got some-other-symbol

(defmacro with-plist-vals (plist &rest body)
  "Bind the values of a plist to variables with the name of the keys."
  (declare (indent 1))
  `(let ,(loop for key in (-slice plist 0 nil 2)
               for val in (-slice plist 1 nil 2)
               collect (list (intern
                              (substring (symbol-name key) 1))
                             val))
     ,@body))

with-plist-vals

(with-plist-vals (:a 4 :b 6)
 (* 2 a))

8

Copyright (C) 2018 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.1.6

I have been thinking of better ways to write code that is more likely to have decent docstrings that are up to date, and maybe that enable automatic validation. One strategy is to keep documentation and code together, and by together I mean close together. The closer the better. I made some interesting progress in the last post, where I used a macro to let me put argument specific documentation in the same place that the argument is defined. Here I expand the idea to also provide argument default values, and validation code where the argument is defined inside the function, in addition to generating docstrings. This post is written in Emacs-lisp, mostly because I am more familiar with the macro language. The idea should apply to other lisps too.

Let's consider this prototypical, vanilla function definition, usage, and docstring.

(defun f1 (arg1 arg2)
  "Add two numbers."
  (+ arg1 arg2))

;; usage
(f1 3 4)

7

Here is what the help looks like from emacs.

(describe-function 'f1)

f1 is a Lisp function.

(f1 ARG1 ARG2)

For more information check the manuals.

Add two numbers.

It is clear I was lazy in writing the docstring; it does not even mention the arguments. There is also no validation of the arguments so if you pass a string and a number, you will get an error. There are no defaults either, so you have to provide both arguments. It seems like there could be significant room for improvement. Of course, I could bite the bullet and write a better function like this one:

(defun f1a (arg1 &optional arg2)
  "Add ARG1 and ARG2 together.
ARG1 and  ARG2 should both be numbers."
  (when (null arg2) (setq arg2 2))
  (unless (and (numberp arg1) (numberp arg2)) (error "arg1 and arg2 should both be numbers"))
  (+ arg1 arg2))

(list (f1a 3 4) (f1a 3))

Yes, I could do that, but it is tedious to do it all the time. And it still leaves something to be desired for me. The docstring does not say what the default value is for example, and that is hard-coded in the code, i.e. not introspectible until you look at the code. Next we consider an alternative way to write the function. Compare that to this function definition, usage and documentation. The function definition is a little more verbose. Providing documentation, defaults and validation code in any form would make it that way no matter what.

(defn f2 ((arg1 "A number" :validate numberp)
          (arg2 "A number" :validate numberp :default 2))
  "Add the arguments."
  (+ arg1 arg2))

;; usage
(list (f2 3 4) (f2 3))

(describe-function 'f2)

f2 is a Lisp function.

(f2 ARG1 &optional ARG2)

For more information check the manuals.

Add the arguments.
ARG1 : A number (valid = numberp)
ARG2 : A number (default = 2) (valid = numberp)

The documentation is built up from the information in the function definition, in a form that is mostly consistent with emacs-lisp documentation standards. defn is not a regular emacs-lisp function; it is a macro I developed to generate the function code. It turned out to be long, but the gist of it is that before defining the function I loop through the arguments and collect the docstrings, along with any information about default values and/or validation functions. Then I build up the list of arguments to put in the function. Then if any default values are set, I generate some code to set those values if they are not set in the function call, and finally a similar block of validation code. At the end, I construct the defun and return it. You can check out the code if you want here: https://github.com/jkitchin/scimax/blob/master/scimax-macros.el.

Let's take a look at what this code expands to.

(macroexpand-1
 '(defn f2 ((arg1 "A number" :validate numberp)
            (arg2 "A number" :validate numberp :default 2))
    "Add the arguments."
    (+ arg1 arg2)))

(defun f2
    (arg1 &optional arg2)
  "Add the arguments.\nARG1 : A number (valid = numberp)\nARG2 : A number (default = 2) (valid = numberp)\n"
  (progn
    (when
        (null arg2)
      (setq arg2 2)))
  (progn
    (unless
        (funcall 'numberp arg1)
      (error "In (%s %s) Expected %s to pass %S. Got %S" "f2" "(arg1 &optional arg2)" "arg1" 'numberp arg1))
    (unless
        (funcall 'numberp arg2)
      (error "In (%s %s) Expected %s to pass %S. Got %S" "f2" "(arg1 &optional arg2)" "arg2" 'numberp arg2)))
  (+ arg1 arg2))

You can see it expands to a regular defun, with a generated docstring, generated default settings code block, and generated validation code. Pretty nice.

Let's see what happens with a function that fails the validation. We should get an error. Here we capture the error so we can see it in the post.

(condition-case err
    (f2 "oak")
  (error
   (error-message-string err)))

In (f2 (arg1 &optional arg2)) Expected arg1 to pass numberp. Got "oak"

So we even get a useful error message when the wrong type of argument is provided. Compare that to the error message from the original version of this function. It tells us we got the wrong type, but not which argument.

(condition-case err
    (f1 "oak" 4)
  (error
   (error-message-string err)))

Wrong type argument: number-or-marker-p, "oak"

One last example to check out the &rest argument, with validation that every arg is a number.

(defn f4 ((rarg :rest
                :validate (lambda (x)
                            (-all-p 'identity (mapcar 'numberp x)))))
  "multiply all the arguments."
  (apply '* rarg))

(f4 1 2 3)

6

(condition-case err
    (f4 "oak" 4)
  (error
   (error-message-string err)))

In (f4 (&rest rarg)) Expected rarg to pass (lambda (x) (-all-p (quote identity) (mapcar (quote numberp) x))). Got ("oak" 4)

(describe-function 'f4)

f4 is a Lisp function.

(f4 &rest RARG)

For more information check the manuals.

multiply all the arguments.
RARG : No documentation

That looks ok too.

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7