The Kitchin Research Group

The loop macro in emacs lisp

Thu, 20 Nov 2014 09:50:00 EST

I was reading The Land Of Lisp chapter on the loop macro in Common Lisp. I am not too familiar with it, or the implementation in emacs-lisp, so in this post we explore what it can do. Here I will explore some uses of the loop macro to do things I used to do in Python all the time.

Here is a simple example to generate a list of numbers with the loop macro..

(loop for i
      below 5
      collect i)

Evidently, i starts at 0, and increments by one. We can specify a different value like this. Here we use the to token, which also includes the last value.

(loop for i from 2 to 10
  collect i)

IF you want to go backwards:

(loop for i downfrom 10 to 2 collect i)

And if you want an (de)increment different than one, use the by token.

(loop for i downfrom 10 to 2 by 3 collect i)

We can use this to iterate over a list too. Let us collect the square of each element in a simple list. This is similar to the mapcar function.

(loop for i in '(1 2 3 4)
  collect (* i i))

You can combine the ideas to get something similar to the enumerate function in python.

(loop for i
      from 0
      for month
      in '(january february march april may june july august september
                   october november december)
      collect (cons i month))

((0 . january)
 (1 . february)
 (2 . march)
 (3 . april)
 (4 . may)
 (5 . june)
 (6 . july)
 (7 . august)
 (8 . september)
 (9 . october)
 (10 . november)
 (11 . december))

The loop stops because we run out of months to iterate over. Here is a variation like the zip function in python.

(loop for element1 in '(a b c d)
      for element2 in '(1 2 3 4)
      collect (list element1 element2))

a	1
b	2
c	3
d	4

We can sum in the loop:

(loop for i in '(100 200 300) sum i)

We can conditionally sum things in the loop, e.g. sum only the odd numbers.

(loop for i in '(1 2 3 4 5)
  when (oddp i)
  sum i)

We can find the minima and maxima in a list

(loop for i in '(-1 0 1)
  minimize i)

-1

(loop for i in '(-1 0 1)
  maximize i)

You may want to do some action in the loop. Say we want to print even numbers from a list.

(loop for i in '(1 2 3 4)
      when (evenp i)
      do (print i))

2

4

There are some ways to break out of a loop using return like this.

(loop for i upto 10
      when (= i 3)
      return 'done
      do (print i))

Alternatively, you can use while/until.

(loop for i downfrom 10
      do (print i)
      until (= i 6))

Or the while variation:

(loop for i downfrom 10
      do (print i)
      while (> i 6))

1 Summary

This is not everything the loop macro can do! Here is what the help for that function says.

loop is an alias for `cl-loop' in `cl.el'.

(loop CLAUSE...)

The Common Lisp `loop' macro.
Valid clauses include:
  For clauses:
    for VAR from/upfrom/downfrom EXPR1 to/upto/downto/above/below EXPR2 by EXPR3
    for VAR = EXPR1 then EXPR2
    for VAR in/on/in-ref LIST by FUNC
    for VAR across/across-ref ARRAY
    for VAR being:
      the elements of/of-ref SEQUENCE [using (index VAR2)]
      the symbols [of OBARRAY]
      the hash-keys/hash-values of HASH-TABLE [using (hash-values/hash-keys V2)]
      the key-codes/key-bindings/key-seqs of KEYMAP [using (key-bindings VAR2)]
      the overlays/intervals [of BUFFER] [from POS1] [to POS2]
      the frames/buffers
      the windows [of FRAME]
  Iteration clauses:
    repeat INTEGER
    while/until/always/never/thereis CONDITION
  Accumulation clauses:
    collect/append/nconc/concat/vconcat/count/sum/maximize/minimize FORM
      [into VAR]
  Miscellaneous clauses:
    with VAR = INIT
    if/when/unless COND CLAUSE [and CLAUSE]... else CLAUSE [and CLAUSE...]
    named NAME
    initially/finally [do] EXPRS...
    do EXPRS...
    [finally] return EXPR

For more details, see Info node `(cl)Loop Facility'.

It is obviously quite powerful, although the syntax seems quite different than the usual lisp code I have been writing. It is not clear when this is superior to something like mapcar/mapconcat, or the dolist/dotimes functions.

org-mode source

Org-mode version = 8.2.7c

Generate emacs-lisp documentation

Fri, 17 Oct 2014 14:39:49 EDT

Emacs has some pretty amazing features to get help on a function (describe-function), to navigate quickly to functions in an elisp file (speedbar and imenu). Other languages have tools for generating documentation for all the functions in a file, e.g. epydoc, javadoc, Doxygen,… I have not found an equivalent to this in emacs-lisp. Here, we explore some options to get something similar to this. Our goal will be to take an emacs-lisp file, and generate an org-file of documentation, and then convert that to PDF for reading.

Say we have a function, jmax-bibtex-next-entry, and we want some information about it. Here are three functions that give us the argument list, documentation string, and function definition.

(help-function-arglist 'jmax-bibtex-next-entry)

&optional

(documentation 'jmax-bibtex-next-entry)

Jump to the beginning of the next bibtex entry. N is a prefix
argument. If it is numeric, jump that many entries
forward. Negative numbers do nothing.

(symbol-function 'jmax-bibtex-next-entry)

lambda

(&optional n)

Jump to the beginning of the next bibtex entry. N is a prefix\nargument. If it is numeric, jump that many entries\nforward. Negative numbers do nothing.

(interactive P)

(if (= (point) (save-excursion (bibtex-beginning-of-entry))) (progn (forward-char) (bibtex-next-entry)))

(if (re-search-forward bibtex-entry-head nil t (and (numberp n) n)) (progn (bibtex-beginning-of-entry)))

That will not always be the code we wrote, but it is functionally similar.

So we could create an org-entry like this:

(defun fun2org (function-symbol)
  (let ((args (help-function-arglist function-symbol))
        (doc  (documentation function-symbol))
        (code (symbol-function function-symbol)))
    (format "** %s %s
%s

#+BEGIN_SRC emacs-lisp
%S
#+END_SRC
" function-symbol args doc code)))

(fun2org 'jmax-bibtex-next-entry)

** jmax-bibtex-next-entry (&optional n)
Jump to the beginning of the next bibtex entry. N is a prefix
argument. If it is numeric, jump that many entries
forward. Negative numbers do nothing.

#+BEGIN_SRC emacs-lisp
(lambda (&optional n) "Jump to the beginning of the next bibtex entry. N is a prefix
argument. If it is numeric, jump that many entries
forward. Negative numbers do nothing." (interactive "P") (if (= (point) (save-excursion (bibtex-beginning-of-entry))) (progn (forward-char) (bibtex-next-entry))) (if (re-search-forward bibtex-entry-head nil t (and (numberp n) n)) (progn (bibtex-beginning-of-entry))))
#+END_SRC

The code is not that beautifully indented, but it is optional.

For variables, there are similar functions to get their documentation:

(documentation-property 'jmax-bibtex-journal-abbreviations 'variable-documentation)

List of (string journal-full-name journal-abbreviation). Find abbreviations at http://cassi.cas.org/search.jsp.

The problem still is, you have to know the variable and function names in advance. I want to take a file, and generate this for each function, and variable.

I posted a question on StackOverflow on how to get the functions defined in a file. The most feasible suggestion was to use the variable load-history, which contains a history of the variables and functions loaded, and the files they are in.

Here is an example of getting the entries associated with jmax-bibtex.el

(cdr (assoc "/Users/jkitchin/Dropbox/kitchingroup/jmax/jmax-bibtex.el" load-history ))

(jmax-bibtex-journal-abbreviations
 (defun . jmax-bibtex-generate-longtitles)
 (defun . jmax-bibtex-generate-shorttitles)
 (defun . jmax-stringify-journal-name)
 (defun . jmax-set-journal-string)
 jmax-nonascii-latex-replacements
 (defun . jmax-replace-nonascii)
 jmax-lower-case-words
 (defun . jmax-title-case-article)
 (defun . jmax-sentence-case-article)
 (defun . jmax-bibtex-next-entry)
 (defun . jmax-bibtex-previous-entry)
 (defun . jmax-bibtex-mode-keys)
 (provide . jmax-bibtex))

Each element in this case is either a variable, defun or provide. Here, we can use this to print some information about the variables defined in this file. I think it is sufficient to check if the element in the list is a symbol, because all the other elements are cons elements. I suppose there are other possibilities, including defcustom, defgroup, defalias, defsubst, and maybe others.

(dolist (element (cdr
                  (assoc
                   "/Users/jkitchin/Dropbox/kitchingroup/jmax/jmax-bibtex.el"
                   load-history )))
  (when (symbolp element)
    (princ 
    (format "%s
Documentation: %s

" element (documentation-property element 'variable-documentation)))))

jmax-bibtex-journal-abbreviations
Documentation: List of (string journal-full-name journal-abbreviation). Find abbreviations at http://cassi.cas.org/search.jsp.

jmax-nonascii-latex-replacements
Documentation: Cons list of non-ascii characters and their LaTeX representations

jmax-lower-case-words
Documentation: List of words to keep lowercase

We can handle functions by checking if an element is a cons cell with a first element of defun.

(dolist (element (cdr
                  (assoc
                   "/Users/jkitchin/Dropbox/kitchingroup/jmax/jmax-bibtex.el"
                   load-history )))
  (when (and (consp element)
             (eq (car element) 'defun))
    (princ (format "%s is a function\n" (cdr element))))))

jmax-bibtex-generate-longtitles is a function
jmax-bibtex-generate-shorttitles is a function
jmax-stringify-journal-name is a function
jmax-set-journal-string is a function
jmax-replace-nonascii is a function
jmax-title-case-article is a function
jmax-sentence-case-article is a function
jmax-bibtex-next-entry is a function
jmax-bibtex-previous-entry is a function
jmax-bibtex-mode-keys is a function

So, we have the important pieces to mash up what I am looking for. Let us refine the goal. I want to create a PDF documentation of what is in an elisp file with a section on variables, and a section on functions.

(let* ((elements (cdr
                  (assoc
                   "/Users/jkitchin/Dropbox/kitchingroup/jmax/jmax-bibtex.el"
                   load-history)))
       (vars (-filter 'symbolp elements))
       (funcons (-filter (lambda (x)
                           (and (consp x)
                                (eq 'defun (car x))))
                         elements))
       (funcs (mapcar 'cdr funcons)))
  (switch-to-buffer "*org-doc*")
  (erase-buffer)
  (insert (format "#+TITLE: Documentation for %s
#+OPTIONS: toc:nil
\\maketitle
\\tableofcontents
" "/Users/jkitchin/Dropbox/kitchingroup/jmax/jmax-bibtex.el"))
  (insert "* Variables\n")
  (dolist (var (sort vars 'string-lessp))
    (insert (format "** %s
Documentation: %s\n\n" var  (documentation-property var 'variable-documentation))))

  (insert "* Functions\n\n")
  (dolist (funcs (sort funcs 'string-lessp))
    (insert (format "** %s %s
Documentation: %s

Code:
#+BEGIN_SRC emacs-lisp
%S
#+END_SRC
"
                    funcs
                    (or (help-function-arglist funcs) "")
                    (documentation funcs)
                    (symbol-function funcs))))

  (org-mode)
  (write-file "jmax-bibtex-doc.org")
  (org-export-to-file 'latex "jmax-bibtex-doc.tex")
  (org-latex-compile "jmax-bibtex-doc.tex")
  (kill-buffer "*org-doc*")
  (kill-buffer "jmax-bibtex-doc.org"))

Here is the resulting pdf: jmax-bibtex-doc.pdf . It is not too bad. The code is not beautiful, and it would take some work to get that looking nice. It might be nice to find all instances of '` and replace them with links to variable names, but I leave that for another day. There is also no information about the header comments, but I leave this for another day to.

org-mode source

Org-mode version = 8.2.7c

Randomize a list in Emacs

Sat, 06 Sep 2014 10:08:04 EDT

I have an application where I have a list of userids, and I want to randomize the order of the list. Today, I explore some ways to do that. The first idea is to simply mimic the algorithm in Python's random.shuffle algorithm.

    def shuffle(self, x, random=None):
        """x, random=random.random -> shuffle list x in place; return None.

        Optional arg random is a 0-argument function returning a random
        float in [0.0, 1.0); by default, the standard random.random.

        """

        if random is None:
            random = self.random
        _int = int
        for i in reversed(xrange(1, len(x))):
            # pick an element in x[:i+1] with which to exchange x[i]
            j = _int(random() * (i+1))
            x[i], x[j] = x[j], x[i]

It looks like we loop through the elements, and swap them at random.

We have a similar feature for xrange in emacs-lisp:

(number-sequence 1 5)

Note that number-sequence includes the last value, unlike xrange. And for reverse:

(reverse (number-sequence 1 5))

Of course, we can select random numbers:

(random 5) ; random between 0 and 5

Last, we need to work out how to swap to elements. It looks like this will swap elements 2 and 3. We store element 3 temporarily, set 3 to 2, and then set 2 to the temporarily stored value of 3.

(let* ((L '(1 2 3 4))
       (tmp (nth 3 L)))
  (setf (nth 3 L) (nth 2 L))
  (setf (nth 2 L) tmp)
L)

So, now we can shuffle our list.

(setq userids '(user1 user2 user3 user4 user5 user6))

(defun swap (LIST el1 el2)
  "in LIST swap indices EL1 and EL2 in place"
  (let ((tmp (nth el1 LIST)))
    (setf (nth el1 LIST) (nth el2 LIST))
    (setf (nth el2 LIST) tmp)))

;; now run the loop
(loop for i in (reverse (number-sequence 1 (1- (length userids))))
      do (let ((j (random (+ i 1))))
           (swap userids i j)))

userids

user4

user6

user3

user2

user1

user5

The order has certainly changed. It is a little difficult to tell how randomized it actually is, but what is important for my application is that the order is different each time I use it. It looks like this will accomplish that objective. I think this basically implements the algorithm in the Python random.shuffle code. That code does something a little differently. It generates a random float between 0-1, multiplies it by i + 1, and converts the result to an integer. We directly get an integer in the range of 0 to i + 1. I think the result is practically the same.

Finally, let us wrap the whole thing up in a nice neat function for future use. We will use elt instead of nth so it works for arrays too.

(defun swap (LIST el1 el2)
  "in LIST swap indices EL1 and EL2 in place"
  (let ((tmp (elt LIST el1)))
    (setf (elt LIST el1) (elt LIST el2))
    (setf (elt LIST el2) tmp)))


(defun shuffle (LIST)
  "Shuffle the elements in LIST.
shuffling is done in place."
  (loop for i in (reverse (number-sequence 1 (1- (length LIST))))
        do (let ((j (random (+ i 1))))
             (swap LIST i j)))
  LIST)

shuffle

Example usage for a list:

(shuffle '(user1 user2 user3 user4 user5 user6))

user4

user2

user3

user5

user6

user1

And for a vector:

(shuffle [user1 user2 user3 user4 user5 user6])

[user3 user2 user6 user4 user5 user1]

1 Addendum

Artur at http://endlessparentheses.com suggested one can use psetf to swap values. Thanks for the tip, I was not aware of that cool function. It evaluates the values first, then sets them, so there is no need for a temporary storage of a value! Here is an example usage. We could rewrite our swap function like this if we wanted.

(let ((LIST '(1 2 3 4 5)))
  (psetf (elt LIST 2) (elt LIST 1)
         (elt LIST 1) (elt LIST 2))
LIST)

org-mode source

Org-mode version = 8.2.7c