More adventures in helm - more than one action

| categories: emacs | tags:

We continue our exploration of helm and now consider how to have more than one action for a selection. When you press enter, helm runs the default action defined, but you can define more than one action, and choose which one to run. How do you know if there are multiple actions? Press C-z in helm and you will get a new helm buffer showing the actions. The first action is the default, and you can select the the actions with function keys, e.g. f1 is the first action, f2 is the second action, or you can select the action and press enter.

The main difference in setting up multiple actions is that instead of a single function for action in the source definition, we provide a list of cons cells for the action element of the helm source. Each action cons cell should have a descriptive string as the car that identifies the action. This will be shown in the helm buffer. The cdr should be the function to run on the candidate. The function will be called with the selection, so the function must take one argument.

Here is an example where we have two actions. The default action will just show us the email address of the selected candidates in a message box. It will show as a list. The second action opens an email window and inserts the selected candidates in the To: field as a comma separated list. I use helm-selected-candidates in these functions instead of the just the current selected candidate so we can have multiple selections. I define the first function as a lambda function, and the second one as a defun to illustrate how to use both approaches. You can have as many actions as you want, so you could consider functions that open notes about the person, or open your contacts to look up a phone number, or functions with template emails you send often, etc…

Now, you have these options to run those actions.

  1. Make a selection and press enter. That runs the first (and default) action to show you a message box.
  2. Make a selection and press C-z to see what actions are available. Select the action you want, and press enter.
  3. Make a selection and press F1 to run the default action, or F2 to run the second action.

Here is our code.

(setq data '(("John" . "john@email.com")
             ("Jim" . "jim@email.com")
             ("Jane" . "jane@email.com")
             ("Jill" . "jill@email.com")))


(defun open-email (candidates)
  "Compose an email to the candidates. Fill in the addresses and
move point to the subject."
  (compose-mail)
  (message-goto-to)
  (insert
   (mapconcat
    'identity
    (helm-marked-candidates)
    ","))
  (message-goto-subject))

(setq some-helm-source
      `((name . "HELM at the Emacs")
        (candidates . ,data)
        (action . (("show email address" . (lambda (candidate)
                                             (message-box
                                              "selected: %s"
                                              (helm-marked-candidates))))
                   ("send email" . open-email)))))

(helm :sources '(some-helm-source))
t

Now, you can define multiple actions for your selection in helm!

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Anatomy of a helm source

| categories: helm, emacs | tags:

I have been integrating helm into my emacs work flows almost anywhere I need to make interactive selections and do something with them. In this post, I will go through the simplest helm examples I can think of that get you to writing your own example.

To run a helm selection process you basically just call a function that calls this minimal function:

(helm :sources '(some-helm-source))

In that code, the symbol some-helm-source will provide the input for the helm buffer. Let us look at the simplest example here. Each source should have a name, a list of candidates, and an action that works on the selected candidate. We construct a source as a list of cons cells. Here, we make a source with the name "HELM at the Emacs", a static list of candidates, which are simply a list of numbers, and a single action that will operate on the selected candidate.

If you run this block, you will get a helm buffer, you can select an entry, press enter, and you should see a message box pop up telling you what entry you selected. I like to separate the source definition from the helm call like this, but only for readability.

(setq some-helm-source
      '((name . "HELM at the Emacs")
        (candidates . (1 2 3 4))
        (action . (lambda (candidate)
                    (message-box "%s" candidate)))))

(helm :sources '(some-helm-source))
3

Not bad, but what if we want some dynamic candidates? The usual way we will do that is to define a function that calculates the candidates for us. Let us work out an example that just shows us random numbers between 0 and 10 to select from. In a real example, you would use this function to generate a list of candidates like bibtex keys, email-addresses, etc…

(defun random-candidates ()
  "Return a list of 4 random numbers from 0 to 10"
  (loop for i below 4 collect (random 10)))

(setq some-helm-source
      '((name . "HELM at the Emacs")
        (candidates . random-candidates)
        (action . (lambda (candidate)
                    (message "%s" candidate)))))

(helm :sources '(some-helm-source))

So far, we have looked at the simplest list of candidates: a simple list. It may be that this is not the most convenient way to see the candidates. We might like to have one set of candidates that we use for searching, but another set of equivalent candidates used for the action. For example, we might want a list of names for selecting, but then have the action work on the corresponding email address. Let us consider a case where we have a list of cons cells of names and email addresses.

We use the `, way to create the source variable to make sure our list of candidates is constructed. Then, in our function we take the selection and get the corresponding entry in the data a-list.

(setq data '(("John" . "john@email.com")
             ("Jim" . "jim@email.com")
             ("Jane" . "jane@email.com")
             ("Jill" . "jill@email.com")))

(setq some-helm-source
      `((name . "HELM at the Emacs")
        (candidates . ,(mapcar 'car data))
        (action . (lambda (candidate)
                    (message "%s" (cdr (assoc candidate data)))))))

(helm :sources '(some-helm-source))
jim@email.com

That is not too bad, and might be a general way to get to the data you want. But, helm can integrate this directly by using the a-list directly as the list of candidates. Helm will show you the car of each cell, but return the cdr of the selected entry.

Let us try this to make a function that will give us a helm buffer to select some names from, and then insert a comma separated list of emails from our selection at the point. We make our action function just return the list of marked candidates. Then we create a function that calls helm, and inserts a concatenated string.

(setq data '(("John" . "john@email.com")
             ("Jim" . "jim@email.com")
             ("Jane" . "jane@email.com")
             ("Jill" . "jill@email.com")))

(setq some-helm-source
      `((name . "HELM at the Emacs")
        (candidates . ,data)
        (action . (lambda (candidate)
                    (helm-marked-candidates)))))

(defun helm-select-and-insert-emails ()
  (interactive)
  (insert
   (mapconcat 'identity
              (helm :sources '(some-helm-source))
              ",")))
helm-select-and-insert-emails

Here is what I get when I run the command, select John and Jill, and press enter: john@email.com,jill@email.com

That is it for this post. We looked at:

  1. the simplest kind of helm interface with a fixed set of candidates
  2. A simple dynamic set of candidates
  3. A simple fixed set of candidates from a list of cons cells.

This barely scratches the surface of helm, but is already enough to do some useful things.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

A prototype implementation of jasp in emacs-lisp

| categories: lisp, ase, vasp, emacs | tags:

I want to try implementing an interface to VASP in emacs-lisp. VASP is a program that uses density functional theory to calculate properties of materials. VASP was designed for the user to create several text-based input files that contain an atomic geometry, and calculation parameters. Then you run the VASP program, which reads those input files and generates output files. Finally, you parse out the results you want from the output files.

It is moderately tedious to do that, so we already have a very extensive Python interface (http://github.com/jkitchin/jasp ) that automates the input file creation and output file parsing, and with org-mode we have a pretty good literate research environment to document what we do. But, the Python/emacs environment is not as integrated as the emacs-lisp/emacs environment is, particularly when it comes to accessing documentation. So, I want to try out a lisp approach to see what this would look like, and if it has any benefits.

The bare bones implementation will have an Atom object, to store the type of atom and its coordinates, an Atoms object which will be a collection of atoms, and a unit cell, and a Calculator object that will store calculation parameters.

Then, we will try using it to see what advantages it might have. This will be a moderately long post.

1 The Atom class

This is my first serious effort to use the object system in emacs-lisp, so I do not claim it is optimal, or beautiful. It is functional though. We make a simple Atom class that holds the chemical symbol, and xyz coordinates. We do not consider having a magnetic moment at this point, and this class has no methods.

(defclass Atom ()
  ((symbol :initarg :symbol
           :documentation "A string for the chemical symbol")
   (x :initarg :x
      :documentation "The x coordinate")
   (y :initarg :y
      :documentation "The y coordinate")
   (z :initarg :z
      :documentation "The z coordinate"))
 "A class to represent an atom.")

(provide 'Atom)
Atom

Let us try it out. We make an Atom, then get the symbol using the oref function. I don't think we need a "Name" for the atom, so the second argument is nil.

(Atom nil :symbol "C" :x 0 :y 0 :z 0)
[object Atom nil "C" 0 0 0]
(let ((a1 (Atom nil :symbol "C" :x 0 :y 0  :z 0)))
  (oref a1 symbol))
C

It is not difficult to modify an Atom object. We use oset.

(let ((a1 (Atom nil :symbol "C" :x 0 :y 0  :z 0)))
  (oset a1 x 2.5)
  a1)
[object Atom nil "C" 2.5 0 0]

2 The Atoms class

We need at Atoms object, which will store a list of Atom objects, and a unit cell. I do not know how to make this class act like a list the way that the Atoms object in ase acts. We will have to access the list of atoms as a slot. We define one method here to get the ith atom from the object.

(defclass Atoms ()
  ((atoms :initarg :atoms
          :documentation "A list of `Atom' objects")
   (cell :initarg :cell
         :documentation "A 3-tuple of lengths of an orthorhombic unit cell"))
  "A class to represent an atoms object.")


(defmethod get-atom ((atoms Atoms) &rest args)
  "Get a list of atoms in ARGS.
Return atom if ARGS contains one element, a list of atom objects
otherwise."
  (cond
   ((= 1 (length args))
    (elt (oref atoms atoms) (car args)))
   (t
    (mapcar (lambda (i)
              (elt (oref atoms atoms) i))
            args))))

(provide 'Atoms)
get-atom

That seems to do what we need. It has some limitations:

  1. We only allowed for orthorhombic unit cells
  2. We did not enable any constraints or magnetic moments on the atoms.

Here is an example Atoms object. Note we use `, notation to ensure each Atom is created, since Atom is a constructor function that returns the object.

(Atoms nil
       :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
       :cell '(8 9 10))
[object Atoms nil ([object Atom nil "C" 0 0 0] [object Atom nil "O" 1.1 0 0]) (8 9 10)]

We can drill into the object, e.g. to get the second atom:

(let ((A1 (Atoms nil
                 :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                          ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
                 :cell '(8 9 10))))
  (get-atom A1 1))
[object Atom nil "O" 1.1 0 0]

We can modify the atoms in place like this. Suppose we want to change the symbol of the first atom to "O". We use setf for this too.

(let ((A1 (Atoms nil
                 :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                          ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
                 :cell '(8 9 10))))
  (oset (get-atom A1 0) symbol "O")
  A1)
[object Atoms nil ([object Atom nil "O" 0 0 0] [object Atom nil "O" 1.1 0 0]) (8 9 10)]

The only think I do not like about this syntax is the need to get the list of atoms from the object. That is a little clunkier than the Python analog where the object is a list itself. That may be just my inexperience with emacs-lisp. Probably you can define some getter function to smooth this over.

This Atoms class lacks much of the functionality of the ase.Atoms class, but it is sufficient for this prototype.

3 The Calculator class

Next, we need our Calculator. This will store the parameters, and be responsible for creating the INCAR, POSCAR, KPOINTS, and POTCAR files, running a calculation, and getting data from the output. We also create a with-current-directory macro that will temporarily change the working directory since VASP uses the same filenames over and over, but in different directories.

(defmacro with-current-directory (directory &rest body)
  "Set the working directory temporarily set to DIRECTORY and run BODY.
DIRECTORY is expanded, and create it and its parents if needed."
  `(progn
     (unless (file-exists-p (file-name-as-directory
                             (expand-file-name ,directory)))
       (make-directory ,directory t))
     
     (let ((default-directory (file-name-as-directory
                                (expand-file-name ,directory)))) 
        ,@body)))


(defclass Jasp ()
  ((wd :initarg :wd
       :initform "."  ; default to the current working directory
       :documentation "Directory to run calculation in.")
   (encut :initarg :encut
          :documentation "Positive number in eV for planewave cutoff.
See URL `http://cms.mpi.univie.ac.at/vasp/vasp/ENCUT_tag.html'.")
   (nbands :initarg :nbands
           :documentation "Integer number of bands.
See URL `http://cms.mpi.univie.ac.at/vasp/vasp/NBANDS_tag.html'.")
   (kpts :initarg :kpts
         :initform (1 1 1)  ; default value
         :documentation "3-tuple for Monkhorst-Pack K-point grid.")
   (xc :initarg :xc
       :documentation "String of exchange correlation functional.")
   (atoms :initarg :atoms
          :documentation "An `Atoms' object."))
 "A class to represent a calculator that runs VASP.")


(defmethod view-atoms ((calc Jasp))
  "Open the ase-gui"
  (unless (and (file-exists-p "POSCAR")
               (file-exists-p "POTCAR"))
    (write-poscar calc)
    (write-potcar calc))
  (shell-command "ase-gui POSCAR"))


(defmethod write-poscar ((calc Jasp))
  "create a POSCAR file for CALC."
  (with-temp-file "POSCAR"
    (insert "Created by jasp.el\n")
    (insert "  1.0") ; unit cell scale factor
    (let* ((atoms (oref calc atoms))
           (cell (oref atoms cell)))
      (loop for v in cell
            for i below (length cell)     
            do
            (insert "\n")
            (loop for j below (length cell)
                  do
                  (if (equal i j)
                      (insert (format " %f " (float (elt cell i))))
                    (insert (format " %f " 0.0 ))))))
    ;; The next line is counts for each atom type. For each number in
    ;; this line, there will be a copy of the POTCAR in the POTCAR
    ;; file. In ase, we sort the atoms and group them so that there is
    ;; only one POTCAR per atom. We do not do that here yet. We will
    ;; have a POTCAR for each atom.
    (insert "\n")
    (loop for atom in (oref (oref calc atoms) atoms)
          do (insert (format "1 ")))
    
    ;; now we do the atoms
    (insert "\nCartesian\n")
    (loop for atom in (oref (oref calc atoms) atoms)
          do
          (insert
           (format "%f %f %f\n"
                   (oref atom x)
                   (oref atom y)
                   (oref atom z))))
    (buffer-string)))


(defmethod write-kpoints ((calc Jasp))
  "Create KPOINTS file for CALC. 
Limited to automatic generation, and no offset."
  (with-temp-file "KPOINTS"
    (insert "Automatic mesh
0
Monkhorst-Pack
")
    (dolist (k (oref calc kpts))
      (insert (format "%4d " k)))
    (insert "\n0.0 0.0 0.0")
    (buffer-string)))


(defmethod write-potcar ((calc Jasp))
  "Generate the POTCAR file for CALC.
No `Atom' grouping is done, there is one POTCAR for each atom."
  (with-temp-file "POTCAR"
    (let ((xc (oref calc xc))
          (atoms (oref calc atoms))
          (vasp_pp_path (getenv "VASP_PP_PATH")))
      (loop for atom in (oref atoms atoms)
            do
            (insert-file-contents
             (f-join
              vasp_pp_path
              (concat "potpaw_" xc)
              (oref atom symbol)
              "POTCAR"))))
    (buffer-string)))


(defmethod write-incar ((calc Jasp))
  "Generate the INCAR file for CALC."
  (with-temp-file "INCAR"
    (insert (format "ENCUT = %f\n" (oref calc encut)))
    (insert (format "NBANDS = %d\n" (oref calc nbands)))
    (buffer-string)))


(defmethod run ((calc Jasp))
  "Write out input files, and run VASP as a simple shell command"
  (write-poscar calc)
  (write-kpoints calc)
  (write-potcar calc)
  (write-incar calc)
  (shell-command "vasp"))


(defmethod update ((calc Jasp))
  "Run vasp if needed for CALC.
We just check for a properly ended OUTCAR."
  (with-current-directory
   (oref calc wd)
   (unless (and (file-exists-p "OUTCAR")
                (with-temp-buffer
                  (insert-file-contents "OUTCAR")
                  (re-search-forward
                  "                 Voluntary context switches:"
                  (point-max)
                  t)))
     (run calc))))


(defmethod get-potential-energy ((calc Jasp))
  "Get potential energy from CALC."
  (update calc)
  (with-current-directory
   (oref calc wd)
   (with-temp-buffer
     (insert-file-contents "OUTCAR")
     ;; go to last entry
     (while (re-search-forward
             "free  energy   TOTEN  =\\s-*\\([-]?[0-9]*\\.[0-9]*\\) eV"
             (point-max)
             t)
       nil)
     ;; return last match
     (string-to-number  (match-string 1)))))

(provide 'jasp)
get-potential-energy

This is worth some discussion. On one hand, the constructor is a bit more verbose than the implementation in Python. In Python we use a context handler in place of the macro here. On the other hand, that verbosity comes with detailed, accessible documentation for each argument. We only considered the simplest of input arguments. It might be trickier to include lists, and other types of input. But I think those can all be worked out like they were in ase. We only implemented the simplest job control logic, but that also can be worked out. The biggest challenge might be getting more complex data from the output. Nearly everything can be obtained from vasprun.xml also, in the event that parsing is to slow or difficult.

Now, let us test this out. We can make a calculator:

(setq calc (Jasp
            nil
            :xc "PBE"
            :encut 350
            :nbands 6
            :atoms (Atoms
                    nil
                    :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                             ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
                    :cell '(8 9 10))))
[object Jasp nil "." 350 6 (1 1 1) "PBE" [object Atoms nil ([object Atom nil "C" 0 0 0] [object Atom nil "O" 1.1 0 0]) (8 9 10)]]

We can call the class functions like this. Here we write out the corresponding POSCAR:

(write-poscar calc)
Created by jasp.el
  1.0
 8.000000  0.000000  0.000000 
 0.000000  9.000000  0.000000 
 0.000000  0.000000  10.000000 
1 1 
Cartesian
0.000000 0.000000 0.000000
1.100000 0.000000 0.000000

It looks a little backward if you have only seen Python, where this would be calc.writeposcar(). It is almost the same characters, just a different order (and no . in lisp)!

Here we get the KPOINTS file:

(write-kpoints calc)
Automatic mesh
0
Monkhorst-Pack
   1    1    1 
0.0 0.0 0.0

I cannot show the POTCAR file for licensing reasons, but it works.

(write-potcar calc)

and the INCAR file:

(write-incar calc)
ENCUT = 350.000000
NBANDS = 6

We run a calculation like this. This will run vasp directly (not through the queue system).

(run calc)
0

The returned 0 means the shell command finished correctly.

And we retrieve the potential energy like this:

(get-potential-energy calc)
-14.687906

Not bad. That is close to the result we got from a similar calculation here .

4 Putting it all together to run calculations

If we put this all together the way we might use it in practice, it looks like this.

(load-file "Atom.el")
(load-file "Atoms.el")
(load-file "Jasp.el")

(let* ((co (Atoms
            nil
            :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                     ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
            :cell '(8 9 10)))

       (calc (Jasp
              nil
              :xc "PBE"
              :nbands 6
              :encut 350
              :atoms co)))
  
  (get-potential-energy calc))
-14.687906

Compare this with this Python code which does approximately the same thing:

from ase import Atoms, Atom
from jasp import *

co = Atoms([Atom('C', [0,   0, 0]),
            Atom('O', [1.1, 0, 0])],
            cell=(6., 6., 6.))

with jasp('molecules/simple-co', #output dir
          xc='PBE',  # the exchange-correlation functional
          nbands=6,  # number of bands
          encut=350, # planewave cutoff
          atoms=co) as calc:
    print 'energy = {0} eV'.format(co.get_potential_energy())

They look pretty similar. One thing clearly missing from emacs-lisp that Python has is full support for numerics and plotting. Some of this could be addressed via Pymacs , but certainly not all of it. Some of it could also be handled using org-mode to enable data from emacs-lisp to go to other code blocks that can handle it.

Finally, for a little fun, we illustrate mapping over a range of bond lengths. There is more than one way to do this. For example, we could create a list of calculators, and then run over them. Here we create one calculator, and just change the x position in a loop. We use the more general setf approach instead of oset to see what it looks like.

(let* ((co (Atoms
            nil
            :atoms `(,(Atom nil :symbol "C" :x 0 :y 0 :z 0)
                     ,(Atom nil :symbol "O" :x 1.1 :y 0 :z 0))
            :cell '(8 9 10)))
       (calc (Jasp
              nil
              :wd nil
              :xc "PBE"
              :nbands 6
              :encut 350
              :atoms co)))
  (dolist (d '(1.05 1.1 1.15 1.2 1.25))
    ;; change working directory
    (setf (oref calc wd) (format "co-%s" d))
    ;; set x-coordinate on oxygen atom
    (setf (oref (elt (oref co atoms) 1) x) d)
    (print (format "d = %s\nEnergy = %s eV"
                   d
                   (get-potential-energy calc)))))
"d = 1.05
Energy = -14.195892 eV"

"d = 1.1
Energy = -14.698456 eV"

"d = 1.15
Energy = -14.814809 eV"

"d = 1.2
Energy = -14.660395 eV"

"d = 1.25
Energy = -14.319904 eV"

See http://kitchingroup.cheme.cmu.edu/dft-book/dft.html#sec-3-4-1 for how this was done in Python. It looks pretty similar to me.

5 Summary thoughts

We implemented a bare bones emacs-lisp calculator for VASP. The library automates creation of input files, running the calculation, and parsing the output.

It seems pretty feasible to implement a pretty complete interface to VASP in emacs-lisp. The main reasons to do this are:

  1. Integrated access to documentation
  2. Emacs editing of emacs-lisp code
  3. Integration with org-mode

Even with Python editor that had access to documentation as deeply integrated as emacs has with emacs-lisp, it would still just be a Python editor, i.e. you probably could not use the editor to write org-mode, LaTeX, etc… It is time to recognize we need both scientific document creation and code editing capability in the same place! This kind of suggests a need to get a better Python environment going in Emacs, which deeper integration of the documentation. See this post for some progress in that area!

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Capturing stderr from Python in org-mode - take 2

| categories: python, orgmode, emacs | tags:

In a previous post I wrote about a sandbox module to help capture stderr in Python code blocks in org-mode. That module worked, but ran as a script.

stderr is not captured in the output of a code block in org-mode. For example:

import sys
print >>sys.stdout, "message on stdout"
print >>sys.stderr, "testing stderr"
message on stdout

The messages to stderr just disappears. Not good for code like this:

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol
[[ 1.]
 [ 1.]]

There is an error in that code, k is not defined. If you run that as a script, you get this output:

>>> Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined

But, that is evidently going to stderr, and not getting captured in org-mode. Boo. A silent error that returns a value! This behavior of odeint may be fixed in scipy 0.15, but it is a general deficiency of org-mode babel code blocks. So, today I am looking back into a way to fix it. We try something as mundane as just redefining stderr in Python at runtime.

import sys
sys.stderr = sys.stdout

print >>sys.stdout, "message on stdout"
print >>sys.stderr, "testing stderr"
message on stdout
testing stderr

That works fine. Let us test it with the other block.

import sys
sys.stderr = sys.stdout

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
[[ 1.]
 [ 1.]]

Sweet, we get the errors. We still get the returned value, but it is immediately obvious something is wrong. I have wrapped that little tidbit into a Python module in pycse.orgmode , which you can import to get the same effect.

import pycse.orgmode

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined

[[ 1.]
 [ 1.]]

Finally, you can avoid the import by setting your org-babel Python command like this:

(setq org-babel-python-command "python -i -c \"import pycse.orgmode\"")
python -i -c "import pycse.orgmode"

Now, we run our faulty block again:

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined

[[ 1.]
 [ 1.]]

Excellent. The stderr is captured.

And we get basically the same output as before for regular code blocks. There is an extra line before and after the output for some reason. I can live with that!

print 6 + 7
13

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

A new mode for Python documentation

| categories: python, emacs | tags:

[2014-12-22 Mon] update: See this in action at http://www.youtube.com/watch?v=G_r7wTcVK54 , and see the latest source at https://github.com/jkitchin/jmax/blob/master/pydoc.el .

The emacs-lisp documentation in Emacs is inspiring. It is interlinked, you can click on links to open source files, other commands, etc… Python documentation is not that nice. It should be.

I wrote a little pydoc function:

(defun pydoc (name)
  "Display pydoc information for NAME in a buffer named *pydoc*."
  (interactive "sName of function or module: ")
  (switch-to-buffer-other-window "*pydoc*")
  (erase-buffer)
  (insert (shell-command-to-string (format "python -m pydoc %s" name)))
  (goto-char (point-min)))

which at least accesses python documentation in emacs. It looks like this:

But, this lacks functionality. I want there to be useful links in this, so I can click on the filename to open the source, or click on the packages to get their documentation. Below, we walk through a few functions that will operate on the buffer and put text properties on different pieces.

First, let us make the source file clickable so it opens the source.

(defun pydoc-make-file-link ()
  "Find FILE in a pydoc buffer and make it a clickable link"
  (goto-char (point-min))
  (when (re-search-forward "^FILE
    \\(.*\\)$" nil t)

    (let ((map (make-sparse-keymap))
          (start (match-beginning 1))
          (end (match-end 1))
          (source-file (match-string 1)))
      
      ;; set file to be clickable to open the source
      (define-key map [mouse-1]
        `(lambda ()
          (interactive)
          (find-file ,source-file)))

      (set-text-properties
       start end
       `(local-map, map
                   font-lock-face (:foreground "blue"  :underline t)
                   mouse-face highlight
                   help-echo "mouse-1: click to open")))))
pydoc-make-file-link

Next, sometimes there are URLs in the python documentation. These should all open up in a browser when you click on them. Here we propertize anything we recognize as a URL to make it open when clicked on.

(defun pydoc-make-url-links ()
  (goto-char (point-min))
  (while (re-search-forward "\\(http\\(s\\)?://.*$\\)" nil t)
    (let ((map (make-sparse-keymap))
          (start (match-beginning 1))
          (end (match-end 1)))
        
      (define-key map [mouse-1]
        `(lambda ()
          (interactive)
          (browse-url ,(buffer-substring start end))))
        
      (set-text-properties
       start end
       `(local-map ,map
                   font-lock-face (:foreground "blue"  :underline t)
                   mouse-face highlight
                   help-echo (format "mouse-1: click to open"))))))

When we get documentation for a package, we should make each entry of the package clickable, so we can get to the documentation for that package easily. We store the name of the current package so we can construct the path to the subpackage.

(defun pydoc-get-name ()
  "get NAME and store locally"
  (make-variable-buffer-local 'pydoc-name)
  (goto-char (point-min))
  (when (re-search-forward "^NAME
\\s-*\\([^-][a-zA-Z]*\\)" nil t)
    (setq pydoc-name (match-string 1))))


(defun pydoc-make-package-links ()
  "make links in PACKAGE CONTENTS"
  (goto-char (point-min))
  (when (re-search-forward "^PACKAGE CONTENTS" nil t)
    (forward-line)

    (while (string-match
            "^    \\([a-zA-Z0-9_]*\\)[ ]?\\((package)\\)?"
            (buffer-substring
             (line-beginning-position)
             (line-end-position)))
                
      (let ((map (make-sparse-keymap))
            (start (match-beginning 1))
            (end (match-end 1))
            (package (concat
                      pydoc-name "."
                      (match-string 1
                                    (buffer-substring
                                     (line-beginning-position)
                                     (line-end-position))))))
        
        (define-key map [mouse-1]
          `(lambda ()
            (interactive)
            (pydoc ,package)))
          
        (set-text-properties
         (+ (line-beginning-position) start)
         (+ (line-beginning-position) end)
         `(local-map, map
                      font-lock-face (:foreground "blue"  :underline t)
                      mouse-face highlight
                      help-echo (format "mouse-1: click to open %s" ,package))))
      (forward-line))))

Next, we put some eye candy on function names and arguments. This won't do anything functionally, but it breaks up the monotony of all black text.

(defun pydoc-colorize-functions ()
  "Change color of function names and args."
  (goto-char (point-min))
  (when (re-search-forward "^Functions" nil t)  
    (while (re-search-forward "    \\([a-zA-z0-9-]+\\)(\\([^)]*\\))" nil t)
      (set-text-properties
       (match-beginning 1)
       (match-end 1)
       '(font-lock-face (:foreground "brown")))

      (set-text-properties
       (match-beginning 2)
       (match-end 2)
       '(font-lock-face (:foreground "red"))))))

I have gotten used to the [back] link in emacs-lisp documentation, so we try to emulate it here.

(defun pydoc-insert-back-link ()
  "Insert link to previous buffer"
  (goto-char (point-max)) 
  (insert "
[back]")
  (let ((map (make-sparse-keymap)))
    
    ;; set file to be clickable to open the source
    (define-key map [mouse-1]
      (lambda ()
        (interactive)
        (pydoc *pydoc-last*)))

      (set-text-properties
       (line-beginning-position)
       (line-end-position)
       `(local-map, map
                    font-lock-face (:foreground "blue"  :underline t)
                    mouse-face highlight
                    help-echo "mouse-1: click to return"))))
pydoc-insert-back-link

Ok, finally we remake the pydoc function.

(defvar *pydoc-current* nil
 "Stores current pydoc command")

(defvar *pydoc-last* nil
 "Stores the last pydoc command")

(defun pydoc (name)
  "Display pydoc information for NAME in a buffer named *pydoc*."
  (interactive "sName of function or module: ")

  (switch-to-buffer-other-window "*pydoc*")
  (setq buffer-read-only nil)
  (erase-buffer)
  (insert (shell-command-to-string (format "python -m pydoc %s" name)))
  (goto-char (point-min))

  ;; save 
  (when *pydoc-current*
      (setq *pydoc-last* *pydoc-current*))
  (setq *pydoc-current* name)


  (save-excursion
    (pydoc-get-name)
    (pydoc-make-url-links)
    (pydoc-make-file-link)
    (pydoc-make-package-links)
    (pydoc-colorize-functions)
    (pydoc-insert-back-link))

  ;; make read-only and press q to quit
  (setq buffer-read-only t)
  (use-local-map (copy-keymap org-mode-map))
  (local-set-key "q" #'(lambda () (interactive) (kill-buffer)))

  (font-lock-mode))
pydoc

Now, we get a much more functional pydoc:

Figure 2: Annotated screenshot

and with the colorized function names:

Admittedly, there seems to be a lot of boilerplate code for propertizing the strings, but it doesn't seem too bad. I will probably use this documentation tool this spring, so maybe I will think of new functionality to add to pydoc. Any ideas?

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter
« Previous Page -- Next Page »