The Kitchin Research Group

In a previous post I integrated some linear algebra into Emacs using the GNU Scientific library and a dynamic module. In this post, I use a similar approach that uses the Intel MKL library in conjunction with some helper elisp functions to mimic the array broadcasting features in Numpy. I thought this might be easier and lead to at least a complementary set of functionalities.

Note: I had to follow the directions here to disable some security feature on my Mac so that it would use the MKL libraries. Thanks Apple.

It is convenient to use vectors for the representation of arrays in Emacs because there are nice functions in the emacs-module.h for accessing vector properties. Also vectors sound closer to an array than a list. So what about array broadcasting, e.g. the way numpy lets you multiply a 2d array with a 1d array? If you multiply two arrays with size (m1, n1) * (m2, n2), it is required that the number of columns in the first array (n1) be equal to the number of rows in the second one (m2), and the resulting size of the array product will be (m1, n2). What should happen though when we have 1d array? This is neither a row or column vector itself, but we can treat as either one if we choose too. For example the vector [1 2 3] can be thought of as an array with the shape (1, 3), e.g. a single row with three columns, or (3, 1), i.e. three rows in a single column. We will build this capability into the module for convenience.

I still find it moderately tedious to write c functions that take emacs arguments, transform them to c arguments, do some c computations, and convert the results back to emacs values. So, we only implement one c function for this that multiplies two 2d arrays together using the cblas_dgemm routine in the MKL library. Then, we will create a complementary elisp library that will provide some additional functionality to get the shapes of vector arrays, dimensions, and allow us to multiply 1d and 2d vectors together the same way Numpy does array broadcasting.

The dynamic module code is listed in The module code. The elisp code is listed in Elisp helper functions. In the following sections we just demonstrate how to use the results.

(vector-shape [[1 2 3]
               [3 4 5]])

[2 3]

(vector-ncols [[1 2 3]
               [3 4 5]])

3

(matrix-multiply [1 1 1] [2 2 2])

6.0

(let ((A [[1 2]
          [3 4]])
      (b [1 1]))
  (matrix-multiply  A b))

[3.0 7.0]

(matrix-multiply [1 1]
                 [[1 2]
                  [3 4]])

[4.0 6.0]

(matrix-multiply [[0 1]
                  [0 0]]
                 [[0 0]
                  [1 0]])

[[1.0 0.0] [0.0 0.0]]

(let ((A [[1 2 -2 0]
          [-3 4 7 2]
          [6 0 3 1]])
      (B [[-1 3]
          [0  9]
          [1 -11]
          [4 -5]]))
  (pp (matrix-multiply A B)))

[[-3.0 43.0]
 [18.0 -60.0]
 [1.0 -20.0]]

#include "emacs-module.h"
#include "emacs-module-helpers.h"
#include <mkl.h>

int plugin_is_GPL_compatible;

emacs_value Fmkl_dgemm (emacs_env *env, ptrdiff_t nargs, emacs_value args[], void *data)
{
  double *A, *B, *C;
  int m, n, k, i, j;
  double alpha = 1.0;
  double beta = 0.0;
  
  // These will be 2d vectors
  emacs_value M0 = args[0]; // array 1 - A (m x k)
  emacs_value M1 = args[1]; // array 2 - B (k x n)

  // I need to get the number of rows and columns of each one.
  m = env->vec_size(env, M0);
  k  = 0;
  // We assume that we have a 2d array.
  emacs_value el1 = env->vec_get (env, M0, 0);
  k = env->vec_size(env, el1);
  
  // Now we know A has dimensions (m, k)
 
  emacs_value el2 = env->vec_get (env, M1, 0);
  n = env->vec_size(env, el2);
  
  // Now we know M1 had dimensions (k, n)
  
  // Now we have to build up arrays.
  // We are looking at a * b = c
  A = (double *)mkl_malloc( m*k*sizeof( double ), 64 );
  B = (double *)mkl_malloc( k*n*sizeof( double ), 64 );
  C = (double *)mkl_malloc( m*n*sizeof( double ), 64 );
  if (A == NULL || B == NULL || C == NULL) {
    printf( "\n ERROR: Can't allocate memory for matrices. Aborting... \n\n");
    mkl_free(A);
    mkl_free(B);
    mkl_free(C);
    return 1;
  }

  //populate A
  emacs_value row, ij;
  for (int i = 0; i < m; i++)
    {
      row = env->vec_get(env, M0, i);
      for (int j = 0; j < k; j++)
        {
          // get M0[i, j]
          ij = env->vec_get(env, row, j);
          A[k * i + j] = extract_double(env, ij);
        }
    }

  // populate B
  for (int i = 0; i < k; i++)
    {
      row = env->vec_get(env, M1, i);
      for (int j = 0; j < n; j++)
        {         
          // get M0[i, j]
          ij = env->vec_get(env, row, j);
          B[n * i + j] = extract_double(env, ij);
        }
    }

  // initialize C.  The solution will have dimensions of (rows1, cols2).
  for (int i = 0; i < m; i++)
    {
      for (int j = 0; j < n; j++)
        {
          C[n * i + j] = 0.0;
        }
    }

  // the multiplication is done here.
  cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, 
                m, n, k, alpha, A, k, B, n, beta, C, n);

  // now we build up the vector to return
  emacs_value vector = env->intern(env, "vector");
  emacs_value *array = malloc(sizeof(emacs_value) * m);
  emacs_value *row1;
  emacs_value vec;
  for (int i = 0; i < m; i++)
    {
      row1 = malloc(sizeof(emacs_value) * n);
      for (int j = 0; j < n; j++)
        {
          row1[j] = env->make_float(env, C[j + i * n]);
        }
      vec = env->funcall(env, vector, n, row1);
      array[i] = vec;
      free(row1);
    }

  emacs_value result = env->funcall(env, vector, m, array);
  free(array);
  return result;
}


int emacs_module_init(struct emacs_runtime *ert)
{
  emacs_env *env = ert->get_environment(ert);
  
  DEFUN("mkl-dgemm", Fmkl_dgemm, 2, 2,
        "(mkl-dgemm A B)\n"\
        "Multiply the matrices A and B. A and B must both be 2d vectors.\n" \
        "Returns the product as a vector.",
        NULL);
  provide(env, "mkl");
  
  return 0;
}

sh /opt/intel/mkl/bin/mklvars.sh intel64
gcc -Wall -m64 -I${MKLROOT}/include -fPIC -c mkl.c
gcc -shared -L${MKLROOT}/lib -Wl,-rpath,${MKLROOT}/lib -lmkl_rt -lpthread -lm -ldl -L. -lemacs-module-helpers -o mkl.so mkl.o

(add-to-list 'load-path (expand-file-name "."))
(require 'mkl)
(require 'cl)
(require 'seq)

(defun vector-shape (vec)
  "Return a vector of the shape of VEC."
  (let ((shape (vector (length vec))))
    (if (vectorp (aref vec 0))
        (vconcat shape (vector-shape (aref vec 0)))
      shape)))

(defun vector-ndims (vec)
  "Returns the number of dimensions in VEC."
  (length (vector-shape vec)))


(defun vector-numel (vec)
  "Returns the number of elements in VEC."
  (if (> (length vec) 0)
      (seq-reduce '* (vector-shape vec) 1)
    0))


(defun vector-nrows (vec)
 "Return the number of rows in VEC."
 (aref (vector-shape vec) 0))


(defun vector-ncols (vec)
 "Return the number of columns in VEC."
 (aref (vector-shape vec) 1))


(defun matrix-multiply (A B)
  "Return A * B in the matrix multiply sense."
  (cond
   ;; 1d * 1d i.e. a dot-product
   ((and (= 1 (vector-ndims A))
         (= 1 (vector-ndims B))
         (= (length A) (length B)))
    ;; this is easy to compute so we don't use dgemm.
    (seq-reduce '+ (mapcar* (lambda (a b) (* a b)) A B) 0.0))

   ;; 2d * 1d (m1, n1) * (n2, 1)
   ((and (= 2 (vector-ndims A))
         (= 1 (vector-ndims B))
         ;; ncols-A = len-B
         (= (vector-ncols A) (length B)))
    ;; transform B into a 2d column vector
    (let* ((B2d (apply 'vector (mapcar 'vector B)))
           (result  (mkl-dgemm A B2d)))
      ;; Now call (dgemm A B2d) -> (m2, 1) column vector
      ;; and convert it back to a 1d result
      (cl-map 'vector (lambda (v) (aref v 0)) result)))

   ;; 1d * 2d (1, n1) * (m2, n2) len-A = nrows-B
   ((and (= 1 (vector-ndims A))
         (= 2 (vector-ndims B))
         (= (length A) (vector-nrows B)))
    ;; transform B into a 2d row vector
    (let* ((A2d (vector A))
           (result  (mkl-dgemm A2d B)))
      ;; should be a 2d row vector
      (aref result 0)))

   ;; 2d * 2d (m1, n1) * (m2, n2) rows-A = ncols-B
   ((and (= 2 (vector-ndims A))
         (= 2 (vector-ndims B))
         (= (vector-ncols A)
            (vector-nrows B)))
    ;; call (dgemm A B) and return result
    (mkl-dgemm A B))
   (t
    ;; Error checking, getting here means none of the cases above were caught.
    ;; something is probably wrong.
    (cond
     ((or (> (vector-ndims A) 2)
          (> (vector-ndims B) 2))
      (error "One of your arrays has more than 2 dimensions. Only 1 or 2d arrays are supported"))
     ((and (= 1 (vector-ndims A))
           (= 1 (vector-ndims B))
           (not (= (length A) (length B))))
      (error "A and B must be the same length.
len(A) = %d
len(B) = %d" (length A) (length B)))
     ((and
       (= (vector-ndims A) 2)
       (= (vector-ndims B) 2)
       (not (= (vector-nrows A) (vector-ncols B))))
      (error "Your array shapes are not correct.
The number of rows in array A must equal the number of columns in array B.
There are %d rows in A and %d columns in B" (vector-nrows A) (vector-ncols B)))
     ((and
       (= (vector-ndims A) 2)
       (= (vector-ndims B) 1)
       (not (= (vector-nrows A) (length B))))
      (error "Your array shapes are not correct.
The number of rows in array A must equal the number of columns in array B.
There are %d rows in A and %d columns in B" (vector-nrows A) (length B)))
     (t
      (error "Unknown error"))))))

matrix-multiply

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7

An alternative approach to writing your own dynamic module (which requires some proficiency in c) is to use a foreign function interface (ffi). There is one for emacs at https://github.com/tromey/emacs-ffi, and it is also a dynamic module itself that uses libffi. This lets you use elisp to create functions in Emacs that actually call functions in some other library installed on your system. Here, we use this module to recreate our zeromq bindings that I previously posted.

The emacs-ffi module works fine as it is, but I found it useful to redefine one of the main macros (define-ffi-function) with the following goals in mind:

1. Allow me to specify the argument names and docstrings for each arg that contain its type and a description of the arg.
2. Document what each function returns (type and description).
3. Combine those two things into an overall docstring on the function.

These are important to me because it allows Emacs to work at its fullest potential while writing elisp code, including having the right function signatures in eldoc, and easy access to documentation of each function. You can see the new definition here. For example, here is a docstring for zmq-send using that new macro:

zmq-send is a Lisp function.

(zmq-send *SOCKET *MSG LEN FLAGS)

For more information check the manuals.

send a message part on a socket.
http://api.zeromq.org/4-2:zmq-send

*SOCKET (:pointer) Pointer to a socket.
*MSG (:pointer) Pointer to a C-string to send
LEN (:size_t) Number of bytes to send
FLAGS (:int) 

Returns: Number of bytes sent or -1 on failure. (:int)

That has everything you need to know

(define-ffi-function zmq-send-ori "zmq_send" :int (:pointer :pointer :size_t :int) zmq)

zmq-send-ori

Compare that to this docstring from the original macro.

zmq-send-ori is a Lisp function.

(zmq-send-ori G251 G252 G253 G254)

For more information check the manuals.

You can see the zeromq function definitions in elisp here. Here is a list of the functions we have created:

Type RET on a type label to view its full documentation.

zmq
  Function: Returns a pointer to the libzmq library.
zmq-close
  Function: close ØMQ socket.
zmq-connect
  Function: create outgoing connection from socket.
zmq-ctx-destroy
  Function: terminate a ØMQ context.
zmq-ctx-new
  Function: create new ØMQ context.
zmq-recv
  Function: receive a message part from a socket.
zmq-send
  Function: send a message part on a socket.
zmq-socket
  Function: create ØMQ socket.

Now we can use these to create the client, this time in elisp. Just as in the last post, you need to run the hwserver in a terminal for this to work. Here is the client code.

(let* ((context (zmq-ctx-new))
       (socket (zmq-socket context ZMQ-REQ)))

  (with-ffi-string (endpoint "tcp://localhost:5555")
    (zmq-connect socket endpoint))

  (with-ffi-string (msg "Hi there")
    (zmq-send socket msg 5 0))

  (with-ffi-string (recv (make-string 10 ""))
    (let ((status -1))
      (cl-loop do (setq status (zmq-recv socket recv 10 0)) until (not (= -1 status)))) 
    (print (ffi-get-c-string recv)))

  (zmq-close socket)
  (zmq-ctx-destroy context))

"World     "

This client basically performs the same as the previously one we built. You can see we are mixing some programming styles here. For example, we have to create pointers to string variables in advance that the ffi will be writing to later like we would do in c. We use the with-ffi-string macro which frees the pointer when we are done with it. It basically just avoids me having to create, use, and destroy the pointers myself. So, there it is, a working elisp zeromq client!

(defmacro define-ffi-library (symbol name)
  "Create a pointer named to the c library."
  (let ((library (cl-gensym))
        (docstring (format "Returns a pointer to the %s library." name)))
    (set library nil)
    `(defun ,symbol ()
       ,docstring
       (or ,library
           (setq ,library (ffi--dlopen ,name))))))

(defmacro define-ffi-function (name c-name return args library &optional docstring)
  "Create an Emacs function from a c-function.
NAME is a symbol for  the emacs function to create.
C-NAME is a string of the c-function to use.
RETURN is a type-keyword or (type-keyword docstring)
ARGS is a list of type-keyword or (type-keyword name &optional arg-docstring)
LIBRARY is a symbol usually defined by `define-ffi-library'
DOCSTRING is a string for the function to be created.

An overall docstring is created for the function from the arg and return docstrings.
"
  ;; Turn variable references into actual types; while keeping
  ;; keywords the same.
  (let* ((return-type (if (keywordp return)
                          return
                        (car return)))
         (return-docstring (format "Returns: %s (%s)"
                                   (if (listp return)
                                       (second return)
                                     "")
                                   return-type))
         (arg-types (vconcat (mapcar (lambda (arg)
                                       (if (keywordp arg)
                                           (symbol-value arg)
                                         ;; assume list (type-keyword name &optional doc)
                                         (symbol-value (car arg))))
                                     args)))
         (arg-names (mapcar (lambda (arg)
                              (if (keywordp arg)
                                  (cl-gensym)
                                ;; assume list (type-keyword name &optional doc)
                                (second arg)))
                            args))
         (arg-docstrings (mapcar (lambda (arg)
                                   (cond
                                    ((keywordp arg)
                                     "")
                                    ((and (listp arg) (= 3 (length arg)))
                                     (third arg))
                                    (t "")))
                                 args))
         ;; Combine all the arg docstrings into one string
         (arg-docstring (mapconcat 'identity
                                   (mapcar* (lambda (name type arg-doc)
                                              (format "%s (%s) %s"
                                                      (upcase (symbol-name name))
                                                      type
                                                      arg-doc))
                                            arg-names arg-types arg-docstrings)
                                   "\n"))
         (function (cl-gensym))
         (cif (ffi--prep-cif (symbol-value return-type) arg-types)))
    (set function nil)
    `(defun ,name (,@arg-names)
       ,(concat docstring "\n\n" arg-docstring "\n\n" return-docstring)
       (unless ,function
         (setq ,function (ffi--dlsym ,c-name (,library))))
       ;; FIXME do we even need a separate prep?
       (ffi--call ,cif ,function ,@arg-names))))

define-ffi-function

(add-to-list 'load-path (expand-file-name "."))
(require 'ffi)

(define-ffi-library zmq "libzmq")


(define-ffi-function zmq-ctx-new "zmq_ctx_new"
  (:pointer "Pointer to a context")
  nil zmq
  "create new ØMQ context.
http://api.zeromq.org/4-2:zmq-ctx-new")


(define-ffi-function zmq-ctx-destroy "zmq_ctx_destroy"
  (:int "status")
  ((:pointer *context)) zmq
  "terminate a ØMQ context.
http://api.zeromq.org/4-2:zmq-ctx-destroy")


(define-ffi-function zmq-socket "zmq_socket"
  (:pointer "Pointer to a socket.")
  ((:pointer *context "Created by `zmq-ctx-new '.") (:int type)) zmq
  "create ØMQ socket.
http://api.zeromq.org/4-2:zmq-socket")


(define-ffi-function zmq-close "zmq_close"
  (:int "Status")
  ((:pointer *socket "Socket pointer created by `zmq-socket'")) zmq
  "close ØMQ socket.
http://api.zeromq.org/4-2:zmq-close")


(define-ffi-function zmq-connect "zmq_connect" 
  (:int "Status")
  ((:pointer *socket "Socket pointer created by `zmq-socket'")
   (:pointer *endpoint "Char pointer, e.g. (ffi-make-c-string \"tcp://localhost:5555\")"))
  zmq
  "create outgoing connection from socket.
http://api.zeromq.org/4-2:zmq-connect")


(define-ffi-function zmq-send "zmq_send"
  (:int "Number of bytes sent or -1 on failure.")
  ((:pointer *socket "Pointer to a socket.")
   (:pointer *msg "Pointer to a C-string to send")
   (:size_t len "Number of bytes to send")
   (:int flags)) 
  zmq
   "send a message part on a socket.
http://api.zeromq.org/4-2:zmq-send")


(define-ffi-function zmq-recv "zmq_recv"
  (:int "Number of bytes received or -1 on failure.")
  ((:pointer *socket)
   (:pointer *buf "Pointer to c-string to put result in.")
   (:size_t len "Length to truncate message at.")
   (:int flags)) 
  zmq
   "receive a message part from a socket.
http://api.zeromq.org/4-2:zmq-recv")


;; We cannot get these through a ffi because the are #define'd for the CPP and
;; invisible in the library. They only exist in the zmq.h file.
(defconst ZMQ-REQ 3
  "A socket of type ZMQ_REQ is used by a client to send requests
  to and receive replies from a service. This socket type allows
  only an alternating sequence of zmq_send(request) and
  subsequent zmq_recv(reply) calls. Each request sent is
  round-robined among all services, and each reply received is
  matched with the last issued request.")

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7

I do a lot of scientific programming, and it is one of the reasons I have been learning to extend Emacs with dynamic modules. They have allowed me to add physical constants, numerical integration, root finding and linear algebra from established c-libraries to Emacs. Today I am taking a break from that and finally getting to another one of the reasons I started playing around with dynamic modules: zeromq. Zeromq is a messaging library that Jupyter uses to communicate with kernels. I thought we might get a smoother integration with Emacs and Jupyter if we could use zeromq directly to communicate between org-mode and the kernel. Currently we have to run a web server that does the communication for us via http requests. We won't solve the Jupyter problem today, but we will look at communicating with a Zeromq server from Emacs.

This might have lots of other useful applications. Suppose Emacs could communicate directly with other zeromq servers to retrieve data from, perhaps scientific data. It might even be possible for Emacs to run its own zeromq server, and other instances of Emacs could then communicate with it. Collaborative editing anyone?

Here we just implement the "Hello world" client example in the zeromq guide. The code for the server, a c-client, the mod-zmq library, a makefile and tests can be found at https://github.com/jkitchin/emacs-modules/tree/master/zeromq. All the server does is receive a string, and then send a response (in this case just the string "World") back to the client.

To run this, make sure to run the hwserver executable in a terminal. I wrapped the zeromq commands required to implement the client into a dynamic module. Since this example focuses on strings, the module returns strings to Emacs. I am not sure if that is always the right thing to do, as zeromq more generically uses bytes, but I will have to wait until I know more about zeromq to know if this is an issue.

This dynamic module uses a new feature that none of the previous posts used, and that is the user_ptr. These allow you to essentially return a reference pointer back to emacs that you can pass back to another function. That way they stay alive between function calls. For example, here we have to create a context and socket and pass these items to functions like zmq_send and zmq_recv.

The directory this library is in is not on my path, so we load it like this:

(add-to-list 'load-path (expand-file-name "."))
(require 'mod-zmq)

Here are the functions and their signatures that have been implemented so far. I only implemented the ones I needed for the client. The signatures may change in the future; this is just a proof of concept for now for the purpose of building the client.

(apropos-command "zmq*" t)
(with-current-buffer "*Apropos*" (buffer-string))

Type RET on a type label to view its full documentation.

zmq-close
  Function: (zmq-close SOCKET)
zmq-connect
  Function: (zmq-connect SOCKET ENDPOINT)
zmq-ctx-destroy
  Function: (zmq-ctx-destroy CONTEXT)
zmq-ctx-new
  Function: (zmq-ctx-new)
zmq-recv
  Function: (zmq-recv SOCKET LEN FLAGS)
zmq-send
  Function: (zmq-send SOCKET BUF FLAGS)
zmq-socket
  Function: (zmq-socket CONTEXT TYPE)

You can see the c code for the client here: hwclient.c . Here is a simple elisp version of the hwclient that basically does the same thing! The main difference is I added a while loop around the zmq-recv because sometimes it returns -1 and no result. So, here we loop until the return result is not -1. That seems to do the right thing.

(let* ((context (zmq-ctx-new))
       (socket (zmq-socket context ZMQ-REQ))
       (recv-ret -1)
       (result))

  (zmq-connect socket "tcp://localhost:5555")
  (zmq-send socket "Hello" 0)

  (while (= recv-ret -1)
    (setq result (zmq-recv socket 10 0)
          recv-ret (second result)))

  (print result)

  (zmq-close socket)
  (zmq-ctx-destroy context))

("World" 5)

Basically this creates the context, then the socket, and connects to it on port 5555 of the localhost where the server is running. Then we send the string "Hello". The server returns the string "World" and tells us it sent 5 bytes. Then we close the socket and destroy the context. There is a lot of code in the module to make this happen. A lot of it is converting args in emacs functions to things we can use in c, running a few lines of zmq commands, and then code to convert those results back to emacs values. Finally, there is code to register each function and define docstrings for them. I am not totally convinced this is the best way to do this, but it does work! An alternative might be emacs-ffi, which might enable most of this to be developed in just elisp.

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7

The goal of this post is to be able to solve equations like this one:

\[\left(\begin{array}{cccc} 0.18& 0.60& 0.57& 0.96 \\ 0.41& 0.24& 0.99& 0.58 \\ 0.14& 0.30& 0.97& 0.66 \\ 0.51& 0.13& 0.19& 0.85 \end{array} \right ) \left ( \begin{array}{c} x_0 \\ x_1 \\ x_2 \\ x_3 \end{array} \right ) = \left ( \begin{array}{c} 1.0 \\ 2.0 \\ 3.0 \\ 4.0 \end{array} \right ) \]

The answer is given as

\[x = \left ( \begin{array}{c} -4.05205 \\ -12.6056 \\ 1.66091 \\ 8.69377 \end{array} \right ) \]

The syntax we want to use is shown below, and we want it to return a vector containing the solution:

(let ((A [[0.18 0.60 0.57 0.96]
          [0.41 0.24 0.99 0.58]
          [0.14 0.30 0.97 0.66]
          [0.51 0.13 0.19 0.85]])
      (b [1.0 2.0 3.0 4.0]))
  (gsl-linalg-LU-solve A b))

Rather than put all the code in here like I have for the past several posts, I started a git repo at https://github.com/jkitchin/emacs-modules that contains this code.

The module for this post can be found here: https://github.com/jkitchin/emacs-modules/blob/master/gsl-linalg.c. There are a few notable features in it. First, I started writing/collecting some helper functions to make these modules simpler to write. For example, look how nice this looks to declare the functions and provide the feature.

int emacs_module_init(struct emacs_runtime *ert)
{
  emacs_env *env = ert->get_environment(ert);
  
  DEFUN("gsl-linalg-LU-solve", Fgsl_linalg_LU_solve, 2, 2,
        "(gsl-linalg-LU-solve A b).\n" \
        "Solve A x = b for x.\n" \
        "Returns a vector containing the solution x.",
        NULL);
  provide(env, "gsl-linalg");
  
  return 0;
}

The DEFUN and provide function are defined in https://github.com/jkitchin/emacs-modules/blob/master/emacs-module-helpers.c.

Within the module itself, we have to loop over the inputs to create the arrays that GSL wants to solve this problem. Second, after the solution is obtained, we have to build up a vector to return. The solution is in a gsl_vector, and we need to create an array of emacs_value elements containing each element of the gsl_vector as a float, and then create a vector to return to emacs. I use vectors here because it was easy to get the size of the b vector, which is also related to the size of the A matrix.

The repo has a Makefile in it, so we can build this module with:

make gsl-linalg.so

Once it is compiled, we load it like this. In this post we are in the emacs-modules directory where the gsl-linalg.so library is, and it is not on my load-path, so I add it here.

(add-to-list 'load-path (expand-file-name "."))
(require 'gsl-linalg)

gsl-linalg

Here is one function in the module:

(describe-function 'gsl-linalg-LU-solve)

gsl-linalg-LU-solve is a Lisp function.

(gsl-linalg-LU-solve &rest ARGS)

For more information check the manuals.

(gsl-linalg-LU-solve A b).
Solve A x = b for x.
Returns a vector containing the solution x.

Now, we can solve linear equations like this:

(gsl-linalg-LU-solve
 [[0.18 0.60 0.57 0.96]
  [0.41 0.24 0.99 0.58]
  [0.14 0.30 0.97 0.66]
  [0.51 0.13 0.19 0.85]]
 [1.0 2.0 3.0 4.0])

[-4.052050229573973 -12.605611395906903 1.6609116267088417 8.693766928795227]

We have a limited ability to confirm this answer. I have written a function that uses blas for multiplication of 2d vectors. You can see from this:

(gsl-blas-dgemm [[0.18 0.60 0.57 0.96]
                 [0.41 0.24 0.99 0.58]
                 [0.14 0.30 0.97 0.66]
                 [0.51 0.13 0.19 0.85]]
                [[-4.052050229573973]
                 [-12.605611395906903]
                 [1.6609116267088417]
                 [8.693766928795227]])

[[1.0] [1.9999999999999991] [2.9999999999999996] [4.0]]

That within float that indeed \(A x = b\).

The main limitation of this module at the moment is that you have to use vectors; you cannot put in a list of numbers. It is possible to make it take lists and vectors, but for now I am leaving it at vectors. Also, it only produces solutions of float numbers (not integers).

The module does not handle 1d vectors well,, e.g. in gsl-linalg-LU-solve example, the right hand side is implied to be a column vector, and we don't have the array broadcasting features of Python yet. Those are doable things for some future day perhaps. For now I am happy to have figured out how to handle arrays!

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7

The GNU Scientific Library defines a lot of physical constants. Since we are exploring how to make Emacs a more scientific environment to work in, it would be nice to import these constants to elisp. We do that through a dynamic module. This turned out to be tricky. I thought we could just use a funcall to defconst or defvar, but these are special forms and you cannot funcall them. @polizta on Stackoverflow pointed me to the path that led to success: You make a list like '(defconst sym val doc) and then eval it. That can be funcall'd, and it works nicely in the module below. It is a growing theme that it takes much hacking around to figure out how to do things like this.

The only other notable feature of this module is that I created a defconst function to make adding multiple constants less verbose. Here I only add two constants. There are about 408 constants defined in gsl_const_*.h, so brevity might be a good idea! Here is the module.

#include <gsl/gsl_const_mksa.h>
#include <string.h>
#include "emacs-module.h"

int plugin_is_GPL_compatible;

// I assume here that all values will be double. I can't think of any that would
// be ints
static void defconst (emacs_env *env, const char *name, double value, const char *doc)
{
  // These are functions we will call
  emacs_value eval = env->intern(env, "eval");  
  emacs_value list = env->intern(env, "list");

  // These will make up the list we will eventally eval
  emacs_value fdefconst = env->intern(env, "defconst");
  emacs_value sym = env->intern(env, name);
  emacs_value val = env->make_float(env, value);
  emacs_value sdoc = env->make_string(env, doc, strlen(doc));

  // make a list of (defconst sym val doc)
  emacs_value largs[] = {fdefconst, sym, val, sdoc};
  emacs_value qlist = env->funcall(env, list, 4, &largs);   

  // now eval the list of symbols
  emacs_value args[] = { qlist };  
  env->funcall(env, eval, 1, &args);
}

int emacs_module_init(struct emacs_runtime *ert)
{
  emacs_env *env = ert->get_environment(ert);

  defconst(env, "GSL-CONST-MKSA-SPEED-OF-LIGHT",
           GSL_CONST_MKSA_SPEED_OF_LIGHT,
           "Speed of light in vacuum (m/s).");
  
  defconst(env, "GSL-CONST-MKSA-PLANCKS-CONSTANT-H",
           GSL_CONST_MKSA_PLANCKS_CONSTANT_H,
           "Plank's constant, h");

  // This is what allows the shared library to provide a feature 
  emacs_value provide = env->intern(env, "provide");
  emacs_value provide_args[] = { env->intern(env, "gsl-constants") };
  env->funcall(env, provide, 1, provide_args);
  
  return 0;
}

Regular gcc will work to compile this module.

rm -f gsl-constants.so gsl-constants.o
gcc -Wall -I/usr/local/include -fPIC -c gsl-constants.c
gcc -shared -L/usr/local/include -lgsl -o gsl-constants.so gsl-constants.o

Here is in action.

(add-to-list 'load-path "/Users/jkitchin/vc/blogofile-jkitchin.github.com/_blog/dynamic-module/")
(require 'gsl-constants)
GSL-CONST-MKSA-SPEED-OF-LIGHT

299792458.0

We can see there is a docstring on that constant:

(describe-variable 'GSL-CONST-MKSA-SPEED-OF-LIGHT)

GSL-CONST-MKSA-SPEED-OF-LIGHT's value is 299792458.0


  This variable can be risky when used as a file-local variable.

Documentation:
Speed of light in vacuum (m/s).

For more information check the manuals.

It is worth thinking about what we accomplished here. The value of each constant in GSL is stored in a header file. The units are stored in a comment next to the value, and the documentation is in an html page somewhere. It is not easy to introspect that! Getting it all into an Emacs variable makes that more introspectable, and findable. That means while typing elisp code you will get completion on these variables. Check this out:

(apropos-variable "GSL-*")
(with-current-buffer "*Apropos*" (buffer-string))

Type RET on a type label to view its full documentation.

GSL-CONST-MKSA-PLANCKS-CONSTANT-H
  Variable: Plank's constant, h
GSL-CONST-MKSA-SPEED-OF-LIGHT
  Variable: Speed of light in vacuum (m/s).

It seems like it might be possible to partially automate creation of this module by parsing the gsl_const*.h files. There is no automating adding the doc strings though, I am pretty sure that will have to be done by hand ;(

Copyright (C) 2017 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.0.7