The Kitchin Research Group

Hy allows us to define functions with unicode names. Here we play around with this to define logical operators with the symbols you normally see in formal papers (i.e. the LaTeX symbols). I think in Python3 you can also define unicode names for functions. It definitely does not work for Python2.7 which we use here (although remarkably via Hy it does work).

First, we define the logical operators or, and, xor and not, and a few other interesting ones. In case it is not clear why a unicode representation of and is helpful, note there are three instances of the word and in the previous sentence, and only one is a logical operator! This is just some syntactical beauty, but it will shortly make for a different representation of code. We tangle this code block to logical_operators.hy.

(defn ∨ [a b] (or a b))

(defn ∧ [a b] (and a b))

(defn ⊕ [a b] (xor a b))

(defn ¬ [a] (not a))

(defn ∀ [x func]
  "(func x) is True for every x."
  (every? func x))

(defn ∃ [x func]
  "func(x) is true for at least one x."
  (some func x))

(defn ∃! [x func]
  "func(x) is true for exactly one x."
  (= 1 (len (list (filter func x)))))

Here is an example of using those operators. At the moment, we use the prefix notation of Lisp.

(import [logical_operators [*]])
(import [serialize [stringify]])

(defmacro show [body]
  `(do
    (print (.encode (.format "{0} = {1}" (stringify '~body) ~body) "utf-8"))))

(show (¬ True))
(show (∧ True False))
(show (∧ True True))
(show (∨ True False))
(show (∨ True True))
(show (⊕ True False))
(show (⊕ True True))
(show (⊕ False False))
(show (∀ [2 4 6] even?))
(show (∃ [2 3 4] odd?))
(show (∃! [2 3 4] odd?))
(show (∃! [2 3 5] odd?))

(¬ True) = False
(∧ True False) = False
(∧ True True) = True
(∨ True False) = True
(∨ True True) = True
(⊕ True False) = True
(⊕ True True) = False
(⊕ False False) = False
(∀ [2 4 6] is_even) = True
(∃ [2 3 4] is_odd) = True
(∃_bang [2 3 4] is_odd) = True
(∃_bang [2 3 5] is_odd) = False

Note the exclamation mark got expanded to _bang. It is evidently an ordinary ascii character.

We can get an infix notation if we use our infix module and the #$ reader macro defined in it. Here are some examples. Note it doesn't make sense to use this all time, e.g. it would even be a mistake to do this with the not operator.

(import [logical_operators [*]])
(import [serialize [stringify]])
(import [infix [*]])
(require infix)

(defmacro show [body]
  `(do
    (print (.encode (.format "{0} = {1}" (stringify '~body) ~body) "utf-8"))))

(show (¬ True))
(show #$(True ∧ False))
(show #$(True ∧ True))
(show #$(True ∨ False))
(show #$(True ∨ True))
(show #$(True ⊕ False))
(show #$(True ⊕ True))
(show #$(False ⊕ False))

(¬ True) = False
(dispatch_reader_macro "$" (True ∧ False)) = False
(dispatch_reader_macro "$" (True ∧ True)) = True
(dispatch_reader_macro "$" (True ∨ False)) = True
(dispatch_reader_macro "$" (True ∨ True)) = True
(dispatch_reader_macro "$" (True ⊕ False)) = True
(dispatch_reader_macro "$" (True ⊕ True)) = False
(dispatch_reader_macro "$" (False ⊕ False)) = False

The show macro shows us how the reader macro gets expanded into, you guessed it, regular old function calls. They are just syntactical sugar to help us be more concise. The unicode symbols are not quite as simple to type as ascii names, but there are solutions to this: abbreviations in Emacs (http://ergoemacs.org/emacs/emacs_n_unicode.html ), custom commands, learn the keystrokes (http://www.johndcook.com/blog/emacs_unicode/ ) and C-x 8 RET and the unicode name, etc… Is it worth it? That might depend on how ingrained those logical symbols are in your mental model of your work. If it is deeply ingrained, your code will be better aligned with your thoughts, and easier to understand.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Alex successfully defended his PhD on Tuesday, April 19, 2016!

Title: Micro-scale Approaches to the Bench-scale Evaluation of CO₂ Capture System Properties

Committee Members: Professor John Kitchin (chair), Professor Shelley Anna, Professor Neil Donahue, and Professor Newell Washburn.

Congratulations Alex!

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

This paper uses a kernel regression method trained on a large set of DFT calculations from the Materials Project to design new materials. A notable feature of this approach is it opens the door to inverse design, since the mathematical form of the regression is accessible. In the paper we predict electronic properties and elastic constants for a large number of metal oxides. Congratulations Bruno for this work!

See the paper here: http://onlinelibrary.wiley.com/doi/10.1002/aic.15251/full

@article {AIC:AIC15251,
  author =       {Calfa, Bruno A. and Kitchin, John R.},
  title =        {Property prediction of crystalline solids from composition and
                  crystal structure},
  journal =      {AIChE Journal},
  issn =         {1547-5905},
  url =          {https://doi.org/10.1002/aic.15251},
  doi =          {10.1002/aic.15251},
  pages =        {n/a--n/a},
  keywords =     {crystal property prediction, data analytics, kernel
                  regression, crystal composition and structure, exhaustive
                  enumeration algorithm},
  year =         {2016},
}

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Code completion is often useful in an editor. Today, we add some code completion to Emacs for hy . It isn't that hard; we get a list of known keywords from the hy language, a list of functions and macros, and a list of variables from the current buffer. If you are following this line of development, the code can be found here: https://github.com/jkitchin/jmax/blob/master/mile-hy.el

If not, there might be some interesting tips here on getting completion in Emacs ;)

We will use auto-complete (http://auto-complete.org/doc/manual.html#extend ) for now. First, we can add hy-mode to the list of ac-modes:

;; * auto-complete
(add-to-list 'ac-modes 'hy-mode)

Next, we need to define some sources and functions for completion. Over at https://github.com/jkitchin/hyve/blob/master/hylp.hy#L65 I defined a function that returns a list of all hy core functions and macros that Emacs can directly read.

(defn hy-all-keywords-emacs-completion []
  "Return a string for Emacs completion suitable for read in Emacs.
We unmangle the names and replace _ with -."
  (str
   (+ "("
      (.join " " (list-comp (.format "\"{}\"" (.replace x "_" "-"))
                            [x (hy-all-keywords)]))
      ")")))

Here, we define a source that gets that information from the hy repl using the lispy–eval-hy function. This has the downside of calling the repl, but it seems fast, and I haven't noticed any lags so far. The upside is it only gets called once and has everything hy knows about, i.e. i don't have to update this for new core functions/macros.

(defvar ac-source-hy-keywords
  `((candidates . ,(read (lispy--eval-hy "(hy-all-keywords-emacs-completion)"))))
  "Keywords known from hy. The command is defined in hyve.hylp.")

It would also be nice to have the defns/macros in the current file available for completion. This hackery searches the current buffer for these with a pretty simple regex and accumulates the results.

(defun hy-defns-macros ()
  "Get a list of defns in the current file."
  (let ((defns '()))
    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward "\\(?:defn\\|defmacro\\)[[:space:]]+\\(.*?\\) "nil t)
        (push (match-string 1) defns)))
    defns))

Finally, we would also like the variable names from setv and let. Hy is lispy, so we use a hybrid regex search, followed by read to get every other name in the case of setv, and the vector expression in the let case.

(defun hy-variables ()
  "Collect the variable names in the current buffer.
These are every other name after setv."
  (let ((vars '())
        expr
        set-vars
        let-vars)
    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward "(setv" nil t)
        (save-excursion
          (goto-char (match-beginning 0))
          (setq expr (read (current-buffer)))
          (setq set-vars (loop for x in (cdr expr) by #'cddr
                               collect x)))))
    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward "(let" nil t)
        (save-excursion
          (goto-char (match-beginning 0))
          (setq expr (read (current-buffer)))
          ;; this is read as a vector, so we convert to a list.
          (setq let-vars
                (loop for x in (append (nth 1 expr) nil)
                      by #'cddr collect x)))))
    (append set-vars let-vars)))

Next, we define two new sources for completion that use those two functions:

(defvar ac-source-hy-defns
  '((candidates . hy-defns-macros))
  "Functions/macros defined in the file.")

(defvar ac-source-hy-variables
  '((candidates . hy-variables))
  "Hy variables defined in the file.")

And finally add this to the hy-setup hook function:

(setq ac-sources '(ac-source-hy-keywords
                     ac-source-hy-defns
                     ac-source-hy-variables))

  (ac-set-trigger-key "TAB")
  (auto-complete-mode 1)

And we should be good to go with completion. Let's try it out.

Checkout the video here: https://www.youtube.com/watch?v=L6j5IWkpoz0

(let [some-long-name 5
      boring-and-tedious "tree"]
  (print boring-and-tedious))

(setv another-var nil inline-name (+ 4 5)
      hylarious-var 5)

(+ hylarious-var 8 )

(defn Some-long-function []
  (print 6))

(Some-long-function)

tree
6

Sweet.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Hy lets us do things that either aren't possible, or definitely aren't easy in Python. You may have drank the Python Koolaid and don't think those things are necessary, but we have Hy-C, and we took a sip of that just now, so let's see what we can do.

We can have functions that are punctuation!

(defn ! [arg] (not arg))

(print (! True))
(print (! False))

False
True

How about that function that just returns something truthy? Shouldn't those end in a question-mark? They can and sometimes do. Not a problem when you are hy.

(defn string? [s]
 (isinstance s str))

(print (string? 4))
(print (string? "4"))        ;; haha. strings in hy like "4" are unicode, not a str.
(print (string? (str "4")))

False
False
True
False

Isn't that better than is_a_string?

Underscores. Pfffft…. Dashes in names are awesome. Unless you hate your pinky and are shifty.

(defn 100-yard- [x]
  "Funniest function name ever. Don't do this at home."
  (.format "You ran that in {} seconds! New World Record!" 9.42))

(print (100-yard- 2))

You ran that in 9.42 seconds! New World Record!

Why not build code with code? Here is a fun way to add up only the even numbers in a list. wHy? wHy?? Because we can, and it leads to other interesting opportunities!

(import hy)
(let [a [1 2 3 4 5 6]
      code '()]
  (+= code `(+))  ;; add an operator
  (for [n a]
    (when (even? n)
      (+= code `(~(hy.models.integer.HyInteger n)))))
  (print code)

  (print (eval code)))

(u'+' 2L 4L 6L)
12

Ok, that isn't so beautiful, but it shows we can generate code and then execute it. We could also do that like we do in python where you build up the list of even numbers, and then sum them. It's the beginning of macros.

But I can't live without objects! How else can you encapsulate data? Let's see how and give you some closure to get on with programming. (yea, the puns get worse ;).

This next example illustrates a closure which we can use to encapsulate data. We use let to create a context with the variable i defined. i doesn't exist outside the context, but the lambda function created inside it retains access to the variable i.

(def counter
  (let [i [0]]
    (lambda [] (assoc i 0 (+ 1 (get i 0))) (get i 0))))

(print (counter))
(print (counter))

;; i is not a global var!
(try
 (print i)
 (except [e NameError] (print "i is not defined here!")))

1
2
i is not defined here!

Yes, the use of a list to store the counter is wonky; it is because of namespaces in Python. We get around the issue with a list here, that stores the data. Thanks Paul Tagliamonte (the resident Hypster) for the tip. Apparently Python scoping doesn't work enough here, but the list approach does, as does creating class instances to store the counter. Hylarious.

Let's check out a macro. First, here is a code example. A common pattern is to save a value in a let statement temporarily, so we can reuse it in other expressions.

(let [x (> 2 0)]
  (if x
    (print (.format "{}" x))
   (print (.format "{}" x))))

;; a one line version for comparison
(let [x (< 2 0)] (if x (print (.format "{}" x)) (print (.format "{}" x))))

True
False

That example has a lot of parentheses, and it might nice if there were fewer parentheses. There is a macro form to deal with this (it is actually defined in the hylang contrib directory, but it is short so we look at it here). This is called an anaphoric macro, because it captures a variable called "it" for reuse later in the macro. With the aif macro we can eliminate the use of the let statement in production code, eliminating a set of parentheses, and also the temporary variable.

(defmacro aif [test-form then-form &optional else-form]
  `(let [it ~test-form]
     (if it ~then-form ~else-form)))

;; In this code, it is bound to the first form value.
(print (aif (> 2 0) (.format "{}" it) (.format "{}" it)))
(print (aif (< 2 0) (.format "{}" it) (.format "{}" it)))

;; How does it work? By expanding to code.
(print (macroexpand '(aif (< 2 0) (.format "{}" it) (.format "{}" it))))

True
False
((u'fn' [] (u'setv' u'it' (u'<' 2L 0L)) (u'if' u'it' (u'.format' u'{}' u'it') (u'.format' u'{}' u'it'))))

Here is how you would do this in a regular program if you wanted to use the contrib library in hy.

(require hy.contrib.anaphoric)

(print (ap-if (> 2 0) (.format "{}" it) (.format "{}" it)))

True

Macros are useful for changing syntax and simplifying code. That works because the code in the macro is like data that can be manipulated and selectively evaluated. Here is an example of manipulating code like that. We start with an expression to add two numbers, and then modify it to be a multiplication.

(setv code '(+ 5 6))
(print (eval code))

;; change + to *
(assoc code 0 '*)
(print code)
(print (eval code))

11
(u'*' 5L 6L)
30

That is an indication that we can do some very interesting things with Lisp! Let's be fair and show this can also be done in Python. We just have to parse out the AST, and then we can manipulate it and get back to code. It isn't pretty, but doable.

import ast

# parse the statement
p = ast.parse("print 5 + 6")

exec compile(p, "<string>", "exec")
print ast.dump(p)

# Change + to *
p.body[0].values[0].op = ast.Mult()

print
exec compile(p, "<string>", "exec")
print ast.dump(p)

11
Module(body=[Print(dest=None, values=[BinOp(left=Num(n=5), op=Add(), right=Num(n=6))], nl=True)])

30
Module(body=[Print(dest=None, values=[BinOp(left=Num(n=5), op=Mult(), right=Num(n=6))], nl=True)])

That is not as clear as what we did in hy! Why? Because we had to transform the Python to AST, and manipulate it. In Lisp, the code is already in the abstract tree form, and we manipulate it more directly. It is easier to reason about.

I bet you didn't think we could use a hy program for more than one thing. Sure we may want to run it, but maybe we would like a different representation of the program than the code too. Here we define two macros that both take a program as input. One simply evaluates the program, so we can use it. The other takes the program, and outputs a LaTeX representation of it. It only converts a division expression correctly (and only if all the arguments are numbers and not other expressions), but it illustrates that we can use a program as data, and do different things with it!

(defmacro run [body] `(eval ~body))

(defmacro latex [body]
  `(cond
   [(= (car ~body) '/)
    (.format "\(\\frac{{{0}}} {{{1}}}\)"
            (get ~body 1)
            (.join " \\cdot " (list-comp (str x) [x (cut ~body 2)])))]
   [true (raise (Exception "Unsupported program"))]))

(setv code '(/ 1 2 4.0))

(print (run code))
(print (latex code))

0.125
\(\frac{1} {2 \cdot 4.0}\)

It is possible to do something kind of like this in Python. In this post I put a lisp function onto the base classes of objects so you could transform Python objects to lisp representations.

Well, that is probably enough Hy-C for the day. I am still playing around to figure out what kinds of things can we do with Hy that aren't easy or feasible in Python. These are a few of my favorite examples! If you have other cool things you do, put them in a comment hyre!

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10