Another step towards HyDE

Posted April 06, 2016 at 04:41 PM | categories: hylang | tags:

(In case that isn't clear, it is a Python with a Lisp, and somewhat how I feel trying to see what this is all about ;)

I have been chipping away at HyDE, the Hy Development Environment. So far I have reasonable support for eldoc in hy code, and some integration into lispy.

The executive summary here is:

  1. We have a first approach at getting eldoc to work.
  2. We have better integration with lispy, including the inline describe and arg functionality, and jump to symbol.

You may want to just watch the video to see what it is about: https://www.youtube.com/watch?v=m62oiB6Feeo

The full details of these implementations are described at the end of this post. For now, we just load the functionality here: 

(require 'mile-hy)

mile-hy

Some examples of usage: 

(list (butlast [12 34 56]))

[12L, 34L]

For details of the implementations see the following discussion.

https://github.com/jkitchin/jmax/blob/master/mile-hy.el includes:

  • Setting up hy-mode with a repl (partly from lispy)
  • an eldoc function
  • some hooks for hy-mode to setup some functionality.
  • A hy src block expansion template for org-mode
  • the contents of what I previously called ob-hy.el

These things don't work without the code at https://github.com/jkitchin/hy/blob/hydoc/hy/core/hylp.hy which provides some hy functions to get eldoc strings, and file locations. These are still work in progress, but functional at this point.

I also adapted some parts of lispy to support hy better in my fork:

These are not fully ready to be integrated to lispy yet, as the functionality here depends on the hy code (in hylp.hy) that isn't part of hy yet. I hacked lispy anyway to see how easy it would be to extend it for hy. So far, it looks promising.

All of these put together lead to:

  1. eldoc support in hy code
  2. lispy support for C-1 descriptions and C-2 args inline.
  3. lispy support for M-. to jump to symbol definitions (although it is a tad hacky and uses hy to generate an org-link to open the code files at the line of the definition ;).

It is far from fully functional and there are some issues to resolve. The code will also probably change as I resolve some of those issues, but it works well enough to put some ideas out there. Feedback is welcome!

The eldoc functions work pretty well for the hy stuff. They don't work too well on the python side. The function at https://github.com/jkitchin/hy/blob/hydoc/hy/core/hylp.hy#L120 is where some improvement is needed. Right now it really only catches a few types of symbols.

For learning hylang a documentation sprint would be great.

  1. Add examples to the functions.
  2. Add documentation to the compiler @build definitions. Find a way to make this discoverable the way they are for functions and macros.
  3. Figure out how to get Python help in. It might be confusing because the syntax and examples will be different.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Getting hylp in hy

Posted April 03, 2016 at 07:41 PM | categories: hylang | tags:

Updated April 03, 2016 at 08:18 PM

Hylang is a composite of hy functions, macros, compiler code, and Python. To me, this makes it critical to get to documentation and code easily to figure out what is happening. Here we look at how to get help.

I have hacked something called hydoc for hylang. This was a battle! There are functions, macros, and builtins that are defined in different places, and some things defined in hy, some in python, and some are done at compile time, … I found a lot of things, but there are still some corner cases I am sure. For some information, I resorted to parsing the source files myself to get arguments and file positions.

See the main code here: https://github.com/jkitchin/hy/blob/hydoc/hy/core/hylp.hy and the cmd-line script here: https://github.com/jkitchin/hy/blob/hydoc/bin/hydoc

It is the beginning of the HyDE, or Hy Development in Emacs. This is a precursor to getting eldoc working in Emacs for Hy. So, without further delay, here is an example command-line use. It is not too fast, but it works. 

hydoc butlast

Usage: Usage: (butlast coll)

Returns coll except of last element.

[[/Users/jkitchin/Dropbox/python/hy/hy/core/language.hy::46]]

Yep, that is an org-mode link that jumps right to the source definition (in Emacs of course). We can also use it in hy. I have for now put this library (which contains functions and macros) in hy.core.hylp. It may not stay there. 

(import [hy.core.hylp [*]])
(require hy.core.hylp)

(? "with")

Usage: (with args &rest body)

shorthand for nested with* loops:
  (with [x foo y bar] baz) ->
  (with* [x foo]
    (with* [y bar]
      baz))

[[/Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy::34]]

Compare that to: 

(help butlast)

Help on function butlast in module hy.core.language:

butlast(coll)
    Returns coll except of last element.

pydoc/? is better because it works on macros too, and gives you more information, and in the form that you use in hy, e.g. (butlast coll) not butlast(coll).

I should point out, this is not in hy core anywhere by my github fork right now. It is still being developed. And it isn't perfect or comprehensive yet.

Let's see how good. How about we auto-generate some documentation? We will try to generate help for all the core language functions, shadowed functions, macros, and the compiler @build functions.

This is still hackier than I would like, but there is some tricky name-mangling in hy if there is a - in the name, e.g. what we use for minus, and also if there is a * in the name, it seems to get expanded. I still don't understand why I need to eval all of this here, but it works, and I get an error about no attribute named key if I don't. It seems to have some consequences though of turning some things into Python objects (especially itertools). It is good enough to share for now. Maybe someone will have a good idea ;)

Here is the code that generates the docs with (in org-mode) links to the source! The output follows, and is pretty long. The nice thing about this is the docs are generated, so we can update them pretty readily with new versions.

One thing that stands out is the lack of documentation on the compiler defined things. It might be worth figuring out how to put documentation on them, perhaps as an optional argument to the build decorator? 

(import [hy.core.hylp [*]])
(require hy.core.hylp)

(print "*** hy version " (. hy __version__))
(print "**** Language")
(for [key (sorted (hy-language-keywords))]
  (print "***** " key)
  (print (eval `(? ~(string key)))))

(print "**** Shadowed")
(for [key (sorted (hy-shadow-keywords))]
  (print "***** " key)
  (print (eval `(? ~(string key)))))

(print "**** Macros")
(for [key (sorted (hy-macro-keywords))]
  (print "***** " key)
  (print (eval `(? ~(string key)))))

(print "**** Compiler functions")
(for [key (sorted (hy-compiler-keywords))]
  (print "***** " key)
  (print (eval `(? ~(string key)))))

1 hy version 0.11.0

1.1 Language

  1. *map

    Usage: (*map unknown args)

    starmap(function, sequence) –> starmap object

    Return an iterator whose values are returned from the function evaluated with a argument tuple taken from the given sequence.

    no code::-1

  2. accumulate

    Usage: (accumulate iterable &optional [func operator.add])

    accumulate(iterable[, func]) –> accumulate object

    Return series of accumulated sums (or other binary function results).

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  3. butlast

    Usage: (butlast coll)

    Returns coll except of last element.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  4. calling_module_name

    Usage: (calling_module_name &optional [n 1])

    Get the name of the module calling `n` levels up the stack from the `calling-module-name` function call (by default, one level up)

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  5. chain

    Usage: (chain unknown args)

    chain(*iterables) –> chain object

    Return a chain object whose .next() method returns elements from the first iterable until it is exhausted, then elements from the next iterable, until all of the iterables are exhausted.

    no code::-1

  6. combinations

    Usage: (combinations unknown args)

    combinations(iterable, r) –> combinations object

    Return successive r-length combinations of elements in the iterable.

    combinations(range(4), 3) –> (0,1,2), (0,1,3), (0,2,3), (1,2,3)

    no code::-1

  7. compress

    Usage: (compress unknown args)

    compress(data, selectors) –> iterator over selected data

    Return data elements corresponding to true selector elements. Forms a shorter iterator from selected data elements using the selectors to choose the data elements.

    no code::-1

  8. cons

    Usage: (cons a b)

    Return a fresh cons cell with car = a and cdr = b

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  9. count

    Usage: (count unknown args)

    count(start=0, step=1) –> count object

    Return a count object whose .next() method returns consecutive values. Equivalent to:

    def count(firstval=0, step=1): x = firstval while 1: yield x x += step

    no code::-1

  10. cycle

    Usage: (cycle unknown args)

    cycle(iterable) –> cycle object

    Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely.

    no code::-1

  11. dec

    Usage: (dec n)

    Decrement n by 1

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  12. disassemble

    Usage: (disassemble tree &optional [codegen false])

    Return the python AST for a quoted Hy tree as a string. If the second argument is true, generate python code instead.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  13. distinct

    Usage: (distinct coll)

    Return a generator from the original collection with duplicates removed

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  14. drop

    Usage: (drop count coll)

    Drop `count` elements from `coll` and yield back the rest

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  15. drop_last

    Usage: (drop_last n coll)

    Return a sequence of all but the last n elements in coll.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  16. drop_while

    Usage: (drop_while unknown args)

    dropwhile(predicate, iterable) –> dropwhile object

    Drop items from the iterable while predicate(item) is true. Afterwards, return every element until the iterable is exhausted.

    no code::-1

  17. filter

    Usage: (filter unknown args)

    ifilter(function or None, sequence) –> ifilter object

    Return those items of sequence for which function(item) is true. If function is None, return the items that are true.

    no code::-1

  18. first

    Usage: (first coll)

    Return first item from `coll`

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  19. flatten

    Usage: (flatten coll)

    Return a single flat list expanding all members of coll

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  20. fraction

    Usage: (fraction unknown args)

    This class implements rational numbers.

    In the two-argument form of the constructor, Fraction(8, 6) will produce a rational number equivalent to 4/3. Both arguments must be Rational. The numerator defaults to 0 and the denominator defaults to 1 so that Fraction(3) = 3 and Fraction() = 0.

    Fractions can also be constructed from:

    • numeric strings similar to those accepted by the float constructor (for example, '-2.3' or '1e10')
    • strings of the form '123/456'
    • float and Decimal instances
    • other Rational instances (including integers)

    no code::-1

  21. gensym

    Usage: (gensym &optional [g "G"])

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  22. group_by

    Usage: (group_by unknown args)

    groupby(iterable[, keyfunc]) -> create an iterator which returns (key, sub-iterator) grouped by each value of key(value).

    no code::-1

  23. identity

    Usage: (identity x)

    Returns the argument unchanged

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  24. inc

    Usage: (inc n)

    Increment n by 1

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  25. input

    Usage: (input unknown args)

    raw_input([prompt]) -> string

    Read a string from standard input. The trailing newline is stripped. If the user hits EOF (Unix: Ctl-D, Windows: Ctl-Z+Return), raise EOFError. On Unix, GNU readline is used if enabled. The prompt string, if given, is printed without a trailing newline before reading.

    no code::-1

  26. integer

    Usage: (integer x)

    Return Hy kind of integer

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  27. interleave

    Usage: (interleave &rest seqs)

    Return an iterable of the first item in each of seqs, then the second etc.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  28. interpose

    Usage: (interpose item seq)

    Return an iterable of the elements of seq separated by item

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  29. is_coll

    Usage: (is_coll coll)

    Checks whether item is a collection

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  30. is_cons

    Usage: (is_cons c)

    Check whether c can be used as a cons object

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  31. is_empty

    Usage: (is_empty coll)

    Return True if `coll` is empty

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  32. is_even

    Usage: (is_even n)

    Return true if n is an even number

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  33. is_every

    Usage: (is_every pred coll)

    Return true if (pred x) is logical true for every x in coll, else false

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  34. is_float

    Usage: (is_float x)

    Return True if x is float

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  35. is_instance

    Usage: (is_instance klass x)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  36. is_integer

    Usage: (is_integer x)

    Return True if x in an integer

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  37. is_integer_char

    Usage: (is_integer_char x)

    Return True if char `x` parses as an integer

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  38. is_iterable

    Usage: (is_iterable x)

    Return true if x is iterable

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  39. is_iterator

    Usage: (is_iterator x)

    Return true if x is an iterator

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  40. is_keyword

    Usage: (is_keyword k)

    Check whether k is a keyword

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  41. is_neg

    Usage: (is_neg n)

    Return true if n is < 0

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  42. is_nil

    Usage: (is_nil x)

    Return true if x is nil (None)

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  43. is_none

    Usage: (is_none x)

    Return true if x is None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  44. is_numeric

    Usage: (is_numeric x)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  45. is_odd

    Usage: (is_odd n)

    Return true if n is an odd number

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  46. is_pos

    Usage: (is_pos n)

    Return true if n is > 0

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  47. is_string

    Usage: (is_string x)

    Return True if x is a string

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  48. is_symbol

    Usage: (is_symbol s)

    Check whether s is a symbol

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  49. is_zero

    Usage: (is_zero n)

    Return true if n is 0

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  50. islice

    Usage: (islice unknown args)

    islice(iterable, [start,] stop [, step]) –> islice object

    Return an iterator whose next() method returns selected values from an iterable. If start is specified, will skip all preceding elements; otherwise, start defaults to zero. Step defaults to one. If specified as another value, step determines how many values are skipped between successive calls. Works like a slice() on a list but returns an iterator.

    no code::-1

  51. iterate

    Usage: (iterate f x)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  52. keyword

    Usage: (keyword value)

    Create a keyword from the given value. Strings numbers and even objects with the name magic will work

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  53. last

    Usage: (last coll)

    Return last item from `coll`

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  54. list*

    Usage: (list* hd &rest tl)

    Return a dotted list construed from the elements of the argument

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  55. macroexpand

    Usage: (macroexpand form)

    Return the full macro expansion of form

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  56. macroexpand_1

    Usage: (macroexpand_1 form)

    Return the single step macro expansion of form

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  57. map

    Usage: (map unknown args)

    imap(func, *iterables) –> imap object

    Make an iterator that computes the function using arguments from each of the iterables. Like map() except that it returns an iterator instead of a list and that it stops when the shortest iterable is exhausted instead of filling in None for shorter iterables.

    no code::-1

  58. merge_with

    Usage: (merge_with f &rest maps)

    Returns a map that consists of the rest of the maps joined onto the first. If a key occurs in more than one map, the mapping(s) from the latter (left-to-right) will be combined with the mapping in the result by calling (f val-in-result val-in-latter).

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  59. multicombinations

    Usage: (multicombinations unknown args)

    combinations_with_replacement(iterable, r) –> combinations_with_replacement object

    Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats. combinations_with_replacement('ABC', 2) –> AA AB AC BB BC CC

    no code::-1

  60. name

    Usage: (name value)

    Convert the given value to a string. Keyword special character will be stripped. String will be used as is. Even objects with the name magic will work

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  61. nth

    Usage: (nth coll n &optional [default nil])

    Return nth item in collection or sequence, counting from 0. Return nil if out of bounds unless specified otherwise.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  62. partition

    Usage: (partition coll &optional [n 2] step [fillvalue -sentinel])

    Chunks coll into n-tuples (pairs by default). The remainder, if any, is not included unless a fillvalue is specified. The step defaults to n, but can be more to skip elements, or less for a sliding window with overlap.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  63. permutations

    Usage: (permutations unknown args)

    permutations(iterable[, r]) –> permutations object

    Return successive r-length permutations of elements in the iterable.

    permutations(range(3), 2) –> (0,1), (0,2), (1,0), (1,2), (2,0), (2,1)

    no code::-1

  64. product

    Usage: (product unknown args)

    product(*iterables) –> product object

    Cartesian product of input iterables. Equivalent to nested for-loops.

    For example, product(A, B) returns the same as: ((x,y) for x in A for y in B). The leftmost iterators are in the outermost for-loop, so the output tuples cycle in a manner similar to an odometer (with the rightmost element changing on every iteration).

    To compute the product of an iterable with itself, specify the number of repetitions with the optional repeat keyword argument. For example, product(A, repeat=4) means the same as product(A, A, A, A).

    product('ab', range(3)) –> ('a',0) ('a',1) ('a',2) ('b',0) ('b',1) ('b',2) product((0,1), (0,1), (0,1)) –> (0,0,0) (0,0,1) (0,1,0) (0,1,1) (1,0,0) …

    no code::-1

  65. range

    Usage: (range unknown args)

    xrange(stop) -> xrange object xrange(start, stop[, step]) -> xrange object

    Like range(), but instead of returning a list, returns an object that generates the numbers in the range on demand. For looping, this is slightly faster than range() and more memory efficient.

    no code::-1

  66. read

    Usage: (read &optional [from-file sys.stdin] [eof ""])

    Read from input and returns a tokenized string. Can take a given input buffer to read from

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  67. read_str

    Usage: (read_str input)

    Reads and tokenizes first line of input

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  68. reduce

    Usage: (reduce unknown args)

    reduce(function, sequence[, initial]) -> value

    Apply a function of two arguments cumulatively to the items of a sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). If initial is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty.

    no code::-1

  69. remove

    Usage: (remove unknown args)

    ifilterfalse(function or None, sequence) –> ifilterfalse object

    Return those items of sequence for which function(item) is false. If function is None, return the items that are false.

    no code::-1

  70. repeat

    Usage: (repeat unknown args)

    repeat(object [,times]) -> create an iterator which returns the object for the specified number of times. If not specified, returns the object endlessly.

    no code::-1

  71. repeatedly

    Usage: (repeatedly func)

    Yield result of running func repeatedly

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  72. rest

    Usage: (rest coll)

    Get all the elements of a coll, except the first.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  73. second

    Usage: (second coll)

    Return second item from `coll`

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  74. some

    Usage: (some pred coll)

    Return the first logical true value of (pred x) for any x in coll, else nil

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  75. string

    Usage: (string x)

    Cast x as current string implementation

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  76. take

    Usage: (take count coll)

    Take `count` elements from `coll`, or the whole set if the total number of entries in `coll` is less than `count`.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  77. take_nth

    Usage: (take_nth n coll)

    Return every nth member of coll raises ValueError for (not (pos? n))

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  78. take_while

    Usage: (take_while unknown args)

    takewhile(predicate, iterable) –> takewhile object

    Return successive entries from an iterable as long as the predicate evaluates to true for each entry.

    no code::-1

  79. tee

    Usage: (tee unknown args)

    tee(iterable, n=2) –> tuple of n independent iterators.

    no code::-1

  80. xor

    Usage: (xor a b)

    Perform exclusive or between two parameters

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/language.hy

  81. zip

    Usage: (zip unknown args)

    izip(iter1 [,iter2 […]]) –> izip object

    Return a izip object whose .next() method returns a tuple where the i-th element comes from the i-th iterable argument. The .next() method continues until the shortest iterable in the argument sequence is exhausted and then it raises StopIteration. Works like the zip() function but consumes less memory by returning an iterator instead of a list.

    no code::-1

  82. zip_longest

    Usage: (zip_longest unknown args)

    izip_longest(iter1 [,iter2 […]], [fillvalue=None]) –> izip_longest object

    Return an izip_longest object whose .next() method returns a tuple where the i-th element comes from the i-th iterable argument. The .next() method continues until the longest iterable in the argument sequence is exhausted and then it raises StopIteration. When the shorter iterables are exhausted, the fillvalue is substituted in their place. The fillvalue defaults to None or can be specified by a keyword argument.

    no code::-1

1.2 Shadowed

  1. !=

    Usage: (!= &rest args)

    Shadow != operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  2. *

    Usage: (* &rest args)

    Shadow * operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  3. +

    Usage: (+ &rest args)

    Shadow + operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  4. -

    Usage: (- &rest args)

    Shadow - operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  5. /

    Usage: (/ &rest args)

    Shadow / operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  6. <

    Usage: (< &rest args)

    Shadow < operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  7. <=

    Usage: (<= &rest args)

    Shadow <= operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  8. =

    Usage: (= &rest args)

    Shadow = operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  9. >

    Usage: (> &rest args)

    Shadow > operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  10. >=

    Usage: (>= &rest args)

    Shadow >= operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

1.3 Macros

  1. ?

    Usage: (? sym)

    Return help for SYM which is a string.

    hylp.hy

  2. _>

    Usage: (_> head &rest rest)

    Threads the head through the rest of the forms. Inserts head as the second item in the first form of rest. If there are more forms, inserts the first form as the second item in the second form of rest, etc.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  3. _>>

    Usage: (_>> head &rest rest)

    Threads the head through the rest of the forms. Inserts head as the last item in the first form of rest. If there are more forms, inserts the first form as the last item in the second form of rest, etc.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  4. car

    Usage: (car thing)

    Get the first element of a list/cons

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  5. cdr

    Usage: (cdr thing)

    Get all the elements of a thing, except the first

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  6. cond

    Usage: (cond &rest branches)

    shorthand for nested ifs: (cond [foo bar] [baz quux]) -> (if foo bar (if baz quux))

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  7. defmacro/g_bang

    Usage: (defmacro/g_bang name args &rest body)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  8. defmain

    Usage: (defmain args &rest body)

    Write a function named "main" and do the if main dance

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  9. defn

    Usage: (defn name lambda-list &rest body)

    define a function `name` with signature `lambda-list` and body `body`

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/bootstrap.hy

  10. doto

    Usage: (doto form &rest expressions)

    Performs a sequence of potentially mutating actions on an initial object, returning the resulting object

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  11. for

    Usage: (for args &rest body)

    shorthand for nested for loops: (for [x foo y bar] baz) -> (for* [x foo] (for* [y bar] baz))

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  12. hylp_info

    Usage: (hylp_info sym)

    Return Usage, docstring filename, lineno for the string SYM.

    hylp.hy

  13. if

    Usage: (if &rest args)

    if with elif

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/bootstrap.hy

  14. if_not

    Usage: (if_not test not-branch &optional yes-branch)

    Like `if`, but execute the first branch when the test fails

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  15. if_python2

    Usage: (if_python2 python2-form python3-form)

    If running on python2, execute python2-form, else, execute python3-form

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/bootstrap.hy

  16. let

    Usage: (let variables &rest body)

    Execute `body` in the lexical context of `variables`

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/bootstrap.hy

  17. lif

    Usage: (lif &rest args)

    Like `if`, but anything that is not None/nil is considered true.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  18. lif_not

    Usage: (lif_not test not-branch &optional yes-branch)

    Like `if-not`, but anything that is not None/nil is considered true.

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  19. macro_error

    Usage: (macro_error location reason)

    error out properly within a macro

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/bootstrap.hy

  20. unless

    Usage: (unless test &rest body)

    Execute `body` when `test` is false

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  21. when

    Usage: (when test &rest body)

    Execute `body` when `test` is true

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  22. with

    Usage: (with args &rest body)

    shorthand for nested with* loops: (with [x foo y bar] baz) -> (with* [x foo] (with* [y bar] baz))

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  23. with_gensyms

    Usage: (with_gensyms args &rest body)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  24. yield_from

    Usage: (yield_from expr)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

1.4 Compiler functions

  1. !=

    Usage: (!= &rest args)

    Shadow != operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  2. %

    Usage: % defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  3. %=

    Usage: %= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  4. &

    Usage: & defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  5. &=

    Usage: &= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  6. *

    Usage: (* &rest args)

    Shadow * operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  7. **

    Usage: ** defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  8. **=

    Usage: **= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  9. *=

    Usage: *= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  10. +

    Usage: (+ &rest args)

    Shadow + operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  11. +=

    Usage: += defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  12. ,

    Usage: , defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  13. -

    Usage: (- &rest args)

    Shadow - operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  14. .

    Usage: . defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  15. /

    Usage: (/ &rest args)

    Shadow / operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  16. //

    Usage: // defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  17. //=

    Usage: //= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  18. /=

    Usage: /= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  19. <

    Usage: (< &rest args)

    Shadow < operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  20. <<

    Usage: << defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  21. <<=

    Usage: <<= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  22. <=

    Usage: (<= &rest args)

    Shadow <= operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  23. =

    Usage: (= &rest args)

    Shadow = operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  24. >

    Usage: (> &rest args)

    Shadow > operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  25. >=

    Usage: (>= &rest args)

    Shadow >= operator for when we need to import / map it against something

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/shadow.hy

  26. >>

    Usage: >> defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  27. >>=

    Usage: >>= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  28. @

    Usage: @ defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  29. @=

    Usage: @= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  30. ^

    Usage: ^ defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  31. ^=

    Usage: ^= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  32. _=

    Usage: _= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  33. and

    Usage: and defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  34. apply

    Usage: apply defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  35. assert

    Usage: assert defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  36. assoc

    Usage: assoc defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  37. break

    Usage: break defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  38. continue

    Usage: continue defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  39. cut

    Usage: cut defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  40. def

    Usage: def defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  41. defclass

    Usage: defclass defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  42. defmacro

    Usage: defmacro defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  43. defreader

    Usage: defreader defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  44. del

    Usage: del defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  45. dict_comp

    Usage: dict_comp defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  46. dispatch_reader_macro

    Usage: dispatch_reader_macro defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  47. do

    Usage: do defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  48. eval

    Usage: eval defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  49. eval_and_compile

    Usage: eval_and_compile defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  50. eval_when_compile

    Usage: eval_when_compile defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  51. except

    Usage: except defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  52. fn

    Usage: fn defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  53. for*

    Usage: for* defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  54. genexpr

    Usage: genexpr defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  55. get

    Usage: get defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  56. global

    Usage: global defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  57. if*

    Usage: if* defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  58. import

    Usage: import defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  59. in

    Usage: in defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  60. is

    Usage: is defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  61. is_not

    Usage: is_not defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  62. lambda

    Usage: lambda defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  63. list_comp

    Usage: list_comp defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  64. nonlocal

    Usage: nonlocal defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  65. not

    Usage: not defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  66. not_in

    Usage: not_in defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  67. or

    Usage: or defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  68. quasiquote

    Usage: quasiquote defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  69. quote

    Usage: quote defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  70. raise

    Usage: raise defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  71. require

    Usage: require defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  72. set_comp

    Usage: set_comp defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  73. setv

    Usage: setv defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  74. try

    Usage: try defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  75. unquote

    Usage: unquote defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  76. unquote_splicing

    Usage: unquote_splicing defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  77. while

    Usage: while defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  78. with*

    Usage: with* defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  79. with_decorator

    Usage: with_decorator defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  80. yield

    Usage: yield defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  81. yield_from

    Usage: (yield_from expr)

    None

    file:///Users/jkitchin/Dropbox/python/hy/hy/core/macros.hy

  82. |

    Usage: | defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  83. |=

    Usage: |= defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

  84. ~

    Usage: ~ defined in hy/compiler

    No docstring available.

    file:///Users/jkitchin/Dropbox/python/hy/hy/compiler.py

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

ob-hy.el - or better integration of hylang in org-mode

Posted April 01, 2016 at 07:31 PM | categories: orgmode, hylang, emacs | tags:

The point of this post is to develop and test a more substantial integration of Hy into org-mode. We develop ob-hy.el here. This is based off of ob-clojure.el.

The next few blocks will get tangled to ob-hy.el. First, some variables. 

(require 'ob)

(add-to-list 'org-structure-template-alist
             '("hy" "#+BEGIN_SRC hy\n?\n#+END_SRC" "<src lang=\"hy\">\n?\n</src>"))

(defvar org-babel-tangle-lang-exts)
(add-to-list 'org-babel-tangle-lang-exts '("hy" . "hy"))

(defvar org-babel-default-header-args:hy '())
(defvar org-babel-header-args:hy '((:results . "output")))

org-babel-header-args:hy

Next a function to expand the code body. This will allow us to pass vars in the header. 

(defun org-babel-expand-body:hy (body params)
  "Expand BODY according to PARAMS, return the expanded body."
  (let* ((vars (mapcar #'cdr (org-babel-get-header params :var)))
         (result-params (cdr (assoc :result-params params)))
         (print-level nil)
         (print-length nil)
         (body (org-babel-trim
                (if (> (length vars) 0)
                    (concat "(let ["
                            (mapconcat
                             (lambda (var)
                               (format
                                "%S (quote %S)"
                                (car var)
                                (cdr var)))
                             vars "\n      ")
                            "]\n" body ")")
                  body))))
    (when (not (member "output" result-params))
      (setq body (format "(print (do  %s\n))" body)))
    body))

org-babel-expand-body:hy

And a function to execute the body. We still use a simple approach to write the code to a temp-file, execute it, capture the output, and delete the file. This limits things to 

(defun org-babel-execute:hy (body params)
  "Execute a block of hy code with Babel."
  (let* ((temporary-file-directory ".")
         (tempfile (make-temp-file "hy-"))
         result
         (result-params (cdr (assoc :result-params params)))
         (body (org-babel-expand-body:hy body params)))

    (with-temp-file tempfile
      (insert body))

    (unwind-protect
        (progn
          (cond
           ((member "body" result-params)
            (setq result body))
           ((member "python" result-params)
            (setq result (shell-command-to-string
                          (format "hy2py %s" tempfile))))
           ((member "ast" result-params)
            (setq result (shell-command-to-string
                          (format "hy2py -a -np %s" tempfile))))
           (t
            (setq result (shell-command-to-string
                          (format "hy %s" tempfile)))))

          (org-babel-result-cond result-params
            result
            (condition-case nil (org-babel-script-escape result)
              (error result))))
      (delete-file tempfile))))

(provide 'ob-hy)

ob-hy

Now we tangle and load those blocks. 

(org-babel-tangle)
(load-file "ob-hy.el")

t

Next, we do some tests. They are all simple tests.

1 Tests

1.1 Simple

(print "Hy world")

Hy world

We can see how this turns into Python: 

(print "Hy world")

print(u'Hy world')

or the AST: 

(print "Hy world")

Module(
    body=[Expr(value=Call(func=Name(id='print'), args=[Str(s=u'Hy world')], keywords=[], starargs=None, kwargs=None))])

Let's test :results value. It is not quite the value since we seem to get everything that is output from the script, but if you don't print stuff, it seems to get it right. 

"test"
(+ 1 2 3)

6

1.2 vars in header

Here we test out adding variables to the header lines. 

(print "Hy" data)

Hy world

Interesting, I am not sure where the space between them comes from. Let's check out the :results body option. It will show us the hy script that gets run. 

(print "Hy" data)

(let [data (quote "world")]
(print "Hy" data))

Nothing obvious about the space there. We can test out passing block results in here. 

(print data)

Hy  world

Here is the body of that: 

(print data)

(let [data (quote "Hy world
")]
(print data))

2 Summary

It works well enough to make testing in org-mode pretty convenient. I can't think of anything else it "needs" right now, although communication with a repl might make it faster, and sessions are not supported at the moment. Saving that for another day ;)

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

More on Hy and why I think it is a big deal

Posted March 31, 2016 at 01:41 PM | categories: emacs, hylang, python | tags:

Updated April 01, 2016 at 01:58 PM

Yesterday I talked about hylang , a Lisp that basically compiles to and runs Python code. Today, I want to show a few reasons why this is a great idea, and an important one. Below I give a few examples of why the hylang approach is better (in my opinion of course) than Python with a few examples of things I have always wanted in Python but couldn't get.

1 Editing with hy-mode and lispy

There is a major mode for Hy: https://github.com/hylang/hy-mode also on MELPA. It gives us some syntax highlighting and better access to a REPL.

Let's load lispy (https://github.com/abo-abo/lispy ) for it so we also get amazing editing. I always wanted to use lispy style navigation and editing in Python, but the whitespace and indentation did not make it that easy. Problem solved with these. @abo-abo already added basic eval support for Hy to lispy since the post yesterday (https://github.com/abo-abo/lispy/commit/f7f71e38e241d92b6add05be6628ac987067b11c ); Thanks! 

(add-hook 'hy-mode-hook
          (lambda ()
            (lispy-mode 1)))

2 Python with no whitespace, or commas in lists

You can still use indentation (it is good style), but this works! 

(for [x [0 1 2 3 4 5]]
(if (> x 3) (print "too big")
(print x)))

0
1
2
3
too big
too big

This looks nicer. 

(for [x [0 1 2 3 4 5]]
  (if (> x 3)
    (print "too big")
    (print x)))

0
1
2
3
too big
too big

This is a big deal too. Using Python in sessions in org-mode has always been a little complicated by the indentation and whitespace, especially with nested loops and functions. That problem is probably gone.

3 No confusion in expressions in statements

In Python you can do this: 

a = 5
print(a)
print(a + 5)

5
10

But not this: 

print(a=5)
print(a + 5)

  File "<stdin>", line 1
   print(a=5)
          ^
SyntaxError: invalid syntax

You can't put assignment statements and expression statements anywhere you want, they are only legal syntax in some places. For example, a=5 above actually looks like the print function has an argument of a that set to 5. Not true in Lisp; there are only expressions! So this works fine. 

(print (setv a 5))
(print (+ a 5))

5
10

I just like this style of simple syntax.

4 Proper multiline lambda functions

Python syntax fundamentally limits you to one line lambdas. Not so for Hy. Let's use one in a filter to print even numbers. Here is an example with a two-liner but you could make them more complicated. In Python, you have to make a separate function for this. That isn't terrible, but if it is never used for anything else, it could be avoided. 

(setv a [0 1 2 3 4 5 6 7 8])

(defn display [list filter]
  (for [x list] (if (filter x) (print x))))

(display a (lambda [x]
             (= (% x 2) 0)))

0
2
4
6
8

5 Macros and Extensible syntax

It is not easy to get real macro (code expansion) behavior in Python. Yes, there are decorators, and closures, and related things that get close to it. But there are not lisp-like macros.

Here is a (too) simple macro to allow for infix notation. It only works for two arguments, but could be extended for multiple arguments. 

(defmacro infix [code]
  (quasiquote ((unquote (get code 1))
               (unquote (get code 0))
               (unquote (get code 2)))))

(print (infix (1 + 1)))

2

If we want new syntax we can get it! 

(defreader $ [code]
  (quasiquote
   ((unquote (get code 1))
    (unquote (get code 0))
    (unquote (get code 2)))))

(print #$(1 + 1))

2

Why is this nice? Here is a math example that shows why you might want to change syntax.

5.1 Some math

See http://kitchingroup.cheme.cmu.edu/blog/2013/02/07/Solving-Bessel-s-Equation-numerically/ for the Python version of solving the Bessel equation numerically. Here we do it with hylang.

Why would we want infix notation? Here is a good reason. The prefix notation is not easy to read. Compare: 

dzdx = 1.0 / x**2 * (-x * z - (x**2 - nu**2) * y)

to 

(setv dzdx (* (/ 1.0 (** x 2)) (- (* (* -1 x) z) (* (- (** x 2) (** nu 2)) y))))

The infix notation is simpler to read. Still, the code below is not that hard to figure out, especially if there was a generalized infix notation that allowed (with parens for explicit operation precedence): 

(setv dzdx (nfx (1.0 / x**2) * ((-x * z) - ((x**2 - nu**2) * y))))

So, here is the hylang equivalent to my previous Python version. 

(import [numpy :as np])
(import [scipy.integrate [odeint]])
(import [scipy.special [jn]])
(import [matplotlib.pyplot :as plt])

(defn fbessel [Y x]
  "System of 1st order ODEs for the Bessel equation."
  (setv nu 0.0
        y (get Y 0)
        z (get Y 1))

  ;; define the derivatives
  (setv dydx z
        dzdx (* (/ 1.0 (** x 2)) (- (* (* -1 x) z) (* (- (** x 2) (** nu 2)) y))))
  ;; return derivatives
  [dydx dzdx])

(setv x0 1e-15
      y0 1.0
      z0 0.0
      Y0 [y0 z0])

(setv xspan (np.linspace 1e-15 10)
      sol (odeint fbessel Y0 xspan))

(plt.plot xspan (. sol [[Ellipsis 0]]) :label "Numerical solution")
(plt.plot xspan (jn 0 xspan) "r--" :label "Analytical solution")
(plt.legend :loc "best")

(plt.savefig "hy-ode.png")

2016-04-01 13:48:17.499 Python[12151:d13] CoreText performance note: Client called CTFontCreateWithName() using name "Lucida Grande" and got font with PostScript name "LucidaGrande". For best performance, only use PostScript names when calling this API.
2016-04-01 13:48:17.499 Python[12151:d13] CoreText performance note: Set a breakpoint on CTFontLogSuboptimalRequest to debug.
None

This looks really good to me, except for that prefix math. The array slice syntax is interesting. Not that obvious yet.

6 Interoperability with Python

http://docs.hylang.org/en/latest/tutorial.html#hy-python-interop

Write Hy code and use it in Python. Use Python code in Hy. Repeat. Sweet.

7 Integration of emacs and Hy

This isn't so beautiful but it illustrates a pretty awesome integration of Hy(python) into Emacs! 

(defmacro hy (body)
  `(let* ((temporary-file-directory ".")
          (tempfile (make-temp-file "hy-")))
     (message (format "code: %S" ,body))
     (with-temp-file tempfile
       (mapc (lambda (form) (insert (format "%s" form))) ,body))
     (read (unwind-protect
               (shell-command-to-string
                (format "hy %s" tempfile))
             (delete-file tempfile)))))

(aref (hy '((import numpy)
            (setv a (numpy.array [1 2 3]))
            (setv b (numpy.array [1 2 3]))
            (print (* a b))))
      1)

4

This isn't perfect, and there are many ways it could break down. But if you are careful to make the output "read"able, you can literally embed Hy code in Emacs lisp and use the results, a total win for Science! I feel like it might need something like progn, but that would not change what this does dramatically.

8 Hypster and Hy Society.

http://notes.pault.ag/hy-survival-guide/ ROTFL. ironically of course ;)

And the @hylang Twitter account is run by Hy Society. Nice.

9 What do we still need?

  1. Experience. Hy seems relatively young compared to other Lisps. It isn't clear yet if this could work like Python does at scale in research. I sure look forward to finding out though!
  2. Proper infix notation for engineering math. I could live with no operator precedence if it led to a quicker solution for now. As long as something like (1.0 / x**2 * (-x * z - (x**2 - nu**2) * y)) is legal!
  3. A proper integration with org-mode and the REPL.
  4. Toolchains like emacs-lisp has. I just love those. Killer debugging, access to hyperlinked documentation, code navigation, … Maybe integration with something like SLIME or CIDER? Hyder?
  5. Use it in a proper big project to find out where the limitations are, maybe Hycse as a companion to Pycse (http://kitchingroup.cheme.cmu.edu/pycse/ )? or a rewrite of http://kitchingroup.cheme.cmu.edu/dft-book/ in Hy?

Overall, I am pretty excited about this project. The syntax is a bit reminiscent of Clojure, and Racket, the former by design. Lots of new ideas still seem to be percolating in, so there is likely good stuff to see in the future!

I haven't used it enough to see the warts yet, but already the top issues I had with Python are largely addressed, so I see this as a way to continue progress with all the benefits of Python.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter
« Previous Page