The Kitchin Research Group

We previously examined approaches to running code in a context. With hy, we even managed to remove the need for a with statement through the use of a macro that expanded behind the scenes to manage the context. In our jasp code, we frequently need a context manager that temporarily changes the working directory to run some code, then changes back. The use of the context manager was a design decision to avoid decorating every single function. Why? There are a lot of functions that need decorating, and they are spread over a lot of files. Not all of the entries from the next block are methods, but most of them are.

from jasp import *

c = Vasp()
print(dir(c))

['__doc__', '__init__', '__module__', '__repr__', '__str__', 'add_to_db', 'archive', 'atoms', 'bader', 'bool_params', 'calculate', 'calculation_required', 'check_state', 'chgsum', 'clean', 'clone', 'create_metadata', 'dict', 'dict_params', 'exp_params', 'float_params', 'get_atoms', 'get_beefens', 'get_bz_k_points', 'get_charge_density', 'get_default_number_of_electrons', 'get_dipole_moment', 'get_eigenvalues', 'get_elapsed_time', 'get_electronic_temperature', 'get_elf', 'get_energy_components', 'get_fermi_level', 'get_forces', 'get_ibz_k_points', 'get_ibz_kpoints', 'get_infrared_intensities', 'get_k_point_weights', 'get_local_potential', 'get_magnetic_moment', 'get_magnetic_moments', 'get_name', 'get_nearest_neighbor_table', 'get_neb', 'get_nonselfconsistent_energies', 'get_number_of_bands', 'get_number_of_electrons', 'get_number_of_grid_points', 'get_number_of_ionic_steps', 'get_number_of_iterations', 'get_number_of_spins', 'get_occupation_numbers', 'get_orbital_occupations', 'get_potential_energy', 'get_property', 'get_pseudo_density', 'get_pseudo_wavefunction', 'get_pseudopotentials', 'get_required_memory', 'get_spin_polarized', 'get_stress', 'get_valence_electrons', 'get_version', 'get_vibrational_frequencies', 'get_vibrational_modes', 'get_xc_functional', 'initialize', 'input_params', 'int_params', 'is_neb', 'job_in_queue', 'json', 'list_params', 'name', 'nbands', 'org', 'output_template', 'plot_neb', 'positions', 'post_run_hooks', 'prepare_input_files', 'pretty_json', 'python', 'read', 'read_convergence', 'read_default_number_of_electrons', 'read_dipole', 'read_eigenvalues', 'read_electronic_temperature', 'read_energy', 'read_fermi', 'read_forces', 'read_ibz_kpoints', 'read_incar', 'read_k_point_weights', 'read_kpoints', 'read_ldau', 'read_magnetic_moment', 'read_magnetic_moments', 'read_metadata', 'read_nbands', 'read_number_of_electrons', 'read_number_of_iterations', 'read_occupation_numbers', 'read_outcar', 'read_potcar', 'read_relaxed', 'read_stress', 'read_version', 'read_vib_freq', 'register_post_run_hook', 'register_pre_run_hook', 'restart', 'restart_load', 'results', 'run', 'run_counts', 'set', 'set_atoms', 'set_nbands', 'set_results', 'special_params', 'string_params', 'strip', 'strip_warnings', 'todict', 'track_output', 'update', 'write_incar', 'write_kpoints', 'write_metadata', 'write_potcar', 'write_sort_file', 'xml']

The use of a context manager is really useful for a single calculation, and it saves us a lot of boilerplate code to manage changing directories. It limits us though for looping through calculations. We are stuck with traditional for loops that have the with statement embedded in them. We also cannot get too functional, e.g. with list comprehension.

In other words, this is ok:

E = []
for d in np.linspace(1, 1.5):
    atoms = Atoms(...,d)
    with jasp('calculated-name-{}'.format(d),
              ...,
              atoms=atoms) as calc:
        E.append(atoms.get_potential_energy())

But this code is not possible:

bond_lengths = np.linspace(1, 1.5)

A = [Atoms(...,d) for d in bond_lengths]

calcs = [JASP('calculated-name-{}'.format(d),...,atoms=atoms)
         for d, atoms in zip(bond-lengths, A)]

E = [atoms.get_potential_energy() for atoms in A]

It is not legal syntax to embed a with statement inside a list comprehension. The code will not work because to get the potential energy we have to switch into the calculation directory and read it from a file there, then switch back.

To make that possible, we need to decorate the class functions so that the right thing happens when needed. I still do not want to decorate each function manually. Although there is a case to make for it, as I mentioned earlier, the functions are all over the place, and numerous. Now is not the time to fix it.

Instead, we consider a solution that will automatically decorate class functions for us! Enter the Metaclass. This is a class that modifies how classes are created. The net effect of the code below is our Calculator class now has all functions automatically decorated with a function that changes to the working directory, runs the function, and then ensures we change back even in the event of an exception. This approach is adapted from http://stackoverflow.com/questions/3467526/attaching-a-decorator-to-all-functions-within-a-class .

I am pretty sure this is the right way to do this. We cannot simply decorate the functions of ase.calculators.vasp.Vasp because our decorator needs access to the directory defined in a class instance. That is what the init method of the metaclass enables.

We will put this code into a library called meta_calculator.py for reuse in later examples.

import os
import types

class WithCurrentDirectory(type):
   """Metaclass that decorates all of its methods except __init__."""
   def __new__(cls, name, bases, attrs):
      return super(WithCurrentDirectory, cls).__new__(cls, name, bases, attrs)

   def __init__(cls, name, bases, attrs):
      """Decorate all the methods of the class instance with the classmethod cd.

      We skip __init__ because that is where the attributes are actually set.
      It is an error to access them before they are set.
      """
      for attr_name, attr_value in attrs.iteritems():
         if attr_name != '__init__' and isinstance(attr_value, types.FunctionType):
            setattr(cls, attr_name, cls.cd(attr_value))


   @classmethod
   def cd(cls, func):
      """Decorator to temporarily run cls.func in the directory stored in cls.wd.

      The working directory is restored to the original directory afterwards.
      """
      def wrapper(self, *args, **kwargs):
         if self.verbose:
            print('\nRunning {}'.format(func.__name__))
            print("Started in {}".format(os.getcwd()))
         os.chdir(self.wd)
         if self.verbose:
            print("  Entered {}".format(os.getcwd()))
         try:
            result = func(self, *args, **kwargs)
            if self.verbose:
               print('  {}'.format(result))
            return result
         except Exception, e:
            # this is where you would use an exception handling function
            print('  Caught {}'.format(e))
            pass
         finally:
            os.chdir(self.owd)
            if self.verbose:
               print("  Exited to {}\n".format(os.getcwd()))

      wrapper.__name__ = func.__name__
      wrapper.__doc__ = func.__doc__
      return wrapper


class Calculator(object):
   """Class we use for a calculator.

   Every method is decorated by the metaclass so it runs in the working
   directory defined by the class instance.

   """

   __metaclass__ = WithCurrentDirectory

   def __init__(self, wd, verbose=False):
      self.owd = os.getcwd()
      self.wd = wd
      self.verbose = verbose
      if not os.path.isdir(wd):
         os.makedirs(wd)


   def create_input(self, **kwargs):
      with open('INCAR', 'w') as f:
         for key, val in kwargs.iteritems():
            f.write('{} = {}\n'.format(key, val))


   def exc(self):
      "This raises an exception to see what happens"
      1 / 0

   def read_input(self):
      with open('INCAR', 'r') as f:
         return f.read()

   def __str__(self):
      return 'In {}. Contains: {}'.format(os.getcwd(),
                                          os.listdir('.'))

Here is how we might use it.

from meta_calculator import *

c = Calculator('/tmp/calc-1', verbose=True)
print c.create_input(xc='PBE', encut=450)
print c.read_input()
print c.exc()
print c

Running create_input
Started in /Users/jkitchin/blogofile-jkitchin.github.com/_blog
  Entered /private/tmp/calc-1
  None
  Exited to /Users/jkitchin/blogofile-jkitchin.github.com/_blog

None

Running read_input
Started in /Users/jkitchin/blogofile-jkitchin.github.com/_blog
  Entered /private/tmp/calc-1
  xc = PBE
encut = 450

  Exited to /Users/jkitchin/blogofile-jkitchin.github.com/_blog

xc = PBE
encut = 450


Running exc
Started in /Users/jkitchin/blogofile-jkitchin.github.com/_blog
  Entered /private/tmp/calc-1
  Caught integer division or modulo by zero
  Exited to /Users/jkitchin/blogofile-jkitchin.github.com/_blog

None

Running __str__
Started in /Users/jkitchin/blogofile-jkitchin.github.com/_blog
  Entered /private/tmp/calc-1
  In /private/tmp/calc-1. Contains: ['INCAR']
  Exited to /Users/jkitchin/blogofile-jkitchin.github.com/_blog

In /private/tmp/calc-1. Contains: ['INCAR']

As we can see, in each function call, we evidently do change into the path that /tmp/calc-1 points to (it is apparently /private/tmp on Mac OSX), runs the method, and then changes back out, even when exceptions occur.

Here is a functional approach to using our new calculator.

from meta_calculator import *

encuts = [100, 200, 300, 400]
calcs = [Calculator('encut-{}'.format(encut)) for encut in encuts]

# list-comprehension for the side-effect
[calc.create_input(encut=encut) for calc,encut in zip(calcs, encuts)]

inputs = [calc.read_input() for calc in calcs]

print(inputs)
print([calc.wd for calc in calcs])

['encut = 100\n', 'encut = 200\n', 'encut = 300\n', 'encut = 400\n']
['encut-100', 'encut-200', 'encut-300', 'encut-400']

Sweet. And here is our evidence that the directories got created and have the files in them.

find . -type f -print | grep "encut-[1-4]00" | xargs -n 1 -I {} -i bash -c 'echo {}; cat {}; echo'

./encut-100/INCAR
encut = 100

./encut-200/INCAR
encut = 200

./encut-300/INCAR
encut = 300

./encut-400/INCAR
encut = 400

This looks like another winner that will be making its way into jasp soon. I guess it will require at least some minor surgery on a class in ase.calculators.vasp. It might be time to move a copy of it all the way into jasp.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

We have been exploring various ways to add documentation and validation to arbitrary arguments that our molecular simulation codes use. In our previous we derived a method where we created functions that provide docstrings, and validate the input. One issue we had was the duplication of keywords and function names. Here we consider an approach that allows this kind of syntax:

calc = Calculator('/tmp',
                  encut(400),
                  xc('PBE'),
                  generic('kpts', [1, 1, 1]))

Those are regular *args, not **kwargs.

Compare this to:

calc = Calculator('/tmp',
                  encut=encut(400),
                  xc=xc('PBE'),
                  kpts=generic('kpts', [1, 1, 1]))

where those are kwargs. The duplication of the keywords is what we aim to eliminate, because 1) they are redundant, 2) why type things twice?

Here we work out an approach with *args that avoids the duplication. We use the same kind of validation functions as before, but we will decorate each one so it returns a tuple of (key, value), where key is based on the function name. This is so we don't have to duplicate the function name ourselves; we let the decorator do it for us. Then, in our Calculator class init function, we use this tuple to assign the values to self.key as the prototypical way to handle the *args. Other setter functions could also be used.

Here is the template for this approach.

def input(func):
    """Input decorator to convert a validation function to input function."""
    def inner(*args, **kwargs):
        res = func.__name__, func(*args, **kwargs)
        print('{} validated to {}'.format(func.__name__, res))
        return res
    # magic incantations to make the decorated function look like the old one.
    inner.__name__ = func.__name__
    inner.__doc__ = func.__doc__
    return inner

@input
def encut(cutoff):
    "Planewave cutoff in eV."
    assert isinstance(cutoff, int) and (cutoff > 0)
    return cutoff

@input
def xc(s):
    """Exchange-correlation functional.

    Should be 'PBE' or 'LDA'

    """
    assert isinstance(s, str)
    assert s in ['PBE', 'LDA']
    return s

def generic(key, val):
    """Generic function with no validation.

    Use this for other key,val inputs not yet written.

    """
    return (key, val)

class Calculator(object):
    def __init__(self, wd, *args):
        """each arg should be of the form (attr, val)."""
        self.wd = wd
        self.args = args
        for attr, val in args:
            setattr(self, attr, val)

    def __str__(self):
        return '\n'.join(['{}'.format(x) for x in self.args])

##################################################################

calc = Calculator('/tmp',
                  encut(400),
                  xc('PBE'),
                  generic('kpts', [1, 1, 1]))

print(calc)

print(help(encut))

encut validated to ('encut', 400)
xc validated to ('xc', 'PBE')
('encut', 400)
('xc', 'PBE')
('kpts', [1, 1, 1])
Help on function encut in module __main__:

encut(*args, **kwargs)
    Planewave cutoff in eV.

None

This approach obviously works. I don't think I like the syntax as much, although in most python editors, it should directly give access to the docstrings of the functions. This is pretty explicit in what is happening, which is an advantage. Compare this to the following approach, which uses our traditional kwarg syntax, with dynamic, but hidden validation.

def encut(cutoff):
    "Planewave cutoff in eV."
    assert isinstance(cutoff, int) and (cutoff > 0)
    return cutoff

def xc(s):
    """Exchange-correlation functional.

    Should be 'PBE' or 'LDA'.

    """
    assert isinstance(s, str), "xc should be a string"
    assert s in ['PBE', 'LDA'], "xc should be 'PBE' or 'LDA'"
    return s


class Calculator(object):
    def __init__(self, wd, **kwargs):
        """each arg should be of the form (attr, val)."""
        self.wd = wd

        for kwarg, val in kwargs.iteritems():
            f = globals().get(kwarg, None)
            if f is not None:
                print('{} evaluated to {}'.format(kwarg, f(val)))
            else:
                print('No validation for {}'.format(kwarg))

            setattr(self, kwarg, val)

##################################################################

calc = Calculator('/tmp',
                  encut=400,
                  xc='PBE',
                  kpts=[1, 1, 1])

print(calc.encut)
help(xc)

xc evaluated to PBE
No validation for kpts
encut evaluated to 400
400
Help on function xc in module __main__:

xc(s)
    Exchange-correlation functional.

    Should be 'PBE' or 'LDA'.

The benefit of this approach is no change in the syntax we are used to. We still get access to docstrings via tools like pydoc. It should not be too hard to get helpful tooltips in Emacs for this, using pydoc to access the docstrings. This might be the winner.

It is up for debate if we should use assert or Exceptions. If anyone uses python with -O the assert statements are ignored. That might not be desirable though. Maybe it would be better to use Exceptions, with a user customizable variable that determines if validation is performed.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

We run a lot of molecular simulations using Python. Here is a typical script we would use. It creates an instance of a calculator inside a context manager.

from ase import Atoms, Atom
from jasp import *

co = Atoms([Atom('C',[0,   0, 0]),
            Atom('O',[1.2, 0, 0])],
            cell=(6., 6., 6.))

with jasp('molecules/simple-co', #output dir
          xc='PBE',  # the exchange-correlation functional
          nbands=6,  # number of bands
          encut=350, # planewave cutoff
          ismear=1,  # Methfessel-Paxton smearing
          sigma=0.01,# very small smearing factor for a molecule
          atoms=co) as calc:
    print 'energy = {0} eV'.format(co.get_potential_energy())
    print co.get_forces()

This basic approach has served us for more than a decade! Still, there are things about it that bug me. Most significantly is the arbitrary keyword args. We keep a list of legitimate kwargs in the module, but there is no documentation or validation to go with them that is accessible to users (short of reading the code). There are well over 100 kwargs that are possible, so documenting them in the init docstring is not that useful (we did it once, see https://gitlab.com/ase/ase/blob/master/ase/calculators/jacapo/jacapo.py#L143 , and it made a really long docstring). Providing validation for these (some can only be integers, floats, specific strings, lists, or dictionaries) is not easy. I did this for another simulation code by providing validation functions that could be looked up dynamically by name. I never did come up with a way to provide kwarg specific documentation though.

The access to documentation while writing code is becoming increasingly important to me; I don't remember all the kwargs and what values are valid. More importantly, as I teach people how to use these tools, it is not practical to tell them to "read the code". I don't even want to do that while running simulations, I just want to setup the simulation and run it.

Today, I had an idea that a macro in hy would allow me to get documentation and validation of these kwargs.

The pseudocode would look like this. Each "kwarg" will actually be a function that has a docstring, performs validation, and evaluates to its argument. "vaspm" is a macro that will expand to the calculator with the desired kwargs. We will have to be careful that these function names don't conflict with other function names, but that could be addressed in a variety of ways with namespaces and function names.

;; pseudocode of the macro
(setv calc (vaspm "molecules/simple-co"
                  (xc "PBE")
                  (nbands 6)
                  (encut 350)
                  (ismear 1)
                  (sigma 0.01)
                  (atoms co)))

This would expand to the following block, which is equivalent to what we already do today. In the process of expansion though, we gain docstrings and validation!

(setv calc (Vasp "molecules/simple-co"
                 :xc "PBE"
                 :nbands 6
                 :encut 6
                 :ismear 1
                 :sigma 0.01
                 :atoms co))

Here is a toy implementation that illustrates what the functions are, and how we build up the code from the macro.

(defn encut [cutoff]
  "The planewave cutoff energy in eV."
  (assert (integer? cutoff))
  (assert (> cutoff 0))
  (print "encut validated")
  cutoff)

(defn xc [exc]
  "The exchange correlation functional. Should be a string of PBE or LDA."
  (assert (string? exc))
  (assert (in exc ["PBE" "LDA"]))
  (print "exc validated")
  exc)


(defclass Calculator []
  "Toy class representing a calculator."
  (defn __init__ [self wd &kwargs kwargs]
    (setattr self "wd" wd)
    (for [key kwargs]
      (setattr self key (get kwargs key)))))

We tangle that block to calculator.hy so we can reuse it. First we show the traditional syntax.

(import [calculator [*]])

(setv calc (Calculator "some-dir" :encut 400 :xc "PBE"))

(print calc.wd)
(print calc.encut)
(print calc.xc)

some-dir
400
PBE

Note, we can also do this, and get the validation too. It is verbose for my taste, but shows what we need the final code to look like, and incidentally how this would be done in Python too. We just need a macro that expands to this code.

(import [calculator [*]])

(setv calc (Calculator "some-dir" :encut (encut 400) :xc (xc "PBE")))

(print calc.wd)
(print calc.encut)
(print calc.xc)

encut validated
exc validated
some-dir
400
PBE

That is what this macro below does. We build up that code by making a keyword of the function name, and setting it to the value of the form the function is in.

(defmacro vaspm [wd &rest body]
  "Macro to build a Calculator with validation of arguments in BODY"
  (let [code `(Calculator ~wd)]
    (for [form body]
      (.append code (keyword (name (car form))))
      (.append code form))
    code))

Now, lets consider the macro version.

(import [calculator [*]])
(require calculator)

(setv calc (vaspm "some-dir" (encut 400) (xc "PBE")))
(print calc.wd)
(print calc.encut)
(print calc.xc)

;; proof we can get to the encut docstring!
(help encut)

encut validated
exc validated
some-dir
400
PBE
Help on function encut in module calculator:

encut(cutoff)
    The planewave cutoff energy in eV.

Sweet. The macro allows us to simplify our notation to be approximately the same as the original function, but with validation and docstring availability. Here is a variation of the macro that even uses keywords and builds the validation in from the keyword. It is not clear we can access the docstrings so easily here (ok, we can build an eldoc function that works either way, but the function method above is "more native").

(import [calculator [*]])
(require calculator)


(defmacro vasp2 [wd &rest kwargs]
  (let [code `(Calculator ~wd)]
    (for [x (range   0 (len kwargs) 2)]
      (let [kw (nth kwargs x)
            val (nth kwargs (+ 1 x))]
        (.append code kw)
        (.append code `(~(HySymbol (name kw)) ~val))))
    code))

(print (macroexpand '(vasp2 "/tmp" :encut 1 :xc "PBE")))

(setv calc (vasp2 "some-dir"
                  :encut 400
                  :xc "PBE"))
(print calc.wd)
(print calc.encut)
(print calc.xc)

(u'Calculator' u'/tmp' u'\ufdd0:encut' (u'encut' 1L) u'\ufdd0:xc' (u'xc' u'PBE'))
encut validated
exc validated
some-dir
400
PBE

To summarize here, we have looked at some ways to incorporate validation and documentation into kwargs. There are certainly ways to do this in Python, using these auxiliary functions. In fact we use them in hy too. We could build the validation into a Python init function too, using dynamic lookup of the function names, and evaluation of the functions. The macro features of hy give different opportunities for this, and different syntactical sugars to work with. The hy approach leads to less duplication (e.g. only a keyword, not a keyword and a function name that are the same), which will lead to fewer mistakes of the type xc=xd(something). Overall, interesting differences to contemplate.

From a developer point of view there is the burden of writing all the validation functions, but the payoff is access to documentation and optionally, validation. Also, no kwargs that are not allowed will work. Right now, with **kwargs, they might silently fail.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

A common pattern we have in running molecular simulations is to temporarily change to a new directory, do some stuff, and then change back to the directory, even if something goes wrong and an exception is raised. Here we examine several approaches to handling this in Python.

import os
print(os.getcwd())
try:
    cwd = os.getcwd()
    os.chdir('/tmp')
    f = open('some-file', 'w')
    f.write('5')
    print(os.getcwd())
    1 / 0
except:
    pass
finally:
    f.close()
    os.chdir(cwd)

print(os.getcwd())

print(open('/tmp/some-file').read())

/Users/jkitchin/Dropbox/python/hyve
/private/tmp
/Users/jkitchin/Dropbox/python/hyve
5

import contextlib

@contextlib.contextmanager
def cd(wd):
    import os
    cwd = os.getcwd()
    print('Started in {}'.format(os.getcwd()))
    os.chdir(wd)
    print('Entered {}'.format(os.getcwd()))
    try:
        yield
    except:
        pass
    finally:
        os.chdir(cwd)
        print('Entered {}'.format(os.getcwd()))

##################################################################
with cd('/tmp'):
    with open('some-other-file', 'w') as f:
        f.write('5')
    1 / 0

print(open('/tmp/some-other-file').read())

Started in /Users/jkitchin/Dropbox/python/hyve
Entered /private/tmp
Entered /Users/jkitchin/Dropbox/python/hyve
5

import os

def cd(wd):
    def outer(func):
        def inner(*args):
            cwd = os.getcwd()
            print('Started in {}'.format(os.getcwd()))
            os.chdir(wd)
            print('entered {}'.format(os.getcwd()))
            try:
                return func(*args)
            except:
                pass
            finally:
                os.chdir(cwd)
                print('entered {}'.format(os.getcwd()))
        return inner
    return outer
##################################################################

@cd('/tmp')
def f():
    with open('decorated-file', 'w') as f:
        f.write("5")
    1 / 0

f()
print(open("/tmp/decorated-file").read())

Started in /Users/jkitchin/Dropbox/python/hyve
entered /private/tmp
entered /Users/jkitchin/Dropbox/python/hyve
5

(defmacro cd [wd &rest body]
  `(do
    (import os)
    (print "started in " (os.getcwd))
    (let [cwd (os.getcwd)]
      (try
       (do (os.chdir ~wd)
           (print "entered " (os.getcwd))
           ~@body)
       (except [e Exception] nil)
       (finally
        (os.chdir cwd)
        (print "entered " (os.getcwd)))))))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(cd "/tmp"
    (with [f (open "some-hy-file" "w")]
          (.write f "5")
          (/ 1 0)))

(print (.read (open "/tmp/some-hy-file")))

started in  /Users/jkitchin/Dropbox/python/hyve
entered  /private/tmp
entered  /Users/jkitchin/Dropbox/python/hyve
5

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Apologies in advance if you haven't come across the programming language ook before now. It kind of looks like you recorded an orangutan as a program. It cracks me up to look at it. You will not be a better person for learning about it here, or for what is about to follow. This would have been a great April Fool's joke.

Here is a typical ook program. The canonical one for any language. I run it in a Python ook interpreter. You might notice this program actually converts ook to its isomorphic relative and executes that program.

from ook import *

interpreter = BFInterpreter()

program = """Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook!
Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook!
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook. Ook! Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook? Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook? Ook! Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook? Ook. Ook? Ook! Ook. Ook? Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook.
Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook. Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook? Ook!
Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook? Ook. Ook? Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook. Ook? Ook."""

interpreter.run_commands(convertOokToBF(program))

Hello World!

Yes, that odd looking program prints "Hello World!". Ook has just three syntax elements (Ook. Ook? Ook!), which when combined in pairs lead to 8 commands. You can represent any text in ook, as well as programs. It is supposedly Turing complete.

Think it could not get worse? Prepare for disappointment. There are at least 8 different unicode spaces … You see where this is going. Yes, this is just a trivial substitution exercise. Go ahead and queue Disturbed -- Down with the Sickness . Put it on repeat.

Introducing: space - the final frontier. A place where noone should ever go. Yep. A whole, Turing complete language using different kinds of unicode spaces as the commands. This idea is so bad that the only other language similar to it was an April Fool's joke more than a decade ago (https://en.wikipedia.org/wiki/Whitespace_(programming_language) )! They stuck with the ascii whitespace characters. Anyone else who ever had this idea probably had the decency to keep it to themselves. Here is probably the one and only (and hopefully last) space program in existence.

(import [space [*]])

(space "                                                                                                                                                                                                             ")

Hello World!

That's right, a program of nothing but different kinds of spaces that actually does something. Sure space can do anything any other Turing complete language can do, with an infinite sized stack, and a Sysyphean patience and endurance, and a healthy dose of self-loathing. But why would you want to?

That program is hiding out in the open there. It is a special kind of noise. Like a modern ascii art interpreted in unicode spaces. Maybe these paintings really have hidden messages in them. Forget steganography to hide your messages. Just encode them in ook, convert them to space and put them out there. Who would even know they were there if they did not know to look. Even if they did look, what do they see? space. I am probably getting on an NSA list for that suggestion. Space is so useful. You could use a unicode space version of Morse code. Three short spaces regular space three long spaces regular space three short spaces, i.e. "           ". Those quotes are just to hint they are there. A cry for help. If a space is on the page and nobody sees it, is it really there? A space by any other name would look like … a space?

This hackery is actually just a trivial substitution on ook that translates each different space to the ook command it corresponds to and then runs it. I built it off of this Python ook intepreter , which translates the ook command to another program to run it. I implemented space in hy below. As bad as this idea is, I did not want to build a native space interpreter, just show the absurdity of it. Oh, and write a bit of hy code with dictionaries and comprehension. Sorry to drag that interesting project to this dark corner. Hy sure looks reasonable compared to this! Here is the thin layer.

(setv ook_lookup {(, "Ook." "Ook?") "\u2000"
                  (, "Ook?" "Ook.") "\u2001"
                  (, "Ook." "Ook.") "\u2002"
                  (, "Ook!" "Ook!") "\u2003"
                  (, "Ook!" "Ook.") "\u2004"
                  (, "Ook." "Ook!") "\u2005"
                  (, "Ook!" "Ook?") "\u2006"
                  (, "Ook?" "Ook!") "\u2007"}
      space_lookup (dict-comp v k [[k v] (.iteritems ook_lookup)]))

(defn ook2space [program]
  "Convert an ook PROGRAM to space syntax."
  (let [chars (.split (.replace program "\n" " ") " ")
        p1 (cut chars 0 nil 2)
        p2 (cut chars 1 nil 2)
        pairs (zip p1 p2)
        sp ""]
    (for [ch pairs] (+= sp (get ook_lookup ch)))
    sp))


(defn space2ook [program]
  "Convert a space PROGRAM to ook syntax."
  (.join " " (list-comp (.join " " (get space_lookup char)) [char program])))


(import [ook [*]])
(defn space [space_program]
  "Run a space program."
  (let [interpreter (BFInterpreter)
        bf_program (convertOokToBF (space2ook space_program))]
    (.run_commands interpreter bf_program)))

We can convert an ook program to a space program. I put the brackets in just for a visual boundary of the space program.

(import [space [*]])

(let [program "Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook!
Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook!
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook. Ook! Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook? Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook? Ook! Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook? Ook. Ook? Ook! Ook. Ook? Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook.
Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook? Ook! Ook! Ook. Ook? Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook? Ook. Ook? Ook! Ook. Ook? Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook! Ook. Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook. Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook. Ook? Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook! Ook? Ook!
Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook! Ook! Ook? Ook. Ook? Ook! Ook. Ook? Ook! Ook! Ook! Ook! Ook!
Ook! Ook! Ook. Ook? Ook."]
 (print (.encode (+ "[" (ook2space program) "]") "utf-8")))

[                                                                                                                                                                                                             ]

Nothing but space here folks. Move along.

What did we learn today? Evidently I have strange ideas sometimes, and I am not alone in that. Just because you can do something, certainly doesn't mean you should. Sorry for this use of your time if you got here. Please come back again!

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10