The Kitchin Research Group

Calling remote code-blocks in org-mode

Tue, 09 Feb 2016 20:40:59 EST

1. References

We often write code in supporting information files that generates figures or tables in a scientific manuscript. Today, we explore how to call those code blocks remotely but get the output in the file we call it from. We will write code in si.org that generates an interactive figure that is presented in this file. We will use data published in hallenbeck-2015-compar-co2. You can find the data we used in the SI for that paper, or more conveniently here .

So, we make a named code block in the si.org file called "figure-1". Then we call it like this:

#+call: si.org:figure-1() :wrap html

That executes the code block in the other file, and wraps the output in an HTML block in this file! I do not like my code blocks to execute when I export because they are usually expensive calculations, so I have to manually run the line with C-c C-c, but you can override that behavior with a local setting of org-export-babel-evaluate. So, without further delay, here is the result. Now we have a nice, neat blog post file, with code in an si.org file!

1 References

Bibliography

[hallenbeck-2015-compar-co2] "Alexander Hallenbeck, Adefemi Egbebi, Kevin Resnik, , David Hopkinson, Shelley Anna & John Kitchin", Comparative Microfluidic Screening of Amino Acid Salt Solutions for Post-Combustion \ceCO2 Capture, "International Journal of Greenhouse Gas Control ", 43, 189 - 197 (2015). link. doi.

org-mode source

Org-mode version = 8.2.10

Interactive figures in blog posts with mpld3

Mon, 08 Feb 2016 07:33:23 EST

Continuing the exploration of interactive figures, today we consider the Python plotting library mpld3 . We will again use our own published data. We wrote this great paper on core level shifts (CLS) in Cu-Pd alloys boes-2015-core-cu. I want an interactive figure that shows the name of the calculation on each point as a tooltip. This data is all stored in the supporting information file, and you can see how we use it here. This figure shows how the core level shift of a Cu atom changes depending on the number of nearest neighbor Cu atoms. Just hover your mouse over a point to see the name and CLS for that point.

1 Data and code

You can check out our preprint at https://github.com/KitchinHUB/kitchingroup-51 . We are going to adapt the code to make Figure 6a in the manuscript interactive. The code needed a somewhat surprising amount of adaptation. Apparently the ase database interface has changed a lot since we write that paper, so the code here looks a bit different than what we published. The biggest difference is due to name-mangling so each key that started with a number now starts with _, and and periods are replaced by _ also. The rest of the script is nearly unchanged. At the end is the very small bit of mpld3 code that generates the figure for html. We will add tooltips onto datapoints to indicate what the name associated with each data point is. Here is the code.

import matplotlib.pyplot as plt
from ase.db import connect

# loads the ASE database and select certain keywords
db = connect('~/Desktop/cappa/kitchingroup-51/supporting-information/data.json')

keys = ['bcc', 'GS', '_54atom', 'ensam']

CLS, IMP, labels = [], [], []
for k in db.select(keys + ['_1cl']):
    name = k.keywords[-2]

    Cu0 = db.select('bcc,GS,_72atom,_0cl,_1_00Cu').next().energy
    Cu1 = db.select('bcc,GS,_72atom,_1cl,_1_00Cu').next().energy
    x0 = db.select(','.join(keys + [name, '_0cl'])).next().energy
    x1 = k.energy

    cls0 = x0 - Cu0
    cls1 = x1 - Cu1

    IMP.append(int(name[1]))
    CLS.append(cls1 - cls0)
    labels += ['{0} ({1}, {2})'.format(name, int(name[1]), cls1 - cls0)]

Cu0 = db.select(','.join(['bcc', 'GS', '_72atom',
                          '_0cl', '_1_00Cu'])).next().energy
Cu1 = db.select(','.join(['bcc', 'GS', '_72atom',
                          '_1cl', '_1_00Cu'])).next().energy

x0 = db.select(','.join(['bcc', 'GS', '_54atom',
                         '_0cl', '_1'])).next().energy
x1 = db.select(','.join(['bcc', 'GS', '_54atom',
                         '_1cl', '_1'])).next().energy

cls0 = x0 - Cu0
cls1 = x1 - Cu1

IMP.append(1)
CLS.append(cls1 - cls0)
labels += ['(1, {0})'.format(cls1 - cls0)]

Cu0 = db.select(','.join(['bcc', 'GS', '_72atom',
                          '_0cl', '_1_00Cu'])).next().energy
Cu1 = db.select(','.join(['bcc', 'GS', '_72atom',
                          '_1cl', '_1_00Cu'])).next().energy

x0 = db.select(','.join(['bcc', 'GS', '_54atom',
                         '_0cl', '_0'])).next().energy
x1 = db.select(','.join(['bcc', 'GS', '_54atom',
                         '_1cl', '_0'])).next().energy

cls0 = x0 - Cu0
cls1 = x1 - Cu1

IMP.append(0)
CLS.append(cls1 - cls0)
labels += ['(0, {0})'.format(cls1 - cls0)]

fig = plt.figure()

p = plt.scatter(IMP, CLS, c='g', marker='o', s=25)
ax1 = plt.gca()
ax1.set_ylim(-1.15, -0.6)
ax1.set_xlim(-0.1, 5.1)

ax1.set_xlabel('# Cu Nearest neighbors')
ax1.set_ylabel('Cu 2p(3/2) Core Level Shift (eV)')

ax1.set_title('Hover over a point to see the calculation name')

# Now the mpld3 stuff.
import mpld3
from mpld3 import plugins

tooltip = plugins.PointHTMLTooltip(p, labels, voffset=0, hoffset=10)
plugins.connect(fig, tooltip)

print mpld3.fig_to_html(fig)

I like this workflow pretty well. It seems less functional than plotly and Bokeh (e.g. it does not look like it you can export the data from the html here), but it is well integrated with Matplotlib, with my blogging style, and does not require a server, oran account. The code outputs html that is self-contained in the body of the html. The smooth integration with Matplotlib means I could have static images in org-mode, and dynamic images in HTML potentially. Overall, this is a nice tool for making interactive plots in blog posts.

2 References

Bibliography

[boes-2015-core-cu] Jacob Boes, Peter Kondratyuk, Chunrong Yin, James, Miller, Andrew Gellman & John Kitchin, Core Level Shifts in Cu-Pd Alloys As a Function of Bulk Composition and Structure, Surface Science, 640, 127-132 (2015). link. doi.

org-mode source

Org-mode version = 8.2.10

Interactive Bokeh plots in HTML

Sun, 07 Feb 2016 10:53:45 EST

1. The data and code
2. References

In our last post we examined the use of plotly to generate interactive plots in HTML. Today we expand the idea, and use Bokeh . One potential issue with plotly is the need for an account and API-key, some limitations on how many times a graph can be viewed per day (although I should aspire to have my graphs viewed 1000+ times a day!), and who knows what happens to the graphs if plotly ever goes out of business. While the static images we usually use have limited utility, at least they stick around.

So, today we look at Bokeh which allows you to embed some json data in your HTML, which is made interactive by your browser with more javascript magic. We get straight to the image here so you can see what this is all about. Briefly, this data shows trends (or lack of) in the adsorption energies of some atoms on the atop and fcc sites of several transition metals as a function of adsorbate coverage xu-2014-probin-cover. The code to do this is found here.

Using Bokeh does not integrate real smoothly with my blog workflow, which only generates the body of HTML posts. Bokeh needs some javascript injected into the header to work. To get around that, I show the plot in a frame here. You can see a full HTML version here: bokeh-plot.html .

This is somewhat similar to the plotly concept. The data is embedded in the html in this case, which is different. For very large plots, I actually had some trouble exporting the blog post (it was taking a long time to export and I killed it)! I suspect that is a limitation of the org-mode exporter though, because I could save the html files from Python and view them fine. I also noted that having all the javascript in the org-file make font-lock work very slow. It would be better to generate that only on export.

Note to make this work, we need these lines in our HTML header:

#+HTML_HEAD: <link rel="stylesheet" href="http://cdn.pydata.org/bokeh/release/bokeh-0.11.1.min.css" type="text/css" />
#+HTML_HEAD: <script type="text/javascript" src="http://cdn.pydata.org/bokeh/release/bokeh-0.11.1.min.js"></script>

Since we do not host those locally, if they ever disappear, our plots will not show ;(

1 The data and code

We will get the data from our paper on coverage dependent adsorption energies xu-2014-probin-cover. There are some data rich figures there that would benefit from some interactivity. You can get the data here: http://pubs.acs.org/doi/suppl/10.1021/jp508805h . Extract out the supporting-information.org and energies.json file to follow here. We will make Figure 2a in the SI document here, and make it interactive with hover tooltips.

import json

from collections import OrderedDict
from bokeh import mpl
from bokeh.plotting import *
from bokeh.models import HoverTool
from bokeh.embed import components

with open('/users/jkitchin/Desktop/energies.json', 'r') as f:
    data = json.load(f)


# color for metal
# letter symbol for adsorbate
colors = {'Cu':'Orange',
          'Ag':'Silver',
          'Au':'Yellow',
          'Pd':'Green',
          'Pt':'Red',
          'Rh':'Blue',
          'Ir':'Purple'}

all_ads = ['O', 'S']

TOOLS="crosshair,pan,wheel_zoom,box_zoom,reset,hover,previewsave"
p = figure(title="Correlation between atop and fcc sites", tools=TOOLS)

for metal in ['Rh', 'Pd', 'Cu', 'Ag']:
    for adsorbate in all_ads:
        E1, E2 = [], []
        for coverage in '0.25', '0.5', '0.75', '1.0':
            if (isinstance(data[metal][adsorbate]['ontop'][coverage], float) and
                isinstance(data[metal][adsorbate]['fcc'][coverage], float)):
                E1.append(data[metal][adsorbate]['ontop'][coverage])
                E2.append(data[metal][adsorbate]['fcc'][coverage])
        labels = ['{0}-{1} {2} ML'.format(metal, adsorbate, x)
                  for x in ['0.25', '0.5', '0.75', '1.0']]
        p.line('x', 'y', color=colors[metal],
               source=ColumnDataSource(data={'x': E1,
                                             'y': E2,
                                             'label': labels}))
        p.circle('x', 'y', color=colors[metal],
               source=ColumnDataSource(data={'x': E1,
                                             'y': E2,
                                             'label': labels}))


hover =p.select({'type': HoverTool})
hover.tooltips = OrderedDict([("(atop,fcc)", "(@x, @y)"),
                              ("label", "@label")])

p.xaxis.axis_label = 'Adsorption energy on the atop site'
p.yaxis.axis_label = 'Adsorption energy on the fcc site'

script, div = components(p)
script = '\n'.join(['#+HTML_HEAD_EXTRA: ' + line for line in script.split('\n')])

print '''{script}

#+BEGIN_HTML
<a name="figure"></a>
{div}
#+END_HTML
'''.format(script=script, div=div)

2 References

Bibliography

[xu-2014-probin-cover] Zhongnan Xu & John Kitchin, Probing the Coverage Dependence of Site and Adsorbate Configurational Correlations on (111) Surfaces of Late Transition Metals, J. Phys. Chem. C, 118(44), 25597-25602 (2014). link. doi.

org-mode source

Org-mode version = 8.2.10

Interactive plots in HTML with Plotly

Sat, 06 Feb 2016 12:44:53 EST

Most of the plots in this blog are static. Today, I look at making them interactive. I will use https://plot.ly for this. I want to use some data from a paper we published on the relative stabilities of oxide polymorphs mehta-2015-ident-poten. We will make an interactive figure showing the relative stabilities of the RuO₂ polymorphs. When you hover on a point, it will show you which polymorph the point refers to. Let's see the figure first here. If you think its interesting read on to see how we made it!

We get our data source here: http://pubs.acs.org/doi/suppl/10.1021/am4059149/suppl_file/am4059149_si_001.pdf .

Now, we extract the data files:

pdftk ~/Desktop/am4059149_si_001.pdf  unpack_files

That extracts a json file called supporting-information.json. We use it as suggested in the SI pdf to plot the equations of state for RuO₂ for several polymorphs.

# coding=utf-8

import plotly.plotly as py
import plotly.graph_objs as go
import plotly.tools as tls
import numpy as np

import json
import matplotlib.pyplot as plt
from ase.utils.eos import EquationOfState
with open('supporting-information.json', 'rb') as f:
    d = json.loads(f.read())

BO2 = 'RuO2'
xc = 'PBE'

layout = go.Layout(title='Energy vs. Volume for RuO<sub>2</sub> polymorphs',
                   xaxis=dict(title='Volume (Å<sup>3</sup>)'),
                   yaxis=dict(title='Energy (eV)'))

traces = []

for polymorph in ['rutile','anatase','brookite','columbite','pyrite','fluorite']:

    # number of atoms in the unit cell - used to normalize
    natoms= len(d[BO2][polymorph][xc]['EOS']['calculations']
                [0]['atoms']['symbols'])
    volumes = [entry['data']['volume']*3./natoms for entry in
               d[BO2][polymorph][xc]['EOS']['calculations']]
    energies =  [entry['data']['total_energy']*3./natoms for entry in
                 d[BO2][polymorph][xc]['EOS']['calculations']]

    trace = go.Scatter(x=np.array(volumes),
                       y=np.array(energies),
                       mode='lines+markers',
                       name=polymorph,
                       text=polymorph)

    traces += [trace]

fig = go.Figure(data=traces, layout=layout)
plot_url = py.plot(fig, filename='ruo2-2')

print tls.get_embed(plot_url)

Pretty nice, now we should have an interactive plot in our browser with the data points labeled with tags, zooming, etc… That is nice for the blog. It isn't so nice for daily work, as there is no visual version of the plot in my org-file. Of course, I can visit the url to see the plot in my browser, it is just different from what I am used to. For everyone else, this is probably better. It looks like you can actually get the data from the web page, including some minimal analysis like regression, and save your view to an image! That could be pretty nice for some data sets.

1 Using Plotly yourself

First, go to https://plot.ly and sign up for an account. You will want to register your API key like this, which will save it in a file for your convenience. Then you can do things like I did above too.

import plotly.tools as tls
tls.set_credentials_file(username='jkitchin', api_key='xxxxxxx')

2 References

Bibliography

[mehta-2015-ident-poten] Prateek Mehta, Paul Salvador & John Kitchin, Identifying Potential \ceBO2 Oxide Polymorphs for Epitaxial Growth Candidates, ACS Appl. Mater. Interfaces, 6(5), 3630-3639 (2015). link. doi.

org-mode source

Org-mode version = 8.2.10

The Kitchin Research Group

Calling remote code-blocks in org-mode

Table of Contents

1 References

Bibliography

Interactive figures in blog posts with mpld3

1 Data and code

2 References

Bibliography

Interactive Bokeh plots in HTML

Table of Contents

1 The data and code

2 References

Bibliography

Interactive plots in HTML with Plotly

1 Using Plotly yourself

2 References

Bibliography