The Kitchin Research Group

In a previous post I wrote about a sandbox module to help capture stderr in Python code blocks in org-mode. That module worked, but ran as a script.

stderr is not captured in the output of a code block in org-mode. For example:

import sys
print >>sys.stdout, "message on stdout"
print >>sys.stderr, "testing stderr"

message on stdout

The messages to stderr just disappears. Not good for code like this:

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol

[[ 1.]
 [ 1.]]

There is an error in that code, k is not defined. If you run that as a script, you get this output:

>>> Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "/var/folders/5q/lllv2yf95hg_n6h6kjttbmdw0000gn/T//python-69413hLF.py", line 4, in ode
    return -k * x
NameError: global name 'k' is not defined

But, that is evidently going to stderr, and not getting captured in org-mode. Boo. A silent error that returns a value! This behavior of odeint may be fixed in scipy 0.15, but it is a general deficiency of org-mode babel code blocks. So, today I am looking back into a way to fix it. We try something as mundane as just redefining stderr in Python at runtime.

import sys
sys.stderr = sys.stdout

print >>sys.stdout, "message on stdout"
print >>sys.stderr, "testing stderr"

message on stdout
testing stderr

That works fine. Let us test it with the other block.

import sys
sys.stderr = sys.stdout

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol

Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 6, in ode
NameError: global name 'k' is not defined
[[ 1.]
 [ 1.]]

Sweet, we get the errors. We still get the returned value, but it is immediately obvious something is wrong. I have wrapped that little tidbit into a Python module in pycse.orgmode , which you can import to get the same effect.

import pycse.orgmode

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol

Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined

[[ 1.]
 [ 1.]]

Finally, you can avoid the import by setting your org-babel Python command like this:

(setq org-babel-python-command "python -i -c \"import pycse.orgmode\"")

python -i -c "import pycse.orgmode"

Now, we run our faulty block again:

from scipy.integrate import odeint

def ode(y, x):
    return -k * x

xspan = [0, 1]
y0 = 1

sol = odeint(ode, y0, xspan)
print sol

Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined
Traceback (most recent call last):
  File "<stdin>", line 2, in ode
NameError: global name 'k' is not defined

[[ 1.]
 [ 1.]]

Excellent. The stderr is captured.

And we get basically the same output as before for regular code blocks. There is an extra line before and after the output for some reason. I can live with that!

print 6 + 7

13

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10