Creating a transportable zip-archive of an org-file

Posted March 05, 2014 at 10:49 AM | categories: org-mode | tags:

Updated March 05, 2014 at 12:31 PM

This post explores a method to save an org-buffer to a zip archive, with all the referencing files. The challenge is that you may want to share the org-file with someone, but the links break if you send them the file, and it is not that trivial to find all the links and change them, and to copy the files to a place where the new links work.

The idea is to export the buffer to an org-file and process all the links to copy the files to a new directory, and change the links to point to these new files. For example, blog.pdf would be copied to the temporary directory, given a unique name, and then relinked. The text below includes some examples of the links that need to be modified.

A figure looks like:

Alternatively, we might use a to a file. We do not want to change urls, such as this one: http://kitchingroup.cheme.cmu.edu/blog/2013/09/28/Changing-links-to-files-so-they-work-in-a-blog/ . As in that example, we will create a list of all the links in the buffer, but only modify the links that are files. We can parse the buffer and get the links like this. 

(let ((parsetree (org-element-parse-buffer))
      (counter 0))
  (org-element-map parsetree 'link
    (lambda (link) 
      (let ((type (nth 0 link))
            (plist (nth 1 link))
            (content (nth 2 link)))
        (princ (format "%s %s: %s %s\n" 
                       counter 
                       (plist-get plist ':type) 
                       (plist-get plist :path) 
                       content))
        (setq counter (+ counter 1))))))

0 file: ./blog.pdf nil
1 file: ./images/cos-plot.png nil
2 file: ./images/eos.png link
3 http: //kitchingroup.cheme.cmu.edu/blog/2013/09/28/Changing-links-to-files-so-they-work-in-a-blog/ nil

So, our overall strategy will be to create a new directory to store the new versions of the files in. Then, we will copy the files that links point to into that directory, and give them new unique names. We will rename the links to point to these new names. We do this because you may, for some reason have links to files with the same name but in different directories. We want to make sure we do not clobber the files by overwriting them. We use a simple method here, based on unique, temporary filenames. There are other ways to do it to, this way worked first. Finally, we will zip that new directory, and delete the new directory. 

;; directory to save all exports in, using the current date
(setq org-archive (concat "org-archive-" (format-time-string "%Y-%m-%d" (current-time))))

;; delete directory and zip file if it exists
(when (file-exists-p (concat org-archive ".zip")) 
    (delete-file (concat org-archive ".zip") t))

(when (file-exists-p org-archive) 
    (delete-directory org-archive t))

;; make directory
(make-directory org-archive t)

;; get list of links, copy files and save names
(setq link-list (let ((parsetree (org-element-parse-buffer))
                     (counter 0))
                 (org-element-map parsetree 'link
                   (lambda (link) 
                     (let* ((type (nth 0 link))
                            (plist (nth 1 link))
                            (content (nth 2 link))
                            (path (plist-get plist :path))
                            (type (plist-get plist ':type))
                            (fname (car (last (split-string path "/"))))
                            (temporary-file-directory org-archive)
                            (new-file)
                            )     
                       (cond
                        ;; regular file with content
                        ((and (string= type "file")  content)
                         (setq new-file  (make-temp-file (file-name-sans-extension fname) nil 
                                                         (concat "." (file-name-extension fname))))
                         (with-temp-file new-file
                           (insert-file-contents path))
                         (format "[[./%s][%s]] " (file-name-nondirectory new-file) content))
                        ;; regular file with no content
                        ((and (string= type "file"))
                         (setq new-file  (make-temp-file (file-name-sans-extension fname) nil 
                                                         (concat "." (file-name-extension fname))))
                         (with-temp-file new-file
                           (insert-file-contents path))
                         (format "[[./%s]] " (file-name-nondirectory new-file)))
                        (t nil)))))))

;; save current buffer name
(setq current-name (buffer-name))

;; create filter for links and export org buffer
(let ((counter 0))
  (defun ox-mrkup-filter-link (text back-end info)
    (let ((link (nth counter link-list)))
      (if (not (string= link "nil")) (setq output   (format "%s" link))
        (setq output (format "%s" text)))
      (setq counter (+ counter 1))
      output))

  (let ((org-export-filter-link-functions '(ox-mrkup-filter-link)))
    (org-org-export-as-org)))

(switch-to-buffer "*Org ORG Export*")
(write-file (expand-file-name current-name org-archive))
(shell-command (concat "zip -R " org-archive ".zip  *"))
(rename-file (concat org-archive ".zip") (concat "../"org-archive ".zip"))
(kill-buffer)

(switch-to-buffer current-name)
(delete-directory org-archive t)  ;; get rid of temp-dir

This example works fine! The result is here: org-archive-2014-03-05.zip This code would ideally be put into a function, and cleaned up a little so there are not global variables being set here and there. A subsequent function might make it easy to attach this file to an email. That code might look something like this: 

(mail)
(mail-to)
(insert "jkitchin@andrew.cmu.edu")
(mml-attach-file "./org-archive-2014-03-05.zip")

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.5h

Discuss on Twitter

Getting a list of figures in an org-buffer

Posted March 02, 2014 at 10:03 AM | categories: org-mode | tags:

Updated March 03, 2014 at 02:50 PM

Similar to the previous example of getting a list of tables, here we examine getting a list of figures. Here are two figure links, one with a label, and one with a caption.

Figure 1: An equation of state. this is the caption of the figure.

Figure 2: another figure

We define a link that will parse the buffer, and create links in a new buffer to the figures. We define a figure as a link with a :type of "file" that has a path that points to a file ending with png or pdf. We will improve on the list of tables by making the buffer read-only, and making a local key binding to kill the buffer by pressing "q". Here is our attempted code. 

;; http://www.emacswiki.org/emacs/ElispCookbook#toc4
(defun string/ends-with (s ending)
  "return non-nil if string S ends with ENDING."
  (cond ((>= (length s) (length ending))
         (let ((elength (length ending)))
           (string= (substring s (- 0 elength)) ending)))
        (t nil)))

(org-add-link-type 
 "list-of-figures"
 (lambda (link-string)
   (let* ((c-b (buffer-name))
          (counter 0)
          (list-of-figures 
           (org-element-map (org-element-parse-buffer) 'link
             (lambda (link) 
               "create a link for to the figure"
               (when 
                   (and (string= (org-element-property :type link) "file")
                        (string-match-p  
                         "[^.]*\\.\\(png\\|jpg\\)$"
                         (org-element-property :path link)))                   
                 (incf counter)
                 
                 (let* ((start (org-element-property :begin link))
                        (parent (car (cdr (org-element-property :parent link))))
                        (caption (caaar (plist-get parent :caption)))
                        (name (plist-get parent :name)))
                   (if caption 
                       (format 
                        "[[elisp:(progn (switch-to-buffer \"%s\")(goto-char %s))][figure %s: %s]] %s\n" 
                        c-b start counter (or name "") caption)
                     (format 
                      "[[elisp:(progn (switch-to-buffer \"%s\")(goto-char %s))][figure %s: %s]]\n" 
                      c-b start counter (or name "")))))))))
          (switch-to-buffer "*List of Figures*")
          (org-mode)
          (erase-buffer)
          (insert (mapconcat 'identity list-of-figures ""))
          (setq buffer-read-only t)
          (use-local-map (copy-keymap org-mode-map))
          (local-set-key "q" #'(lambda () (interactive) (kill-buffer)))))
   (lambda (keyword desc format)
     (cond
      ((eq format 'latex)
       (format "\\listoffigures")))))

This is a test to see if our function works for other image types. smiley.jpg

And a link to test it out:

This works too. I am not sure I am getting the figure name and caption in a bulletproof way. They seem to be buried in the :parent of the element, which is a paragraph element. The caption seems to be buried in a few sets of parentheses, hence the use of caaar to get the caption out. I am not sure if the caption is always at that depth or not. As a proof of concept though, this is not too bad.

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.5h

Discuss on Twitter

Getting a list of tables in an org-buffer

Posted March 01, 2014 at 06:12 PM | categories: org-mode | tags:

Updated March 02, 2014 at 08:02 AM

In a large document it might be nice to quickly get a list of tables. Preferrably by clicking on a link that generates the list, and exports appropriately, e.g. listoftables for LaTeX. A link like this:

Before getting to the code that does what we need, let us make some tables. We make three different kinds of tables for fun. A named table, an unnamed table, and a table with a caption.

1 2
a b
t y
5 6
Table 1: column of numbers
34
6
6

We would like a function that creates a buffer with a list of the tables, and links to them. We include the table name, and caption if there is one. We will create an org-buffer, and use org-links to the tables. Here is a link definition that will do that. 

(org-add-link-type 
 "list-of-tables"
 (lambda (link-string)
   (let* ((c-b (buffer-name))
          (counter 0)
          (list-of-tables 
           (org-element-map (org-element-parse-buffer 'element) 'table
             (lambda (table) 
               "create a link for to the table"
               (incf counter)
               (let ((start (org-element-property :begin table))
                     (name  (org-element-property :name table))
                     (caption (caaar (org-element-property :caption table))))
                 (if caption 
                     (format 
                      "[[elisp:(progn (switch-to-buffer \"%s\")(goto-char %s))][table %s: %s]] %s\n" 
                      c-b start counter (or name "") caption)
                   (format 
                    "[[elisp:(progn (switch-to-buffer \"%s\")(goto-char %s))][table %s: %s]]\n" 
                    c-b start counter (or name ""))))))))
     (switch-to-buffer "*List of Tables*")
     (org-mode)
     (erase-buffer)
     (insert (mapconcat 'identity list-of-tables ""))))
 (lambda (keyword desc format)
   (cond
    ((eq format 'latex)
     (format "\\listoftables")))))

A list of figures would only be a little trickier. You would map over the links, and find the file type links that have a select number of extensions, e.g. png, jpg, etc…

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.5h

Discuss on Twitter

Extracting bibtex file from an org-buffer

Posted February 19, 2014 at 06:45 PM | categories: bibtex, org-mode | tags:

Updated February 19, 2014 at 07:27 PM

Table of Contents

We use citation links a lot in our org-files, like this: 

cite:thompson-2014-co2-react
. Sometimes there are multiple citations like this 
cite:mehta-2014-ident-poten,hallenbeck-2013-effec-o2
. It would be convenient at times to extract a bibtex file from these citations. That way we could easily share files. This is possible in RefTeX from a LaTeX file. Org makes it easy to export to LaTeX, so this seems like it should be easy. It would be easy, if I always put the bibliography link in the file. I usually do not, so let us check if that is the case, and if it is not add the bibliography to the end before we export. Then, with the LaTeX file in hand, we open it, and call the RefTeX functions to get the bibliography. Finally, we will create a link to the actual created file, and add it as a source block that can be tangled at the end of the file.

Here is a function that does the extraction and some house cleaning. We actually take the contents of the buffer and save it in a temporary file, so that we do not accidentally clobber a tex or bibtex file here. 

(defun kg-extract-bibtex ()
  "create bibtex file of entries cited in this buffer"

  (let* ((tempname (make-temp-file "extract-bib"))
         (contents (buffer-string))
         (cb (current-buffer))
         basename texfile bibfile results)

    (find-file tempname)
    (insert contents)
    (setq basename (file-name-sans-extension
                    (file-name-nondirectory buffer-file-name))
          texfile (concat basename ".tex")
          bibfile (concat basename ".bib"))

  (save-excursion
    (goto-char (point-min))
    (unless (re-search-forward "^bibliography:" (point-max) 'end)
      (insert (format "\nbibliography:%s" (mapconcat 'identity reftex-default-bibliography ",")))))

    (org-latex-export-to-latex)
    (find-file texfile)
    (reftex-parse-all)
    (reftex-create-bibtex-file bibfile)
    (setq results (buffer-string))
    (kill-buffer bibfile)
    (kill-buffer texfile)
    (delete-file texfile)
    (delete-file tempname)

    (switch-to-buffer cb)
    (save-excursion
      (goto-char (point-max))
      (insert (format "

** Bibtex entries

#+BEGIN_EXAMPLE: 
%s
#+END_EXAMPLE" results)))))

(kg-extract-bibtex)

There it is! The src block does not render in HTML very well, since it appears to be simple text. It looks fine in the org file though.

It might be a good idea to replace the bibliography line with the new file, but I will leave that as an exercise for later.

1 Bibtex entries

#+BEGINEXAMPLE: @article{hallenbeck-2013-effec-o2, author = "Hallenbeck, Alexander P. and Kitchin, John R.", title = "Effects of \ce{O_2} and \ce{SO_2} on the Capture Capacity of a Primary-Amine Based Polymeric \ce{CO_2} Sorbent", year = 2013, doi = "10.1021/ie400582a", eprint = "http://pubs.acs.org/doi/pdf/10.1021/ie400582a ", journal = "Industrial \& Engineering Chemistry Research", pages = "10788-10794", url = "http://pubs.acs.org/doi/abs/10.1021/ie400582a ", }

@article{mehta-2014-ident-poten, author = {Mehta, Prateek and Salvador, Paul A. and Kitchin, John R.}, title = {Identifying Potential BO2 Oxide Polymorphs for Epitaxial Growth Candidates}, journal = {ACS Applied Materials \& Interfaces}, volume = 0, number = 0, pages = {null}, year = 2014, doi = {10.1021/am4059149}, URL = {http://pubs.acs.org/doi/abs/10.1021/am4059149 }, eprint = {http://pubs.acs.org/doi/pdf/10.1021/am4059149 } }

@Article{thompson-2014-co2-react, author = {Thompson, Robert L. and Albenze, Erik and Shi, Wei and Hopkinson, David and Damodaran, Krishnan and Lee, Anita and Kitchin, John and Luebke, David Richard and Nulwala, Hunaid}, title = {\ce{CO_2} Reactive Ionic Liquids: Effects of functional groups on the anion and its influence on the physical properties}, journal = {RSC Adv.}, year = 2014, pages = "-", publisher = {The Royal Society of Chemistry}, doi = {10.1039/C3RA47097K}, url = {https://doi.org/10.1039/C3RA47097K }, abstract = "Next generation of gas separation materials are needed to alleviate issues faced in energy and environmental area. Ionic liquids (ILs) are promising class of material for CO2 separations. In this work{,} CO2 reactive triazolides ILs were synthesized and characterized with the aim of developing deeper understanding on how structural changes affect the overall properties for CO2 separation. Important insights were gained illustrating the effects of substituents on the anion. It was found that substituents play a crucial role in dictating the overall physical properties of reactive ionic liquids. Depending upon the electronic and steric nature of the substituent{,} CO2 capacities between 0.07-0.4 mol CO2/mol IL were observed. Detailed spectroscopic{,} CO2 absorption{,} rheological{,} and simulation studies were carried out to understand the nature and influence of these substituents. The effect of water content was also evaluated{,} and it was found that water had an unexpected impact on the properties of these materials{,} resulting in an increased viscosity{,} but little change in the CO2 reactivity." } #+ENDEXAMPLE

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.5h

Discuss on Twitter

A dynamic snippet for a task due 7 days from now

Posted February 16, 2014 at 01:50 PM | categories: org-mode, emacs | tags:

Updated February 16, 2014 at 03:52 PM

I have been playing with yasnippets. A pretty cool feature is that you can run elisp code in the template to generate text. Below, I define a snippet that will create a todo item due 7 days from the time you define it. This is an unconventional way to define a snippet, but I did not want to save it to a file just to try it out. So, I put it in a temporary buffer, and load it from there. When you run this block, it will note it is a new snippet, and ask if you want to save it. You can say no.

We will use the code we developed here to create a timestamp from the current time plus seven days. 

(yas-global-mode)
(with-temp-buffer
  (insert "# name : todo-followup
# --

*************** TODO $1
${2:             DEADLINE: `(let ((seven-days (seconds-to-time (* 7 24 60 60))))
  (format-time-string \"<%Y-%m-%d %a>\" (time-add (current-time) seven-days)))`}$0
*************** END 
")
  (yas-load-snippet-buffer-and-close 'org-mode))

Now, you will have a new entry in the YASnippet menu that is called todo-followup. If you put the cursor on a blank line, and select that entry you get this below (after you fill in the text for the headline, of course!): 

*************** TODO see how many times this was viewed
		DEADLINE: <2014-02-23 Sun>
*************** END

That is pretty nice, as it saves a lot of keystrokes for that particular kind of task. Let us up the ante, and see if we can make it interactive so you can enter the number of days from now the task is due. 

(yas-global-mode)
(with-temp-buffer
  (insert "# name : todo-followup
# --

*************** TODO $1
${2:             DEADLINE: `(let ((ndays (seconds-to-time (* (string-to-int (read-from-minibuffer \"Days until due: \")) 24 60 60))))
  (format-time-string \"<%Y-%m-%d %a>\" (time-add (current-time) ndays)))`}$0
*************** END 
")
  (yas-load-snippet-buffer-and-close 'org-mode))

*************** TODO sweet!
		DEADLINE: <2014-02-26 Wed>
*************** END

Well, that made it just a bit sweeter! I was prompted for the "Days until due:", entered 10 days, and a date 10 days from now was automatically entered!

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.5h

Discuss on Twitter
« Previous Page -- Next Page »