Exporting numbered citations in html with unsorted numbered bibliography

Posted December 03, 2015 at 03:53 PM | categories: org-mode | tags:

Updated December 03, 2015 at 04:49 PM

In this post we illustrated a simple export of org-ref citations to html. This was a simple export that simply replaced each citation with a hyperlink as defined in the export function for each type of link. Today we look at formatting in text citations with superscripted numbers, and having an unsorted (i.e. in order of citation) numbered bibliography. This will take one pass to get the citations and calculate replacements and the bibliography, and one pass to replace them and insert the bibliography.

This text is just some text with somewhat random citations in it for seeing it work. You might like my two data sharing articles ^1,2. We illustrate the use of org-mode in publishing computational work ^3,4,5, experimental ⁶and mixed computational and experimental work ^7,8. This example will correctly number multiple references to a citation, e.g. ¹and ².

This post is somewhat long, and the way I worked it out is at the end (Long appendix illustrating how we got the code to work). The short version is that we do some preprocessing to get the citations in the document, calculate replacement values for them and the bibliography, replace them in the org-buffer before the export in the backend (html) format we want, and then conclude with the export. This is proof of concept work.

The main issues you can see are:

Our formatting code is very rudimentary, and relies on reftex. It is not as good as bibtex, or presumably some citation processor. Major improvements would require abandoning the reftex approach to use something that builds up the bibliography entry, allows modification of author names, and accomodates missing information gracefully.
The bibliography contents reflect the contents of my bibtex file, which is LaTeX compatible. We could clean it up more, by either post-processing to remove some things like escaped &, or by breaking compatibility with LaTeX.
The intext citations could use some fine tuning on spaces, e.g. to remove trailing spaces after words, or to move superscripts to the right of punctuation, or to adjust spaces after some citations.
Changing the bibliography style for each entry amounts to changing a variable for the bibliography. We have to modify a function to change the intext citation style, e.g. to brackets, or (author year).
I stuck with only cite links here, and only articles and books. It would not get a citenum format correct, e.g. it should not be superscripted in this case, or a citeauthor format correct. That would require some code in the replacement section that knows how to replace different types of citations.

The org-ref-unsrt-html-processor function could be broken up more, and could take some parameters to fine-tune some of these things, and generalize some things like getting the citation elements for the buffer. Overall, I think this shows that citations in org-mode with org-ref are actually pretty flexible. It is not as good as bibtex/LaTeX, and won't be for an unforseeably long time unless someone really needs high quality citations in a format other than LaTeX. Note for LaTeX export, we don't have to do any preprocessing at all. If you wanted to try Word export, you might make a pandoc processor that replaces everything in pandoc citation syntax, and then use pandoc for the conversion. If you didn't care to use the bibtex database for anything else, you could just use backend specific markup to make it exactly right for your output. I did this in reference ⁸where you can see the chemical formulas are properly subscripted.

If you would like to see the bibtex file used for this you can get it here: numbered.bib

Bibliography

Kitchin, Examples of Effective Data Sharing in Scientific Publishing, ACS Catalysis, 5(6), 3894-3899 (2015). link. doi.
John Kitchin, Data Sharing in Surface Science, Surface Science , (0), - (2015). link. doi.
Zhongnan Xu & John R Kitchin, Tuning Oxide Activity Through Modification of the Crystal and Electronic Structure: From Strain To Potential Polymorphs, Phys. Chem. Chem. Phys., 17(), 28943-28949 (2015). link. doi.
Prateek Mehta, Paul Salvador & John Kitchin, Identifying Potential BO2 Oxide Polymorphs for Epitaxial Growth Candidates, ACS Appl. Mater. Interfaces, 6(5), 3630-3639 (2014). link. doi.
Curnan & Kitchin, Effects of Concentration, Crystal Structure, Magnetism, and Electronic Structure Method on First-Principles Oxygen Vacancy Formation Energy Trends in Perovskites, The Journal of Physical Chemistry C, 118(49), 28776-28790 (2014). link. doi.
Hallenbeck & Kitchin, Effects of O2 and SO2 on the Capture Capacity of a Primary-Amine Based Polymeric CO2 Sorbent, Industrial & Engineering Chemistry Research, 52(31), 10788-10794 (2013). link. doi.
Spencer Miller, Vladimir Pushkarev, Andrew, Gellman & John Kitchin, Simulating Temperature Programmed Desorption of Oxygen on Pt(111) Using DFT Derived Coverage Dependent Desorption Barriers, Topics in Catalysis, 57(1-4), 106-117 (2014). link. doi.
Jacob Boes, Gamze Gumuslu, James Miller, Andrew, Gellman & John Kitchin, Estimating Bulk-Composition-Dependent H₂ Adsorption Energies on Cu_xPd_1-x Alloy (111) Surfaces, ACS Catalysis, 5(), 1020-1026 (2015). link. doi.

1 The working code

Here is a function to process the org file prior parsing during the export process. This function goes into org-export-before-parsing-hook, and takes one argument, the backend. We simply replace all the citation links with formatted HTML snippets or blocks. If the snippets get longer than a line, it will break.

We use org-ref-reftex-format-citation to generate the bibliography, which uses reftex to format a string with escape characters in it.

(setq org-ref-bibliography-entry-format
      '(("article" . "<li><a name=\"\%k\"></a>%a, %t, <i>%j</i>, <b>%v(%n)</b>, %p (%y). <a href=\"%U\">link</a>. <a href=\"https://doi.org/%D\">doi</a>.</li>")
        ("book" . "<li><a name=\"\%k\"></a>%a, %t, %u (%y).</li>")))

(defun org-ref-unsrt-latex-processor () nil)
(defun org-ref-unsrt-html-processor ()
  "Citation processor function for the unsrt style with html output."
  (let (links
        unique-keys numbered-keys
        replacements
        bibliography-link
        bibliographystyle-link
        bibliography)
    ;; step 1 - get the citation links
    (setq links (loop for link in (org-element-map
                                      (org-element-parse-buffer) 'link 'identity)
                      if (-contains?
                          org-ref-cite-types
                          (org-element-property :type link))
                      collect link))

    ;; list of unique numbered keys. '((key number))
    (setq unique-keys (loop for i from 1
                            for key in (org-ref-get-bibtex-keys)
                            collect (list key (number-to-string i))))


    ;; (start end replacement-text)
    (setq replacements
          (loop for link in links
                collect
                (let ((path (org-element-property :path link)))
                  (loop for (key number) in unique-keys
                        do
                        (setq
                         path
                         (replace-regexp-in-string
                          key (format "<a href=\"#%s\">%s</a>" key number)
                          path)))
                  (list (org-element-property :begin link)
                        (org-element-property :end link)
                        (format "@@html:<sup>%s</sup>@@" path)))))

    ;; construct the bibliography string
    (setq bibliography
          (concat "#+begin_html
<h1>Bibliography</h1><ol>"
                  (mapconcat
                   'identity
                   (loop for (key number) in unique-keys
                         collect
                         (let* ((result (org-ref-get-bibtex-key-and-file key))
                                (bibfile (cdr result))
                                (entry (save-excursion
                                         (with-temp-buffer
                                           (insert-file-contents bibfile)
                                           (bibtex-set-dialect
                                            (parsebib-find-bibtex-dialect) t)
                                           (bibtex-search-entry key)
                                           (bibtex-parse-entry t)))))
                           ;; remove escaped & in the strings
                           (replace-regexp-in-string "\\\\&" "&"
                                           (org-ref-reftex-format-citation
                                            entry
                                            (cdr (assoc (cdr (assoc "=type=" entry))
                                                        org-ref-bibliography-entry-format))))))
                   "")
                  "</ol>
#+end_html"))

    ;; now, we need to replace each citation. We do that in reverse order so the
    ;; positions do not change.
    (loop for (start end replacement) in (reverse replacements)
          do
          (setf (buffer-substring start end) replacement))

    ;; Eliminate bibliography style links
    (loop for link in (org-element-map
                          (org-element-parse-buffer) 'link 'identity)
          if (string= "bibliographystyle"
                      (org-element-property :type link))
          do
          (setf (buffer-substring (org-element-property :begin link)
                                  (org-element-property :end link))
                ""))

    ;; replace the bibliography link with the bibliography text
    (setq bibliography-link (loop for link in (org-element-map
                                                  (org-element-parse-buffer) 'link 'identity)
                                  if (string= "bibliography"
                                              (org-element-property :type link))
                                  collect link))
    (if (> (length bibliography-link) 1)
        (error "Only one bibliography link allowed"))

    (setq bibliography-link (car bibliography-link))
    (setf (buffer-substring (org-element-property :begin bibliography-link)
                            (org-element-property :end bibliography-link))
          bibliography)))


(defun org-ref-citation-processor (backend)
  "Figure out what to call and call it"
  (let (bibliographystyle)
    (setq
     bibliographystyle
     (org-element-property
      :path (car
             (loop for link in
                   (org-element-map
                       (org-element-parse-buffer) 'link 'identity)
                   if (string= "bibliographystyle"
                               (org-element-property :type link))
                   collect link))))
    (funcall (intern (format "org-ref-%s-%s-processor" bibliographystyle backend)))))

(add-hook 'org-export-before-parsing-hook 'org-ref-citation-processor)

(browse-url (org-html-export-to-html))

#<process open ./blog.html>

2 Long appendix illustrating how we got the code to work

The first thing we need is a list of all the citation links, in the order cited. Here they are.

(mapcar
 (lambda (link) (org-element-property :path link))
 (loop for link in (org-element-map (org-element-parse-buffer) 'link 'identity)
       if (-contains? org-ref-cite-types (org-element-property :type link))
       collect link))

kitchin-2015-examp,kitchin-2015-data-surfac-scien

xu-2015-tunin-oxide,mehta-2014-ident-poten,curnan-2014-effec-concen

hallenbeck-2013-effec-o2

miller-2014-simul-temper,boes-2015-estim-bulk

kitchin-2015-examp

kitchin-2015-data-surfac-scien

boes-2015-estim-bulk

Now, we need to compute replacements for each citation link, and construct the bibliography. We will make a numbered, unsorted bibliography, and we want to replace each citation with the corresponding numbers, hyperlinked to the entry.

We start with a list of the keys in the order cited, and a number we will use for each one.

(loop for i from 1
      for key in (org-ref-get-bibtex-keys)
      collect (list key i))

kitchin-2015-examp	1
kitchin-2015-data-surfac-scien	2
xu-2015-tunin-oxide	3
mehta-2014-ident-poten	4
curnan-2014-effec-concen	5
hallenbeck-2013-effec-o2	6
miller-2014-simul-temper	7
boes-2015-estim-bulk	8

Now, we need to compute replacements for each cite link. This will be replacing each key with the number above. We will return a list of ((start end) . "replacement text") that we can use to replace each link. For fun, we make these superscripted html.

(let ((links (loop for link in (org-element-map (org-element-parse-buffer) 'link 'identity)
                   if (-contains? org-ref-cite-types (org-element-property :type link))
                   collect link))
      (replacements (loop for i from 1
                          for key in (org-ref-get-bibtex-keys)
                          collect (list key (number-to-string i)))))
  (loop for link in links
        collect (let ((path (org-element-property :path link)))
                  (dolist (repl replacements)
                    (setq path (replace-regexp-in-string (car repl) (nth 1 repl) path)))
                  (list (org-element-property :begin link)
                        (org-element-property :end link)
                        (format "<sup>%s</sup>" path)))))

950	1004	<sup>1,2</sup>
1073	1145	<sup>3,4,5</sup>
1160	1190	<sup>6</sup>
1236	1286	<sup>7,8</sup>
1364	1388	<sup>1</sup>
1392	1427	<sup>2</sup>
4091	4117	<sup>8</sup>

We also need to compute the bibliography for each key. We will use org-ref-reftex-format-citation to do this. For that we need the parsed bibtex entries, and a format string. org-ref provides most of this.

(setq org-ref-bibliography-entry-format
      '(("article" . "<li>%a, %t, <i>%j</i>, <b>%v(%n)</b>, %p (%y). <a href=\"%U\">link</a>. <a href=\"https://doi.org/%D\">doi</a>.</li>")
        ("book" . "<li>%a, %t, %u (%y).</li>")))

(concat "<h1>Bibliography</h1><br><ol>"
        (mapconcat
         'identity
         (loop for key in (org-ref-get-bibtex-keys)
               collect
               (let* ((result (org-ref-get-bibtex-key-and-file key))
                      (bibfile (cdr result))
                      (entry (save-excursion
                               (with-temp-buffer
                                 (insert-file-contents bibfile)
                                 (bibtex-set-dialect (parsebib-find-bibtex-dialect) t)
                                 (bibtex-search-entry key)
                                 (bibtex-parse-entry)))))
                 (org-ref-reftex-format-citation
                  entry
                  (cdr (assoc (cdr (assoc "=type=" entry))
                              org-ref-bibliography-entry-format)))))
         "")
        "</ol>")

Bibliography

Kitchin, Examples of Effective Data Sharing in Scientific Publishing, {ACS Catalysis}, 5(6), 3894-3899 (2015). link. doi.
"John Kitchin", Data Sharing in Surface Science, "Surface Science ", (0), - (2015). link. doi.
Zhongnan Xu \& John R Kitchin, Tuning Oxide Activity Through Modification of the Crystal and Electronic Structure: From Strain To Potential Polymorphs, {Phys. Chem. Chem. Phys.}, 17(), 28943-28949 (2015). link. doi.
Prateek Mehta, Paul Salvador \& John Kitchin, Identifying Potential BO2 Oxide Polymorphs for Epitaxial Growth Candidates, {ACS Appl. Mater. Interfaces}, 6(5), 3630-3639 (2014). link. doi.
Curnan \& Kitchin, Effects of Concentration, Crystal Structure, Magnetism, and Electronic Structure Method on First-Principles Oxygen Vacancy Formation Energy Trends in Perovskites, {The Journal of Physical Chemistry C}, 118(49), 28776-28790 (2014). link. doi.
"Hallenbeck \& Kitchin, Effects of O2 and SO2 on the Capture Capacity of a Primary-Amine Based Polymeric CO2 Sorbent, "Industrial \& Engineering Chemistry Research", 52(31), 10788-10794 (2013). link. doi.
Spencer Miller, Vladimir Pushkarev, Andrew, Gellman \& John Kitchin, Simulating Temperature Programmed Desorption of Oxygen on Pt(111) Using DFT Derived Coverage Dependent Desorption Barriers, {Topics in Catalysis}, 57(1-4), 106-117 (2014). link. doi.
Jacob Boes, Gamze Gumuslu, James Miller, Andrew, Gellman \& John Kitchin, Estimating Bulk-Composition-Dependent H₂ Adsorption Energies on Cu_xPd_1-x Alloy (111) Surfaces, {ACS Catalysis}, 5(), 1020-1026 (2015). link. doi.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Post-processing an org-buffer on export

Posted December 01, 2015 at 01:58 PM | categories: orgmode, emacs | tags:

Updated December 01, 2015 at 09:01 PM

In a previous post we examined getting tooltips on emacs keybindings and command syntax in an org-buffer. Someone asked in a comment if we could get that to export to html, or LaTeX. The short answer is not directly, org-mode doesn't recognize our functionalized syntax as an element, and there is no direct way to modify their appearance on export.

There is, however, a hook function that runs before parsing, and we can use that to transform these patterns to what we want. Here, I illustrate how to make the key-bindings and commands bold with a tooltip on them for an html export. Basically, we do an export, and then post-process the html output to put what we want in. I found this easier than pre-processing because the documentation for the command tooltip was too big to fit into an html snippet, and an html block causes carriage returns in the html. I didn't find a more elegant solution to that problem.

Here we replace the key-binding syntax with the actual keybinding in bold, and a tooltip of the command, and the command syntax we replace with bold command and a tooltip for the documentation. It works pretty well. The documentation for helm is pretty extensive, and gets cutoff in the tooltip. Otherwise, this seems pretty satisfactory.

This won't show in the blog post, so you will have to checkout the exported html here: blog.html .

Try \C-x C-f to open a file. You might enjoy using helm too. Or this variable org-agenda-files.

(require 'rx)

(defvar elisp-symbol-keybinding-re
  (rx
   ;; opening \\[
   (eval "\\[")
   ;; one or more characters that are not ]
   (group (one-or-more (not (any "]"))))
   ;; The closing ]
   "]")
"Regexp for an elisp command keybinding syntax. \\[some-command]
Regexp group 1 matches src_emacs-lisp[:results html]{(command-html "some-command")}.")

(defun org-process-key-bindings (backend)
  (goto-char (point-min))
  (while (re-search-forward elisp-symbol-keybinding-re nil t)
    (replace-match
     (cond
      ((eq backend 'html)
       (format "<b title=\"The command is %s.\">%s</b>"
               (match-string 1)
               (substitute-command-keys (match-string 0))))))))


(defun org-process-emacs-commands (backend)
  (goto-char (point-min))
  (while (re-search-forward "`\\([^']+\\)'" nil t)
    (replace-match
     (cond
      ((eq backend 'html)
       (format "<b title=\"%s\">%s</b>"
          (if (or (fboundp (intern (match-string 1)))
                  (boundp (intern (match-string 1))))
              (documentation (intern (match-string 1)))
            "No command found.")
          (match-string 1)))))))

(with-current-buffer (org-html-export-as-html)
  (org-process-key-bindings 'html)
  (org-process-emacs-commands 'html)
  (write-file "blog.html")
  (browse-url "blog.html"))

org-process-emacs-commands

1 Update: A filter approach to exporting

Our patterns are all in org-paragraphs. We can use a filter to modify the paragraph after it is "transcoded". Here is the filter function. It basically does the same thing, through another mechanism.

(defun my-paragraph-filter (data backend info)
  (cond
   ((eq 'html backend)
    ;; emacs commands
    (setq data (replace-regexp-in-string
                "`\\([^']+\\)'"
                (lambda (x)
                  (string-match "`\\([^']+\\)'" x)
                  (when (match-string 1 x)
                    (format "<b title=\"%s\">%s</b>"
                            (if (or (fboundp (intern (match-string 1 x)))
                                    (boundp (intern (match-string 1 x))))
                                (documentation (intern (match-string 1 x)))
                              "No command found.")
                            (match-string 1 x))))
                data))
    ;; keybindings
    (setq data (replace-regexp-in-string
                "\\\\\\[\\([^]]+\\)]"
                (lambda (x)
                  (string-match "\\\\\\[\\([^]]+\\)]" x)
                  (when (fboundp  (intern (match-string-no-properties 1 x)))
                    (format "<b title=\"The command is %s.\">%s</b>"
                            (match-string 1 x)
                            (substitute-command-keys (match-string 0 x)))))
                data)))))

(add-to-list 'org-export-filter-paragraph-functions 'my-paragraph-filter)
(browse-url (org-html-export-to-html))

#<process open ./blog.html>

Try the command org-ref to check your document for issues. Insert a citation with org-ref using \M-ESC C.

Now, this should show up in the blog!

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

LDAP lookups from Emacs

Posted November 25, 2015 at 09:31 AM | categories: helm, emacs | tags:

Now that I have email and Cisco Jabber totally integrated into Emacs it would be nice to tap into the CMU LDAP (Lightweight Directory Access Protocol) service to find emails and phone numbers. We to use the ldapsearch command-line utility to query our LDAP service like this to find an email address.

You might like the video explanation here: https://www.youtube.com/watch?v=N7AaKHRd9uw

(shell-command-to-string "ldapsearch -x -LLL -h ldap.andrew.cmu.edu -b ou=Person,dc=cmu,dc=edu cn=\"John Kitchin\"")

dn: guid=1976CCAA-B465-11D8-8000-080020CC75D3,ou=person,dc=cmu,dc=edu
objectClass: cmuPerson
eduPersonPrimaryAffiliation: Faculty
guid: 1976CCAA-B465-11D8-8000-080020CC75D3
cmuPrivate: homePostalAddress
cmuPrivate: homePhone
cn: John Kitchin
givenName: John
sn: Kitchin
cmuPrimaryCampus: Pittsburgh
cmuCampus: Pittsburgh
cmuAndrewId: jkitchin
cmueduId: jkitchin
cmuAndrewCommonNamespaceId: jkitchin
mail: jkitchin@cmu.edu
eduPersonSchoolCollegeName: CIT - Consolidated
cmuPersonPrincipalName: jkitchin@ANDREW.CMU.EDU
postalAddress: DH A207F
cmuDepartment: Chemical Engineering
cmuDepartment: MSE: Materials Science & Engineering
cmuPersonAffiliation: Tenure-Track Faculty
eduPersonAffiliation: Faculty
cmuAccount: uid=jkitchin,ou=account,dc=andrew,dc=cmu,dc=edu
cmuAccount: uid=jkitchin,ou=account,dc=cmu,dc=edu
cmuActiveDN: uid=jkitchin,ou=account,dc=andrew,dc=cmu,dc=edu
cmuActiveDN: uid=jkitchin,ou=account,dc=cmu,dc=edu
title: Professor
telephoneNumber: +1 412 268 7803

We actually get LDIF data from ldapsearch with a lot of details. Next we wrap the output in a function that converts each result from ldapsearch into a p-list that we will use later in a helm function to help us select a match.

(defun ldap-query (query-string)
  "Send QUERY-STRING to our ldap server and parse results into a
list of p-lists for each entry returned."
  (interactive "sLDAP query: ")
  (let ((output (butlast (split-string
                          (shell-command-to-string
                           (format (concat  "ldapsearch -x -LLL "
                                            "-h ldap.andrew.cmu.edu "
                                            "-b ou=Person,dc=cmu,dc=edu %s")
                                   query-string))
                          "\n")))
        (lines '())
        (result '())
        (results '(())))
    ;; cleanup trailing lines and ignore initial lines
    (loop for line in output
          do
          (cond
           ;; join lines that run over
           ((s-starts-with? " " line)
            (setf (car (last lines))
                  (concat (car (last lines)) line)))
           ;; ignore this
           ((string-match "Size limit exceeded" line)
            nil)
           (t
            (add-to-list 'lines line t))))

    ;; now we need to parse the lines. A new entry starts with a dn: line.
    (dolist (line lines)
      (cond
       ((s-starts-with? "dn:" line)
        ;; add new entry
        (add-to-list 'results `(:dn ,line)))
       ((string-match ":" line)
        (let* ((s (split-string line ":"))
               (prop (intern (concat ":" (s-trim (car s)))))
               (val (s-trim (cadr s))))
          (setf (car results) (plist-put (car results) prop val))))))
    ;; last result seems to be nil so we drop it
    (-filter (lambda (x) (not (null x))) results)))

ldap-query

Here is an example of that function:

(ldap-query "cn=\"John Kitchin\"")

((:dn "dn: guid=1976CCAA-B465-11D8-8000-080020CC75D3,ou=person,dc=cmu,dc=edu" :objectClass "cmuPerson" :eduPersonPrimaryAffiliation "Faculty" :guid "1976CCAA-B465-11D8-8000-080020CC75D3" :cmuPrivate "homePhone" :cn "John Kitchin" :givenName "John" :sn "Kitchin" :cmuPrimaryCampus "Pittsburgh" :cmuCampus "Pittsburgh" :cmuAndrewId "jkitchin" :cmueduId "jkitchin" :cmuAndrewCommonNamespaceId "jkitchin" :mail "jkitchin@cmu.edu" :eduPersonSchoolCollegeName "CIT - Consolidated" :cmuPersonPrincipalName "jkitchin@ANDREW.CMU.EDU" :postalAddress "DH A207F" :cmuDepartment "MSE" :cmuPersonAffiliation "Tenure-Track Faculty" :eduPersonAffiliation "Faculty" :cmuAccount "uid=jkitchin,ou=account,dc=cmu,dc=edu" :cmuActiveDN "uid=jkitchin,ou=account,dc=cmu,dc=edu" :title "Professor" :telephoneNumber "+1 412 268 7803"))

Now, we wrap a helm function around that to give us a nice menu to select entries from, and a few actions like sending an email, calling, copying the name and email, and seeing the information in a reasonable way. We also add a fallback method in case we don't find what we want and need to do a new search.

(defun helm-ldap (query-string)
  (interactive "sLDAP query: ")
  (helm
   :sources
   `(((name . "HELM ldap")
      (candidates . ,(mapcar
                      (lambda (x)
                        (cons
                         (format
                          "%20s|%30s|%30s|%20s|%s"
                          (s-truncate
                           20
                           (or (plist-get x :title) " "))
                          (plist-get x :cn)
                          (plist-get x :mail)
                          (plist-get x :cmuDisplayAddress)
                          (or (plist-get x :telephoneNumber) " "))
                         x))
                      (ldap-query
                       (if (string-match "=" query-string)
                           query-string
                         (concat "cn=*" query-string "*")))))
      (action . (("Email" . (lambda (x)
                              (compose-mail)
                              (message-goto-to)
                              (insert (plist-get x :mail))
                              (message-goto-subject)))
                 ("Call" . (lambda (x)
                             (cisco-call
                              (plist-get x :telephoneNumber))))
                 ("Copy Name and email address" . (lambda (x)
                                                    (kill-new
                                                     (format
                                                      "%s <%s>"
                                                      (plist-get x :cn)
                                                      (plist-get x :mail)))))
                 ("Information" . (lambda (x)
                                    (switch-to-buffer
                                     (get-buffer-create "*helm ldap*"))
                                    (erase-buffer)
                                    (dolist (key (cl-loop
                                                  for key in x by #'cddr
                                                  collect key))
                                      (insert (format "|%s | %s|\n"
                                                      key (plist-get x key))))
                                    (org-mode)
                                    (goto-char 0)
                                    (org-ctrl-c-ctrl-c)
                                    (insert "press q to quit.\n\n")
                                    (setq buffer-read-only t)
                                    (use-local-map (copy-keymap org-mode-map))
                                    (local-set-key "q"
                                                   #'(lambda ()
                                                       (interactive)
                                                       (quit-window t))))))))
     ;; fallback action
     ((name . "New search")
      (dummy)
      (action . (lambda (x) (helm-ldap x)))))))

helm-ldap

That is pretty convenient!

John Kitchin <jkitchin@cmu.edu>

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Contextual help in org-mode

Posted November 24, 2015 at 10:08 AM | categories: orgmode, emacs | tags:

org-mode is great, plain text and all, but it can be difficult to figure out all the things you can do at any particular place in the buffer. Here, we explore some ideas on making org-mode a bit more discoverable. One way to do this that we explore here is to create a help function that you run, and it tells you about the element that `org-element-context' knows about, then gives you some hints of what you can do there. To do this, we create a series of functions for each kind of element we provide help on.

You may want to see the video: https://www.youtube.com/watch?v=99uxJWqZGzA

The kind of help we want is a description of the element, some commands we can run on the element and what they do, and if there are some key-bindings. We also want to make sure if the user has changed the key-bindings, the right values get shown. Finally, it would be nice to allow users to add their own documentation if they want.

So, here is the lengthy block of code we use for this purpose. We create a function for each element type that we provide documentation for. We also provide a mechanism for users to add their own notes for future reference. I leverage the help-buffer here to simplify some functional text, e.g. the key commands and clickable functions, as well as history navigation. I could also do most of that in org-mode buffers, with different functionality, but I save that for another day.

(defvar ore-user-directory "~/.emacs.d/ore/"
  "Directory to store user additions to the notes.")


(defun ore-user-documentation (type)
  "Return user documentation for org element TYPE if it exists.
Notes are returned as plain text, and will be rendered in `help-mode'."
  (let ((fname (expand-file-name (format "%s.org" type) ore-user-directory)))
    (concat
     "User documentation:\n"
     (if (file-exists-p fname)
         (with-temp-buffer
           (insert "\n")
           (insert-file-contents fname)
           (indent-rigidly (point-min) (point-max) 5)
           (buffer-string))
       "None defined.")
     (format  "\n\nEdit [[file:%s]]" fname))))


(defun ore-latex (element)
  "`ore' documentation for latex fragment."
  (concat
   (substitute-command-keys "You are on a LaTeX fragment or environment.

\\[org-toggle-latex-overlays] or `org-toggle-latex-overlays' to toggle LaTeX images on it.

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'latex)))


(defun ore-link (element)
  "`ore' documentation for org links."
  (let* ((link (org-element-context))
         (type (org-element-property :type link))
         (protocols (assoc type org-link-protocols))
         (follow-func (cadr protocols))
         (export-func (caddr protocols)))
    (concat
     (format
      (substitute-command-keys "You are on a %s link.

Link path: %s
%s

Clicking on the link will run `%s'.

This link uses this function for export: `%s'

If you are on an image link, you can toggle it with \\[org-toggle-inline-images] or `org-toggle-inline-images'.

You can toggle the link display with `org-toggle-link-display'.

See Info node `(org) Hyperlinks'.

%s

%s\n\n")
      type
      (org-element-property :path link)
      (format "Whole link: %s" (buffer-substring
                                (org-element-property :begin link)
                                (org-element-property :end link)))
      (pp-to-string follow-func)
      (pp-to-string export-func)
      (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
      (ore-user-documentation 'link)))))


(defun ore-src-block-header-p (element)
  "Return whether point is in a src-block header."
  (and (eq 'src-block (car element))
       (save-excursion
         (let ((cp (point))
               (lp (line-number-at-pos)))
           (goto-char (org-element-property :begin element))
           (= lp (line-number-at-pos))))))


(defun ore-src-block-header (element)
  "`ore' documentation for src-block header."
  (concat
   "You are in a src-block header.

This line tells org-mode that
it is a src-block, and language of the src-block. There are
also optional header arguments. See Info node `(org) Header arguments'

"
   (format "The default headers are described here: `org-babel-default-header-args:%s'

" (org-element-property :language element))
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'src-block-header)))


(defun ore-src-block (element)
  "`ore' documentation for a src-block."
  (concat
   (substitute-command-keys "You are in a src-block.

C-c C-c to execute this block.
\\[org-babel-tangle]  org-babel-tangle

You can edit the block with \\[org-edit-special] or `org-edit-special'.

See Info node `(org) Working with source code' for more details.\n\n")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'src-block)))


(defun ore-headline (element)
  "`ore' documentation for a headline."
  (concat
   (when (bolp)
     (format  "You are at the beginning of a headline.

\\[org-cycle] to cycle Info node `(org) Global and local cycling'.

Check `org-use-speed-commands'\n\n"))

   ;; in a headline
   (substitute-command-keys
    "You are in a headline. You can change:

 Visibility with \\[org-cycle]

 TODO state \\[org-shiftleft] and \\[org-shiftright] or
 `org-todo'.

 Your current todo sets can be found in `org-todo-sets'.

 Priority \\[org-shiftup] (`org-priority-up') and
 \\[org-shiftdown] (`org-priority-down')

 Tags  \\[org-ctrl-c-ctrl-c] or `org-set-tags'

 Set a property with \\[org-set-property] `org-set-property'.

 Delete a property with \\[org-delete-property] or `org-delete-property'.

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'headline)))


;; timestamp
(defun ore-timestamp (element)
  "`ore' documentation for timestamps."
  (concat
   (substitute-command-keys "You are on a timestamp.

If you click on it, you will see the date in the agenda. With the
cursor on the <> or [] \\[org-shiftup] and \\[org-shiftdown] will
switch from active to inactive timestamps.

You can change the date by putting the cursor on a date part and
using \\[org-shiftup] and \\[org-shiftdown] or \\[org-shiftleft]
and \\[org-shiftright]

See Info node `(org) Dates and times'.

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'timestamp)))


;; table table-row table-cell
(defun ore-table (element)
  "`ore' documentation for tables."
  (concat
   "You are in a table.

Move cell to cell with [TAB]. When you are in the last cell, TAB adds a new row.

Move rows up and down with \\[org-metaup] and \\[org-metadown].
Move columns left and right with \\[org-metaleft] and \\[org-metaright].

Sort a column with \\[org-sort] `org-sort'.

Insert a row with `org-table-insert-row'.
Delete a row with `org-table-kill-row'.

Insert a column with `org-table-insert-column'.
Delete a column with `org-table-delete-column'.

`C-c -     (`org-table-insert-hline')'
     Insert a horizontal line below current row.  With a prefix
     argument, the line is created above the current line.

`C-c <RET>     (`org-table-hline-and-move')'
     Insert a horizontal line below current row, and move the cursor
     into the row below that line.

You can transpose a table with `org-table-transpose-table-at-point'.

Info node `(org) Tables'.

"
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'table)))


(defun ore-plain-list (element)
  "`ore' doucmentation for plain lists."
  (concat
   (substitute-command-keys
    "You are on a plain list.
See Info node `(org) Plain lists'.

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'plain-list)))


(defun ore-item (element)
  "`ore' documentation for items in a list"
  (concat
   (substitute-command-keys
    "You are on an item in a list.

You can move items up and down with \\[org-metaup] or `org-metaup' and \\[org-meta-down] or `org-metadown'.

You can add a new item with \\[org-meta-return] or `org-meta-return'.

You can change the indentation of an item with  \\[org-metaleft] or `org-metaleft' and \\[org-meta-right] or `org-meta-right'.

You can change the bullet of the item with  \\[org-shiftleft] or `org-shiftleft' and \\[org-shiftright] or `org-shiftright'.

See Info node `(org) Plain lists' for other things like sorting, cycling, checkboxes, etc...

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'item)))


(defun ore-keyword (element)
  "`ore' documentation for a keyword."
  (concat
   (substitute-command-keys
    "You are on a keyword.

You may need to run \\[org-ctrl-c-ctrl-c] or `org-ctrl-c-ctrl-c' to refresh its value if you change it.

You can move keywords up and down with \\[org-metaup] or `org-metaup' and \\[org-metadown] or `org-metadown'.

")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'keyword)))


(defun ore-paragraph (element)
  "`ore' documentation for a paragraph."
  (concat
   (substitute-command-keys
    "You are in a paragraph.

You can move a paragraph up with \\[org-metaup] or `org-metaup'.

You can move a paragraph up with \\[org-metadown] or `org-metadown'.

You can mark the paragraph with `mark-paragraph'.


")
   (format  "\nClick for details on the face [[face:%s]]\n" (face-at-point))
   (ore-user-documentation 'paragraph)))


(defun ore ()
  "Help function for the org-mode element at point."
  (interactive)
  (with-help-window
      (help-buffer)
    (let* ((oeap (org-element-context))
           (ore-func (intern (format "ore-%s" (car oeap))))
           (s (if (fboundp ore-func)
                  (funcall ore-func oeap)
                (format
                 "No documentation found for %s.

%s"
                 (car oeap)
                 (ore-user-documentation (car oeap))))))
      ;; There are some special cases.
      (cond
       ((and  (eq 'src-block (car oeap))
              (ore-src-block-header-p oeap))
        (setq s (ore-src-block-header oeap)))

       ((or (eq 'table (car oeap))
            (eq 'table-row (car oeap))
            (eq 'table-cell (car oeap)))
        (setq s (ore-table oeap)))

       ((or (eq 'latex-fragment (car oeap))
            (eq 'latex-environment (car oeap)))
        (setq s (ore-latex oeap))))

      (princ s)
      (princ "\n\nHere is how org-mode sees the element.\n\n")
      (pp oeap))))


(defun match-next-ore-file (&optional limit)
  "Font-lock function to make file links clickable in help-mode."
  (when  (re-search-forward "\\[\\[file:\\([^]]*\\)\\]\\]" limit t)
    (let* ((fname (expand-file-name
                   (match-string 1)
                   ore-user-directory))
           (beg (match-beginning 0))
           (end (match-end 0))
           (find-func `(lambda ()
                         (interactive)
                         (find-file ,fname))))

      (add-text-properties
       beg
       end
       `(mouse-face
         highlight
         display "User documentation"
         local-map ,(let ((map (copy-keymap help-mode-map)))
                      (define-key map [mouse-1] find-func)
                      map)
         help-echo (format
                    "Click to edit User documentation.\n%s"
                    fname))))))

(defun match-next-ore-face (&optional limit)
  "Font-lock function to make face links clickable in help-mode."
  (when  (re-search-forward "\\[\\[face:\\([^]]*\\)\\]\\]" limit t)
    (let* ((face (match-string 1))
           (beg (match-beginning 0))
           (end (match-end 0))
           (func `(lambda ()
                    (interactive)
                    (describe-face ,face))))

      (add-text-properties
       beg
       end
       `(mouse-face
         highlight
         local-map ,(let ((map (copy-keymap help-mode-map)))
                      (define-key map [mouse-1] func)
                      map)
         display ,face
         help-echo (format
                    "Click to show face information.\n%s"
                    face))))))

(add-hook 'help-mode-hook
          (lambda ()
            (font-lock-add-keywords
             nil
             '((match-next-ore-file . font-lock-keyword-face)
               (match-next-ore-face . font-lock-keyword-face)))))


;; Let's add to the org menu for "Help at point"
(easy-menu-change
 '("Org")
 "Help"
 '(["Help at point" ore])
 "Show/Hide")

xu-suppor $e^x$

1 TODO elements

a	b
3	4

list (plain/numbered)

add element
- move elements (up/down indent/outdent)
item 1
item 2
item 3

checkbox

[ ] check it

org#External links

lizzit-2001-surfac-ru

$latex_fragment$

$latex fragment$

\begin{equation} 2-3 \end{equation}

(+ 3 4 5)

<2015-10-18 Sun> [2015-10-18 Sun]

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Improving org-ref cite links with tooltips

Posted November 23, 2015 at 07:03 AM | categories: orgmode, emacs, orgref | tags:

Org-ref uses timers to give you messages about the cite link at point. I am not so crazy about the timer, there is always a (short) delay, and I have had trouble debugging timers in the past, and you have to put the point on the link. Since I wrote that code, I have learned some new things about Emacs, including dynamic tooltips. This will allow me to use the mouse to see what a cite link refers to. While reading documents, I am more likely to use a mouse than when typing a document, and getting a tooltip by hovering sounds like a good idea.

Here, we explore using dynamic tooltips on cite links. The idea is pretty simple, we tie into font-lock to add a function to the :help-echo property of a cite link. The function will go to point, and compute the citation string at point, which will be displayed as a tooltip when the mouse hovers over the citation.

Font-lock allows you to specify a function that sets match-data and that can have other side-effects, e.g. setting text properties. Org-ref has a regexp that defines cite links, which we use here, and a function that gets the citation string at point. We just go to the mouse position, and get that string, wrapped in a save-excursion macro so that point does not actually move. Then, we add the function to font-lock keywords, and we are done!

Here are some papers we wrote on using org-mode kitchin-2015-examp,kitchin-2015-data-surfac-scien and some other references in my bibliography zou-2014-cobal-embed,zlotea-2014-nanoal and one final example zhu-2015.

Here is the short code required to do this. You can see the tooltips in action here: https://www.youtube.com/watch?v=ifSmlId2rk0

(defun org-ref-match-next-cite-link (&optional limit)
  (when (re-search-forward org-ref-cite-re limit t)
    (add-text-properties
     (match-beginning 0) (match-end 0)
     (list
      'help-echo (lambda (window object position)
                   (save-excursion
                     (goto-char position)
                     (let ((s (org-ref-get-citation-string-at-point)))
                       (with-temp-buffer
                         (insert s)
                         (fill-paragraph)
                         (buffer-string)))))))))

; do this for this buffer
(font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)
(font-lock-fontify-buffer)

;; do this for every org file
(add-hook
 'org-mode-hook
 (lambda ()
   (font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)))