org-ref is on Melpa

Posted December 22, 2015 at 11:04 AM | categories: orgmode, orgref, emacs | tags:

Updated December 22, 2015 at 11:21 AM

org-ref is out on Melpa !

Checkout this video (≈ 10 min.) of what it can do: https://www.youtube.com/watch?v=2t925KRBbFc

Here are the files that we used/generated:

  1. Emacs configuration: org-ref-melpa.el
  2. Here is the "manuscript" manuscript.org (note, I extracted the bibtex entries into this file)
  3. The resulting PDF: manuscript.pdf

Some killer new features:

  1. Drag-n-drop a PDF or url onto a bibtex file to add bibtex entries. This works when org-ref knows how to get a DOI from the PDF or url.
  2. Tooltips on cite links

Thanks everyone who has already tried it out and reported bugs!

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Introduction to a citation processor in org-ref

Posted December 11, 2015 at 06:05 PM | categories: orgmode, citations, orgref, emacs | tags:

Updated December 12, 2015 at 10:08 AM

As a potential solution for citations in org-mode for non-LaTeX export, here we introduce csl (citation syntax lisp). The idea is heavily influenced by the xml-based Citation Syntax Language, but uses lisp sexps instead.

Briefly, there is a csl file that contains two variables: citation-style and bibliography-style. The citation-style defines how the in-text citations are represented for different types of citations. The bibliography-style defines how the bibliography is constructed.

What do we gain by this?

  1. No need for external citeproc program, and hackability by org-mode experts.
  2. Punctuation transposition and space chomping, i.e. put superscripts on the right side of punctuation if you want it, and remove whitespace before superscripts if you want it.
  3. Total tunability of the citation format to different backends.
  4. Easy to change bibliography format with the bibliographystyle link.
  5. The use of Bibtex databases. These are plain text, and flexible.

The real code for this is too long to blog about. Instead, you should check it out here: https://github.com/jkitchin/org-ref/tree/master/citeproc

1 Reference types

  • A book.1
  • An article2
  • A miscellaneous bibtex type.3

There is work to do in supporting other types of entry types that are common in bibtex files.

2 Citation types

  • Regular citation:2
  • citenum: See Ref. 2
  • citeauthor: Kitchin
  • citeyear: 2015

There is work to do in supporting other types of citations.

3 Multiple citations and sorting within citation

You can specify that the cites within a citation are consistently sorted in the export.

  • a,b:2,4
  • b,a:2,4

There is work to do for range collapsing, e.g. to turn 1,2,3 into 1-3.

4 Space chomping and punctuation testing

I think citations should always be put in the sentence they logically belong to. LaTeX has a feature through natbib I think where for some styles, e.g. superscripts, the citations are moved to the right side of punctuation, and whitespace is chomped so the superscript is next to words, not separated by spaces. We can do that here too.

  • Citation at end of sentence.2
  • Citation in clause,2,4 with a comma.
  • Citation in middle of2,4 a sentence.

5 Building

At the moment, you have to add a hook function to put the replacements in the document before parsing. 

(add-to-list 'load-path ".")
(require 'org-ref-citeproc)

(when (file-exists-p "readme.html") (delete-file "readme.html"))
(let ((org-export-before-parsing-hook '(orcp-citeproc)))
  (browse-url (org-html-export-to-html)))

#<process open ./readme.html>

(add-hook 'org-export-before-parsing-hook 'orcp-citeproc)
orcp-citeproc

6 Summary thoughts

This looks promising. There is probably a lot of work to do to make this as robust as say citeproc-js or the Zotero handler. I am not sure if we could write this in a way to directly use the CSL. My feeling is it would not be as flexible as this, and we would have to add to it anyway.

Here are some remaining things that could be worked on if we continue this direction.

  1. Other bibtex entries need to be tested out.
  2. Remaining bibtex fields need to be defined.
  3. Standardization of styling that can be done. Not all features described in my csl are supported, e.g. et. al. and probably others.
  4. The author-year style needs name disambiguation somehow.
  5. Hyperlinking in html.
  6. Make sure export to other backends works.
  7. Can this work for notes-based styles?

7 Bibliography

You use a bibliographystyle link to specify a csl. These are similar to bibtex styles, and in some cases no change is needed for LaTeX export (although you may have to remove the citeproc hook function).

  1. Kittel, Charles, Introduction to Solid State Physics, (2005).
  2. Kitchin, John R., Examples of Effective Data Sharing in Scientific Publishing, ACS Catalysis, 5(6), pp. 3894-3899 (2015). https://doi.org/10.1021/acscatal.5b00538.
  3. Xu, Zhongnan; Rossmeisl, Jan and Kitchin, John R., Supporting data for: A linear response, {DFT+U} study of trends in the oxygen evolution activity of transition metal rutile dioxides. doi:10.5281/zenodo.12635, https://doi.org/https://zenodo.org/record/12635. https://doi.org/10.5281/zenodo.12635.
  4. Kitchin, John R., Data Sharing in Surface Science, Surface Science , N/A, pp. in press (2015). https://doi.org/10.1016/j.susc.2015.05.007.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Improving org-ref cite links with tooltips

Posted November 23, 2015 at 07:03 AM | categories: orgmode, orgref, emacs | tags:

Org-ref uses timers to give you messages about the cite link at point. I am not so crazy about the timer, there is always a (short) delay, and I have had trouble debugging timers in the past, and you have to put the point on the link. Since I wrote that code, I have learned some new things about Emacs, including dynamic tooltips. This will allow me to use the mouse to see what a cite link refers to. While reading documents, I am more likely to use a mouse than when typing a document, and getting a tooltip by hovering sounds like a good idea.

Here, we explore using dynamic tooltips on cite links. The idea is pretty simple, we tie into font-lock to add a function to the :help-echo property of a cite link. The function will go to point, and compute the citation string at point, which will be displayed as a tooltip when the mouse hovers over the citation.

Font-lock allows you to specify a function that sets match-data and that can have other side-effects, e.g. setting text properties. Org-ref has a regexp that defines cite links, which we use here, and a function that gets the citation string at point. We just go to the mouse position, and get that string, wrapped in a save-excursion macro so that point does not actually move. Then, we add the function to font-lock keywords, and we are done!

Here are some papers we wrote on using org-mode kitchin-2015-examp,kitchin-2015-data-surfac-scien and some other references in my bibliography zou-2014-cobal-embed,zlotea-2014-nanoal and one final example zhu-2015.

Here is the short code required to do this. You can see the tooltips in action here: https://www.youtube.com/watch?v=ifSmlId2rk0 

(defun org-ref-match-next-cite-link (&optional limit)
  (when (re-search-forward org-ref-cite-re limit t)
    (add-text-properties
     (match-beginning 0) (match-end 0)
     (list
      'help-echo (lambda (window object position)
                   (save-excursion
                     (goto-char position)
                     (let ((s (org-ref-get-citation-string-at-point)))
                       (with-temp-buffer
                         (insert s)
                         (fill-paragraph)
                         (buffer-string)))))))))

; do this for this buffer
(font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)
(font-lock-fontify-buffer)

;; do this for every org file
(add-hook
 'org-mode-hook
 (lambda ()
   (font-lock-add-keywords
    nil
    '((org-ref-match-next-cite-link (0  'org-ref-cite-face t)))
    t)))

Bibliography

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Update on org-ref - it is now all emacs-lisp

Posted March 16, 2015 at 08:51 AM | categories: orgmode, orgref, emacs | tags:

The org-ref code is finally all in emacs-lisp! This should make it much easier to install, and is another step closer to getting org-ref into MELPA. Previously, I had written the most significant code in org-mode source blocks that were intended to be tangled out. I found this was not really portable, because what gets tangled depends on your org-mode setup. I had to specifically set example blocks to not tangle, or org-ref would not work for other people, and if I forgot to set a block to tangle, it also would not work for others. That should not happen again now, since there is no more tangling.

There are some relatively new features in org-ref:

  1. New colored org-ref links to differentiate them from other org-links. Citations are greenish, refs and labels are maroonish.
  2. Context messages about links. With your cursor on a cite, ref or label link you will get a context message, e.g. a formatted citation, some context about the label a ref refers to, or a count of the labels in the mini-buffer.
  3. There is now an org-ref menu in the Org menu.
  4. There is a new org-ref-help function that opens an org-file of org-ref documentation.
  5. Pretty thorough integration of helm throughout org-ref, and some integration of hydra.
  6. A few utility libraries: doi-utils, isbn, wos, pubmed, arxiv, jmax-bibtex, sci-id, x2bib. Not all these are new, but if you didn't know about them, check them out.
  7. Cask integration. This mostly provides access to testing and dependencies right now. org-ref is also now tested continuously at https://travis-ci.org/jkitchin/org-ref .

org-ref is basically feature complete I think (which is to say that once again, I do not have any big ideas for new features ;). There are some places where it could be refactored a little, e.g. there are some bibtex only functions in org-ref.el that really should go into jmax-bibtex.el (which also could be renamed). This is a very low priority though, because things are working fine as far as I can tell.

What does it need before going into MELPA? Probably some tests would be a good idea. On Travis, all that is really tested is that it loads with no errors. I would like to see some stability on my end, e.g. at least a week where no commits get made, and no errors are reported. And finally, I would like to make sure I have some time to handle issues that come up when a broader audience is trying it out.

My target date to get this in MELPA is June 1, 2015. Try out the new org-ref, and let me know how it goes!

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter