Sending html emails from org-mode with org-mime

Posted October 29, 2016 at 02:33 PM | categories: email, orgmode | tags:

On the org-mode mailing list there was some discussion about sending html mail using orgmode. The support for this in mu4e is deprecated. There is the org-mime library though, and it supports a lot of what is needed for this. As I played around with it though, I came across some limitations:

  1. I found equations were not rendered as images in the html, and files (in links) were not attached out of the box. I fixed that here.
  2. I found it useful to modify the org-mime commands to leave point in the To: field when composing emails from org-buffers.
  3. For use with mu4e, I created a function to open a message in org-mu4e-compose-org-mode, and added a C-cC-c hook to allow me to send it easily (here).

This post documents some work I did figuring out how to send html mails. After some testing, some of these should probably be patched in org-mime.

First, you need to require this library. 

(require 'org-mime)

You can send a whole org buffer in html like with this command: org-mime-org-buffer-htmlize. Not all of the internal links work for me (at least in gmail).

The default behavior leaves you at the end of the buffer, which is not too nice. We lightly modify the function here to leave in the To: field. 

(defun org-mime-org-buffer-htmlize ()
  "Create an email buffer containing the current org-mode file
  exported to html and encoded in both html and in org formats as
  mime alternatives."
  (interactive)
  (org-mime-send-buffer 'html)
  (message-goto-to))

1 From an org-headline in an org-file

You can compose an email as an org-heading in any org-buffer, and send it as html. In an org-heading, you need to specify a MAIL_FMT property of html, e.g.: 

   :PROPERTIES:
   :MAIL_FMT: html
   :END:

Note the following properties can also be set to modify the composed email. 

           (subject (or (funcall mp "MAIL_SUBJECT") (nth 4 (org-heading-components))))
           (to (funcall mp "MAIL_TO"))
           (cc (funcall mp "MAIL_CC"))
           (bcc (funcall mp "MAIL_BCC"))

Then, send it with org-mime-subtree

Here I modify this function to leave me in the To: field. 

(defun org-mime-subtree ()
  "Create an email buffer containing the current org-mode subtree
  exported to a org format or to the format specified by the
  MAIL_FMT property of the subtree."
  (interactive)
  (org-mime-send-subtree
   (or (org-entry-get nil "MAIL_FMT" org-mime-use-property-inheritance) 'org))
  (message-goto-to))

Here are some sample elements to see if they convert to html reasonably.

1.1 Markup

bold

underlined

italics

strikethrough

code

Subscripts: H2O Superscripts: H+ An entity: To ∞ and beyond

1.2 Equations

\(x^2\)

\[x^4\]

\(e^x\)

1.3 Tables

Table 1: A table for you.
x y
1 2

1.4 Lists

A nested list.

  • one
    • Subentry under one.
  • two

A definition list:

def1
first definition

A checklist:

  • [ ] A checkbox

Here is a numbered list:

  1. number 1
  2. number 2

1.5 Code block

import numpy as np
import matplotlib.pyplot as plt

t = np.linspace(0, 10)
x = np.cos(t) * np.exp(-t)
y = np.sin(t) * np.exp(-t)

plt.plot(x, y)
plt.savefig('spiral.png')

1.6 An image from somewhere other than this directory

2 In a mail message

You might prefer to do this directly in an email. Here is how you can do it in mu4e. I use this command to open a message in org-mode. The mode switches if you are in the header, or in the body. If you always do this, you could use a hook instead on message-mode. I do not want default html so I do not do it. 

(defun mu4e-compose-org-mail ()
 (interactive)
 (mu4e-compose-new)
 (org-mu4e-compose-org-mode))

For sending, we will use org-mime to htmlize it, and add a C-c C-c hook function to send it. This hook is a little tricky, we want to preserve C-c C-c behavior in org-mode, e.g. in code blocks, but send it if there is no other C-c C-c action that makes sense, so we add it to the end of the hook. Alternatively, you could bind a special key for it, or run the special command. Note the C-c C-c hook only works in the body of the email. From the header, a plain text message is sent. 

(defun htmlize-and-send ()
  "When in an org-mu4e-compose-org-mode message, htmlize and send it."
  (interactive)
  (when (member 'org~mu4e-mime-switch-headers-or-body post-command-hook)
    (org-mime-htmlize) 
    (message-send-and-exit)))

(add-hook 'org-ctrl-c-ctrl-c-hook 'htmlize-and-send t)

Here is a way to do this for non-mu4e users. It doesn't have the nice mode switching capability though, so you lose completion in emails, and header specific functions. You can switch back to message-mode to regain those. 

(defun compose-html-org ()
  (interactive)
  (compose-mail)
  (message-goto-body)
  (setq *compose-html-org* t)
  (org-mode))

(defun org-htmlize-and-send ()
  "When in an org-mu4e-compose-org-mode message, htmlize and send it."
  (interactive)
  
  (when *compose-html-org*
    (setq *compose-html-org* nil)
    (message-mode)
    (org-mime-htmlize) 
    (message-send-and-exit)))

(add-hook 'org-ctrl-c-ctrl-c-hook 'org-htmlize-and-send t)

3 Equations and file attachments do not seem to work out of the box

\(e^{i\pi} - 1 = 0\)

Out of the box, org-mime does not seem to attach file links to emails or make images for equations..

html-email.org

Here is an adaptation of org-mime-compose that does that for html messages. 

(defun org-mime-compose (body fmt file &optional to subject headers)
  (require 'message)
  (let ((bhook
         (lambda (body fmt)
           (let ((hook (intern (concat "org-mime-pre-"
                                       (symbol-name fmt)
                                       "-hook"))))
             (if (> (eval `(length ,hook)) 0)
                 (with-temp-buffer
                   (insert body)
                   (goto-char (point-min))
                   (eval `(run-hooks ',hook))
                   (buffer-string))
               body))))
        (fmt (if (symbolp fmt) fmt (intern fmt)))
        (files (org-element-map (org-element-parse-buffer) 'link
                 (lambda (link)
                   (when (string= (org-element-property :type link) "file")
                     (file-truename (org-element-property :path link)))))))
    (compose-mail to subject headers nil)
    (message-goto-body)
    (cond
     ((eq fmt 'org)
      (require 'ox-org)
      (insert (org-export-string-as
               (org-babel-trim (funcall bhook body 'org)) 'org t)))
     ((eq fmt 'ascii)
      (require 'ox-ascii)
      (insert (org-export-string-as
               (concat "#+Title:\n" (funcall bhook body 'ascii)) 'ascii t)))
     ((or (eq fmt 'html) (eq fmt 'html-ascii))
      (require 'ox-ascii)
      (require 'ox-org)
      (let* ((org-link-file-path-type 'absolute)
             ;; we probably don't want to export a huge style file
             (org-export-htmlize-output-type 'inline-css)
             (org-html-with-latex 'dvipng)
             (html-and-images
              (org-mime-replace-images
               (org-export-string-as (funcall bhook body 'html) 'html t)))
             (images (cdr html-and-images))
             (html (org-mime-apply-html-hook (car html-and-images))))
        (insert (org-mime-multipart
                 (org-export-string-as
                  (org-babel-trim
                   (funcall bhook body (if (eq fmt 'html) 'org 'ascii)))
                  (if (eq fmt 'html) 'org 'ascii) t)
                 html)
                (mapconcat 'identity images "\n")))))
    (mapc #'mml-attach-file files)))

4 Summary

This makes it pretty nice to send rich-formatted html text to people.

Copyright (C) 2016 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.3.5

Discuss on Twitter

Checking for email attachments before you send email

Posted September 04, 2015 at 07:03 PM | categories: email, emacs | tags:

Updated September 04, 2015 at 07:04 PM

Does this comic (http://www.phdcomics.com/comics/archive.php?comicid=1817 ) apply to you? Do you miss the Gmail feature that will alert you that it seems like you mention an attachment but there isn't one attached before it lets you send it? Let's make Emacs help us here. We will scan our emails for the word "attach", and if we see it, scan the email for evidence of an attachment. Then create a hook function that will prompt us if it appears we mention an attachment, but don't have one.

An attachment looks like this in my messages: 

<#part type="image/png" filename="~/Desktop/wordcloud.png" disposition=attachment>
<#/part>

So, probably finding "<#part" in the buffer means I probably have an attachment. We will use the message-send-hook to run this function. Here is the code. Some brief testing from me seems to work fine! It is pretty simple, but probably good enough to save me from sending messages with no attachment, and not too intrusive for when no attachment is actually needed, e.g. in replies. Let me know if you have ideas for improvements. 

(defun email-says-attach-p ()
  "Return t if email suggests there could be an attachment."
  (save-excursion
    (goto-char (point-min))
    (re-search-forward "attach" nil t)))

(defun email-has-attachment-p ()
  "Return t if the currently open email has an attachment"
  (save-excursion
    (goto-char (point-min))
    (re-search-forward "<#part" nil t)))

(defun email-pre-send-check-attachment ()
  (when (and (email-says-attach-p)
             (not (email-has-attachment-p)))
    (unless
        (y-or-n-p "Your email suggests you need an attachment, but no attachment was found. Send anyway?")
      (error "It seems an attachment is needed, but none was found. Aborting send."))))

(add-hook 'message-send-hook 'email-pre-send-check-attachment)

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Clickable email addresses in emacs

Posted June 21, 2015 at 04:42 PM | categories: email, emacs | tags:

There are clickable mailto:jkitchin@andrew.cmu.edu links in org-mode, but standalone email addresses like jkitchin@cmu.edu are just ordinary text. Here, I want to explore using clickable text instead. I will use the button-lock package for this. I borrowed an email regexp from EmacsWiki: Regular Expression for this. The idea is to define a regular expression for an email address, and use it to make the emails clickable.

I am still not sure what the canonical way to get the value of the text that was highlighted. Here we use the :additional-property feature to set a property to t, and then use that property to get the characters that have a non-nil "email-address" property. It seems clunky, but it works. The main action is to compose an email in mu4e (my preferred email program in emacs). You could also put a call to helm here, or to a hydra for other options.

I make the email addresses stand out a little by giving them a gray background, and a tooltip so you can see why they are highlighted. I also bind RET so I don't have to use the mouse. Don't forget you can type C-h . to see the local help instead of mousing over it! Finally, we add a text-mode hook so this will get loaded when we open a text file (or one with a mode derived from text-mode like org-mode). 

(defun highlight-email-addresses ()
  "Add button to email address. Clicking or RET will open a compose email window."
  (button-lock-set-button
   "\\w+\\(\\.\\w+\\)?@\\(\\w\\|\\.\\)+"
   (lambda ()
     (interactive)
     (let ((start) (end) (email-address))
       (while (get-text-property (point) 'email-address)
         (backward-char))
       (forward-char)
       (setq start (point))
       (while (get-text-property (point) 'email-address)
         (forward-char))
       (setq end (point))
       (setq email-address (buffer-substring start end))
       (mu4e~compose-mail email-address)))
     :face '((:background "gray80") (:underline t))
     :help-echo "click to send mu4e email"
     :keyboard-binding (kbd "RET")
     :additional-property 'email-address))

(add-hook 'text-mode-hook 'highlight-email-addresses)

That doesn't look too bad. Now, anytime I open an org-mode file with an email address in it, the address is highlighted in light gray, and underlined. I can click on it or put the cursor on it and press return and I get a compose email window open, with the email address pre-filled in! I am sure this will have some other applications.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

A helm-mu4e contact selector

Posted March 14, 2015 at 10:21 AM | categories: email, helm, emacs | tags:

I have been using mu4e in Emacs for email for about three months now. It is pretty good, and I hardly ever use the gmail web interface any more. The email completion in mu4e is ok, but I am frequently surprised at what it does not find, and totally spoiled by how good Gmail is at this. The built in completion seems to get lost if you don't start the search with the first few letters. Not always, but too often for me. I don't always remember the first letters, and want to search by name, or company. I would love to search by tags in org-contacts. This should be simple in helm, where you can build up candidates with different bits of information. Here I explore a helm interface, which I think might be better than the built in mu4e support, and even be better than gmail.

In my dream email completer, I want some easy way to define my own groups, I want to use org-contacts (and its tags), and I want every email address in the mails I have in my archive as completion candidates. helm supports multiple sources, so I initially tried a separate source for each of these. Preliminary efforts suggested it is not possible to mark multiple selections from different sources and pass them all to one function. So, we combine all email candidates into one list of (searchable-string . email-address) cons cells. To get an idea of how many contacts we are looking at:

Here is what I have in my org-contacts file: 

(length (org-contacts-db))

173

And here is what mu4e knows about. Interestingly, it takes a while for this variable to get populated because the request is asynchronous. After the first time though it sticks around. I think just opening mu4e will populate this variable. 

(length mu4e~contacts-for-completion)

12717

So, I have close to 13,000 potential email addresses to choose from. For my email groups, I will just use a list of cons cells like (group-name . "comma-separated emails"). Then, I will loop through the org-contacts-db and the mu4e completion list to make the helm candidates. Finally, we add some functions to open our org-contact, and to tag org-contacts so it is easier to make groups.

Here is the code I have been using. 

;; here we set aliases for groups.
(setq email-groups
      '(("ms" . "email1, email2")
        ("phd" . "email3, email4")))


(defun org-contacts-open-from-email (email)
  "Open org-contact with matching EMAIL. If no match, create new
entry with prompts for first and last name."
  (let ((contact (catch 'contact
                   (loop for contact in  (org-contacts-db)
                         do
                         (when (string= email (cdr (assoc "EMAIL" (elt contact 2))))
                           (throw 'contact contact))))))

    (unless contact
                (set-buffer (find-file-noselect (ido-completing-read
                                                 "Select org-contact file: "
                                                 org-contacts-files)))
                (goto-char (point-max))
                (insert (format  "\n* %s %s\n"
                                 (read-input "First name: ")
                                 (read-input "Last name: ")))
                (org-entry-put (point) "EMAIL" email)
                (save-buffer))

    (when contact
      (find-file  (cdr (assoc "FILE" (elt contact 2))))
      (goto-char (elt contact 1))
      (show-subtree))))


(defun org-contacts-tag-selection (selection)
  "Prompts you for a tag, and tags each entry in org-contacts
that has a matching email in `helm-marked-candidates'. Ignore
emails that are not in an org-contact file. I am not sure what
the best thing to do there is. Probably prompt for a file, and
add an entry to the end of it."
  (save-excursion
    (let ((tag (read-input "Tag: ")))
      (loop for email in (helm-marked-candidates)
            do
            (let ((contact (catch 'contact
                             (loop for contact in  (org-contacts-db)
                                   do
                                   (when (string=
                                          email
                                          (cdr (assoc
                                                "EMAIL"
                                                (elt contact 2))))
                                     (throw 'contact contact))))))
              ;; add new contact and tag it
              (unless contact
                (set-buffer (find-file-noselect (ido-completing-read
                                                 "Select org-contact file: "
                                                 org-contacts-files)))
                (goto-char (point-max))
                (insert (format  "\n* %s %s\n"
                                 (read-input "First name: ")
                                 (read-input "Last name: ")))
                (org-entry-put (point) "EMAIL" email)
                (org-set-tags-to (list tag))
                (save-buffer))
              ;; update tags on existing entry
              (when contact
                (find-file-noselect  (cdr (assoc "FILE" (elt contact 2))))
                (set-buffer (marker-buffer (elt contact 1)))
                (goto-char (elt contact 1))
                (org-set-tags-to (append (org-get-tags) (list tag)))))))))


(defun j-insert-emails ()
  "Helm interface to email addresses"
  (interactive)

  (helm :sources `(((name . "Email address candidates")
                   (candidates . ,(append
                                   ;; my aliases
                                   email-groups
                                   ;; org-contacts
                                   (loop for contact in (org-contacts-db)
                                         collect
                                         (cons (format
                                                "%s %s %s <%s> org-contact"
                                                (cdr (assoc "FIRSTNAME" (elt contact 2)))
                                                (cdr (assoc "LASTNAME" (elt contact 2)))
                                                (cdr (assoc "TAGS" (elt contact 2)))
                                                (cdr (assoc "EMAIL" (elt contact 2))))
                                               (cdr (assoc "EMAIL" (elt contact 2)))))
                                   ;; mu contacts
                                   (loop for contact in mu4e~contacts-for-completion
                                         collect (cons contact contact))))
                   ;; only action is to insert string at point.
                   (action . (("insert" . (lambda (x)
                                            (insert
                                             (mapconcat
                                              'identity
                                              (helm-marked-candidates)
                                              ","))))
                              ("open" . org-contacts-open-from-email)
                              ("tag"  . org-contacts-tag-selection)))))))

;; Finally, let us bind this to something probably convenient. I use c-c ] for
;; citations. Lets try that in compose mode.
(define-key mu4e-compose-mode-map "\C-c]" 'j-insert-emails)

j-insert-emails

Now, I have a sweet helm interface with nearly 13,000 email candidates (there is a decent amount of duplication in this list, and some garbage emails from spam, but helm is so fast, this does not bother me). I can pretty quickly narrow to any tagged set of emails from org-contacts with a search that looks like :phd: for example, or [^phd]:group: to get org-contacts tagged group, but not phd. I can narrow the selection on first name, lastname, parts of email addresses, tags in org-contacts, etc… I can open a contact, or tag contacts, even add new contacts to org-contacts. I have been using this for a few weeks, and so far I like it. Occasionally I find mu4e~contacts-for-completion is empty, and then I only get my org-contacts emails, but that seems to only happen when I first open emacs. Since Emacs is usually open for days at a time, this has not been an issue very often.

Copyright (C) 2015 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.10

Discuss on Twitter

Better integration of org-mode and email

Posted June 08, 2014 at 08:57 PM | categories: email, org-mode | tags:

I like to email org-mode headings and content to people. It would be nice to have some records of when a heading was sent, and to whom. We store this information in a heading. It is pretty easy to write a simple function that emails a selected region. 

(defun email-region (start end)
  "Send region as the body of an email."
  (interactive "r")
  (let ((content (buffer-substring start end)))
    (compose-mail)
    (message-goto-body)
    (insert content)
    (message-goto-to)))

that function is not glamorous, and you still have to fill in the email fields, and unless you use gnus and org-contacts, the only record keeping is through the email provider.

What I would like is to send a whole heading in an email. The headline should be the subject, and if there are TO, CC or BCC properties, those should be used. If there is no TO, then I want to grab the TO from the email after you enter it and store it as a property. You should be able to set OTHER-HEADERS as a property (this is just for fun. There is no practical reason for this yet). After you send the email, it should record in the heading when it was sent.

It turned out that is a relatively tall order. While it is easy to setup the email if you have everything in place, it is tricky to get the information on TO and the time sent after the email is sent. Past lispers had a lot of ideas to make this possible, and a day of digging got me to the answer. You can specify some "action" functions that get called at various times, e.g. after sending, and a return action when the compose window is done. Unfortunately, I could not figure out any way to do things except to communicate through some global variables.

So here is the code that lets me send org-headings, with the TO, CC, BCC properties, and that records when I sent the email after it is sent. 

(defvar *email-heading-point* nil
  "global variable to store point in for returning")

(defvar *email-to-addresses* nil
  "global variable to store to address in email")

(defun email-heading-return ()
  "after returning from compose do this"
  (switch-to-buffer (marker-buffer  *email-heading-point*))
  (goto-char (marker-position  *email-heading-point*))
  (setq *email-heading-point* nil)
  (org-set-property "SENT-ON" (current-time-string))
  ;; reset this incase you added new ones
  (org-set-property "TO" *email-to-addresses*)
  )

(defun email-send-action ()
  "send action for compose-mail"
  (setq *email-to-addresses* (mail-fetch-field "To")))

(defun email-heading ()
  "Send the current org-mode heading as the body of an email, with headline as the subject.

use these properties
TO
OTHER-HEADERS is an alist specifying additional
header fields.  Elements look like (HEADER . VALUE) where both
HEADER and VALUE are strings.

save when it was sent as s SENT property. this is overwritten on
subsequent sends. could save them all in a logbook?
"
  (interactive)
  ; store location.
  (setq *email-heading-point* (set-marker (make-marker) (point)))
  (org-mark-subtree)
  (let ((content (buffer-substring (point) (mark)))
	(TO (org-entry-get (point) "TO" t))
	(CC (org-entry-get (point) "CC" t))
	(BCC (org-entry-get (point) "BCC" t))
	(SUBJECT (nth 4 (org-heading-components)))
	(OTHER-HEADERS (eval (org-entry-get (point) "OTHER-HEADERS")))
	(continue nil)
	(switch-function nil)
	(yank-action nil)
	(send-actions '((email-send-action . nil)))
	(return-action '(email-heading-return)))
    
    (compose-mail TO SUBJECT OTHER-HEADERS continue switch-function yank-action send-actions return-action)
    (message-goto-body)
    (insert content)
    (when CC
      (message-goto-cc)
      (insert CC))
    (when BCC
      (message-goto-bcc)
      (insert BCC))
    (if TO
	(message-goto-body)
      (message-goto-to))       
    ))

This works pretty well for me. Since I normally use this to send tasks to people, it keeps the task organized where I want it, and I can embed an org-id in the email so if the person replies to it telling me the task is done, I can easily navigate to the task to mark it off. Pretty handy.

Copyright (C) 2014 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 8.2.6

Discuss on Twitter
Next Page ยป