## Modern use of helm - sortable candidates

| categories: | tags:

helm continues to be my goto completion engine. I was perusing the source for helm-top, and noticed some cool new features, like sorting the candidates in the completion buffer! I also noticed that helm sources are preferably created with some new factory functions (as opposed to the a-lists I used to use). Here I explore some of these and illustrate how to make a sortable helm source.

First, we need a function to give us some candidates we will select from. I will use a function that returns a list of cons cells from a variable containing some data where each element in the data is a plist containing a number and key. I list strings as the number and key so we can see what sorting does later. The data is just a list of plists containing a "number" and a key that is a string. We will create a helm function with these as candidates, and an ability to sort them in ascending/descending order on either the number or key.

(setq h-data '((:num 1 :key "apple")
(:num 9 :key "berry")
(:num 2 :key "cactus")
(:num 5 :key "dog")
(:num 4 :key "frog")))

(defun h-candidates ()
"Returns candidates for the helm source."
(loop for cand in h-data
collect (cons (format "%s %s"
(plist-get cand :num)
(plist-get cand :key))
cand)))

(print (h-candidates))

(("1 apple" :num 1 :key "apple") ("9 berry" :num 9 :key "berry") ("2 cactus" :num 2 :key "cactus") ("5 dog" :num 5 :key "dog") ("4 frog" :num 4 :key "frog"))


Now, provide sorting, we need to create a candidate transformer function. This function will take the current candidates and source, and return a new list of candidates, possibly sorted. We use a variable to store how to sort the candidates. We also need a way to trigger the sorting. We will bind M-<down> to a function that will set the sort function, and refresh helm. Here is a keymap definition we will use later.

(defvar h-map
(let ((map (make-sparse-keymap)))
(set-keymap-parent map helm-map)
(define-key map (kbd "M-<down>")   'h-sort)
map)
"keymap for a helm source.")

h-map


Now, we define the sort variable, a function that sets the variable, refreshes the candidates, and finally resets the sort variable. A key point here is the sort functions must take two arguments, which will be two candidates, and each candidate is of the form (string . data). We want to sort on one of the elements in the data plists for this example.

(defvar h-sort-fn nil)

(defun h-sort ()
(interactive)
(let ((action (read-char "#decreasing (d) | #increasing (i) | a-z (a) | z-a (z: ")))
(cond
((eq action ?d)
(setq h-sort-fn (lambda (c1 c2) (> (plist-get (cdr c1) :num) (plist-get (cdr c2) :num)))))
((eq action ?i)
(setq h-sort-fn (lambda (c1 c2) (< (plist-get (cdr c1) :num) (plist-get (cdr c2) :num)))))
((eq action ?a)
(setq h-sort-fn (lambda (c1 c2) (string< (plist-get (cdr c1) :key) (plist-get (cdr c2) :key)))))
((eq action ?z)
(setq h-sort-fn (lambda (c1 c2) (string> (plist-get (cdr c1) :key) (plist-get (cdr c2) :key)))))
(t (setq h-sort-fn nil)))
(helm-refresh)
(setq h-sort-fn nil)))

h-sort


Next, we define a candidate transformer. This function takes the list of candidates and the source. Here, if we have defined a sort function, we use it to sort the candidates, and if not, return the candidates. A subtle point here is the use of -sort from dash.el, which does not modify the original list at all. The build in function sort does modify the candidate list somehow, and it does not work the way you want it to here. This function gets run as the helm pattern changes.

(defun h-candidate-transformer (candidates source)
(if h-sort-fn
(progn (message "Sorting with %s" h-sort-fn)
(-sort h-sort-fn candidates))
candidates))

h-candidate-transformer


Now, just for fun, we show that dynamically defined actions are possible. Here, we generate an action list that is different for even and odd numbers. These actions are pretty trivial, but give you an idea of what might be possible; custom, context specific actions.

;; Make dynamic actions based on the candidate selected
(defun h-action-transformer (actions candidate)
"Candidate is the result selected."
(if (evenp (plist-get candidate :num))
'(("Even" . identity))
'(("Odd" . identity))))

h-action-transformer


Finally, we are ready to create a helm source. We use the new factory function for creating the source with our keymap, candidates and transformer functions.

(setq h-source
(helm-build-sync-source "number-selector"
:keymap h-map
:candidates #'h-candidates
:filtered-candidate-transformer #'h-candidate-transformer
:action-transformer #'h-action-transformer))


Now, you can run the helm source like this.

(helm :sources 'h-source)


You can sort the numbers in descending order by typing M-<down> and pressing d. To get ascending order, press i instead. To sort on the keys, type a sort from a to z, and press z to sort on z to a. If you press tab on a selection, you will see that the actions you get depend on whether the selection is an even or odd number! So, you can get some context specific actions depending on your selection. Pretty awesome.

org-mode source

Org-mode version = 8.2.10

## LDAP lookups from Emacs

| categories: | tags:

Now that I have email and Cisco Jabber totally integrated into Emacs it would be nice to tap into the CMU LDAP (Lightweight Directory Access Protocol) service to find emails and phone numbers. We to use the ldapsearch command-line utility to query our LDAP service like this to find an email address.

You might like the video explanation here: https://www.youtube.com/watch?v=N7AaKHRd9uw

(shell-command-to-string "ldapsearch -x -LLL -h ldap.andrew.cmu.edu -b ou=Person,dc=cmu,dc=edu cn=\"John Kitchin\"")

dn: guid=1976CCAA-B465-11D8-8000-080020CC75D3,ou=person,dc=cmu,dc=edu
objectClass: cmuPerson
eduPersonPrimaryAffiliation: Faculty
guid: 1976CCAA-B465-11D8-8000-080020CC75D3
cmuPrivate: homePhone
cn: John Kitchin
givenName: John
sn: Kitchin
cmuPrimaryCampus: Pittsburgh
cmuCampus: Pittsburgh
cmuAndrewId: jkitchin
cmueduId: jkitchin
cmuAndrewCommonNamespaceId: jkitchin
mail: jkitchin@cmu.edu
eduPersonSchoolCollegeName: CIT - Consolidated
cmuPersonPrincipalName: jkitchin@ANDREW.CMU.EDU
cmuDepartment: Chemical Engineering
cmuDepartment: MSE: Materials Science & Engineering
cmuPersonAffiliation: Tenure-Track Faculty
eduPersonAffiliation: Faculty
cmuAccount: uid=jkitchin,ou=account,dc=andrew,dc=cmu,dc=edu
cmuAccount: uid=jkitchin,ou=account,dc=cmu,dc=edu
cmuActiveDN: uid=jkitchin,ou=account,dc=andrew,dc=cmu,dc=edu
cmuActiveDN: uid=jkitchin,ou=account,dc=cmu,dc=edu
title: Professor
telephoneNumber: +1 412 268 7803


We actually get LDIF data from ldapsearch with a lot of details. Next we wrap the output in a function that converts each result from ldapsearch into a p-list that we will use later in a helm function to help us select a match.

(defun ldap-query (query-string)
"Send QUERY-STRING to our ldap server and parse results into a
list of p-lists for each entry returned."
(interactive "sLDAP query: ")
(let ((output (butlast (split-string
(shell-command-to-string
(format (concat  "ldapsearch -x -LLL "
"-h ldap.andrew.cmu.edu "
"-b ou=Person,dc=cmu,dc=edu %s")
query-string))
"\n")))
(lines '())
(result '())
(results '(())))
;; cleanup trailing lines and ignore initial lines
(loop for line in output
do
(cond
;; join lines that run over
((s-starts-with? " " line)
(setf (car (last lines))
(concat (car (last lines)) line)))
;; ignore this
((string-match "Size limit exceeded" line)
nil)
(t

;; now we need to parse the lines. A new entry starts with a dn: line.
(dolist (line lines)
(cond
((s-starts-with? "dn:" line)
((string-match ":" line)
(let* ((s (split-string line ":"))
(prop (intern (concat ":" (s-trim (car s)))))
(setf (car results) (plist-put (car results) prop val))))))
;; last result seems to be nil so we drop it
(-filter (lambda (x) (not (null x))) results)))

ldap-query


Here is an example of that function:

(ldap-query "cn=\"John Kitchin\"")

((:dn "dn: guid=1976CCAA-B465-11D8-8000-080020CC75D3,ou=person,dc=cmu,dc=edu" :objectClass "cmuPerson" :eduPersonPrimaryAffiliation "Faculty" :guid "1976CCAA-B465-11D8-8000-080020CC75D3" :cmuPrivate "homePhone" :cn "John Kitchin" :givenName "John" :sn "Kitchin" :cmuPrimaryCampus "Pittsburgh" :cmuCampus "Pittsburgh" :cmuAndrewId "jkitchin" :cmueduId "jkitchin" :cmuAndrewCommonNamespaceId "jkitchin" :mail "jkitchin@cmu.edu" :eduPersonSchoolCollegeName "CIT - Consolidated" :cmuPersonPrincipalName "jkitchin@ANDREW.CMU.EDU" :postalAddress "DH A207F" :cmuDepartment "MSE" :cmuPersonAffiliation "Tenure-Track Faculty" :eduPersonAffiliation "Faculty" :cmuAccount "uid=jkitchin,ou=account,dc=cmu,dc=edu" :cmuActiveDN "uid=jkitchin,ou=account,dc=cmu,dc=edu" :title "Professor" :telephoneNumber "+1 412 268 7803"))


Now, we wrap a helm function around that to give us a nice menu to select entries from, and a few actions like sending an email, calling, copying the name and email, and seeing the information in a reasonable way. We also add a fallback method in case we don't find what we want and need to do a new search.

(defun helm-ldap (query-string)
(interactive "sLDAP query: ")
(helm
:sources
(((name . "HELM ldap")
(candidates . ,(mapcar
(lambda (x)
(cons
(format
"%20s|%30s|%30s|%20s|%s"
(s-truncate
20
(or (plist-get x :title) " "))
(plist-get x :cn)
(plist-get x :mail)
(or (plist-get x :telephoneNumber) " "))
x))
(ldap-query
(if (string-match "=" query-string)
query-string
(concat "cn=*" query-string "*")))))
(action . (("Email" . (lambda (x)
(compose-mail)
(message-goto-to)
(insert (plist-get x :mail))
(message-goto-subject)))
("Call" . (lambda (x)
(cisco-call
(plist-get x :telephoneNumber))))
("Copy Name and email address" . (lambda (x)
(kill-new
(format
"%s <%s>"
(plist-get x :cn)
(plist-get x :mail)))))
("Information" . (lambda (x)
(switch-to-buffer
(get-buffer-create "*helm ldap*"))
(erase-buffer)
(dolist (key (cl-loop
for key in x by #'cddr
collect key))
(insert (format "|%s | %s|\n"
key (plist-get x key))))
(org-mode)
(goto-char 0)
(org-ctrl-c-ctrl-c)
(insert "press q to quit.\n\n")
(use-local-map (copy-keymap org-mode-map))
(local-set-key "q"
#'(lambda ()
(interactive)
(quit-window t))))))))
;; fallback action
((name . "New search")
(dummy)
(action . (lambda (x) (helm-ldap x)))))))

helm-ldap


That is pretty convenient!

John Kitchin <jkitchin@cmu.edu>

org-mode source

Org-mode version = 8.2.10

## Insert org-entities into org-mode with helm

| categories: | tags:

org-mode has a lot of pre-defined entities (see http://kitchingroup.cheme.cmu.edu/blog/2013/10/03/Exporting-accented-characters-to-latex-from-org-mode/ ), otherwise known to me as non-ascii characters. I rarely remember what these are, and occasionally want to insert the LaTeX or HTML code, so here we build a helm command to show them to me, and allow me to select one for insertion. We generate the helm sources from org-entities below. It works pretty well!

(defun helm-insert-org-entity ()
"Helm interface to insert an entity from org-entities'.
F1 inserts utf-8 character
F2 inserts entity code
F3 inserts LaTeX code (does not wrap in math-mode)
F4 inserts HTML code"
(interactive)
(helm :sources (reverse
(let ((sources '())
toplevel
secondlevel)
(dolist (element (append
'("* User" "** User entities")
org-entities-user org-entities))
(when (and (stringp element)
(s-starts-with? "* " element))
(setq toplevel element))
(when (and (stringp element)
(s-starts-with? "** " element))
(setq secondlevel element)
'sources
((name . ,(concat
toplevel
(replace-regexp-in-string
"\\*\\*" " - " secondlevel)))
(candidates . nil)
(action . (("insert utf-8 char" . (lambda (candidate)
(insert (nth 6 candidate))))
("insert org entity" . (lambda (candidate)
(insert (concat "\\" (car candidate)))))
("insert latex" . (lambda (candidate)
(insert (nth 1 candidate))))
("insert html" . (lambda (candidate)
(insert (nth 3 candidate)))))))))
(when (and element (listp element))
(setf (cdr (assoc 'candidates (car sources)))
(append
(cdr (assoc 'candidates (car sources)))
(list (cons
(format "%10s %s" (nth 6 element) element)
element))))))
sources))))

helm-insert-org-entity


Now I can write things like the particle was 60 Å in diameter at a temperature of 600°C, leading to an expansion coefficient of α=0.2 ± 0.01. It isn't quite as fast as knowing the keyboard shortcuts for those symbols, but a lot faster than looking them up then copy and pasting them. So far it seems like these export to HTML and LaTeX just fine, and they are more convenient and better looking than using the org-entities codes. This will make its way into jmax soon.

org-mode source

Org-mode version = 8.2.10

## A helm-mu4e contact selector

| categories: | tags:

I have been using mu4e in Emacs for email for about three months now. It is pretty good, and I hardly ever use the gmail web interface any more. The email completion in mu4e is ok, but I am frequently surprised at what it does not find, and totally spoiled by how good Gmail is at this. The built in completion seems to get lost if you don't start the search with the first few letters. Not always, but too often for me. I don't always remember the first letters, and want to search by name, or company. I would love to search by tags in org-contacts. This should be simple in helm, where you can build up candidates with different bits of information. Here I explore a helm interface, which I think might be better than the built in mu4e support, and even be better than gmail.

In my dream email completer, I want some easy way to define my own groups, I want to use org-contacts (and its tags), and I want every email address in the mails I have in my archive as completion candidates. helm supports multiple sources, so I initially tried a separate source for each of these. Preliminary efforts suggested it is not possible to mark multiple selections from different sources and pass them all to one function. So, we combine all email candidates into one list of (searchable-string . email-address) cons cells. To get an idea of how many contacts we are looking at:

Here is what I have in my org-contacts file:

(length (org-contacts-db))

173


And here is what mu4e knows about. Interestingly, it takes a while for this variable to get populated because the request is asynchronous. After the first time though it sticks around. I think just opening mu4e will populate this variable.

(length mu4e~contacts-for-completion)

12717


So, I have close to 13,000 potential email addresses to choose from. For my email groups, I will just use a list of cons cells like (group-name . "comma-separated emails"). Then, I will loop through the org-contacts-db and the mu4e completion list to make the helm candidates. Finally, we add some functions to open our org-contact, and to tag org-contacts so it is easier to make groups.

Here is the code I have been using.

;; here we set aliases for groups.
(setq email-groups
'(("ms" . "email1, email2")
("phd" . "email3, email4")))

(defun org-contacts-open-from-email (email)
"Open org-contact with matching EMAIL. If no match, create new
entry with prompts for first and last name."
(let ((contact (catch 'contact
(loop for contact in  (org-contacts-db)
do
(when (string= email (cdr (assoc "EMAIL" (elt contact 2))))
(throw 'contact contact))))))

(unless contact
"Select org-contact file: "
org-contacts-files)))
(goto-char (point-max))
(insert (format  "\n* %s %s\n"
(org-entry-put (point) "EMAIL" email)
(save-buffer))

(when contact
(find-file  (cdr (assoc "FILE" (elt contact 2))))
(goto-char (elt contact 1))
(show-subtree))))

(defun org-contacts-tag-selection (selection)
"Prompts you for a tag, and tags each entry in org-contacts
that has a matching email in helm-marked-candidates'. Ignore
emails that are not in an org-contact file. I am not sure what
the best thing to do there is. Probably prompt for a file, and
add an entry to the end of it."
(save-excursion
(loop for email in (helm-marked-candidates)
do
(let ((contact (catch 'contact
(loop for contact in  (org-contacts-db)
do
(when (string=
email
(cdr (assoc
"EMAIL"
(elt contact 2))))
(throw 'contact contact))))))
;; add new contact and tag it
(unless contact
"Select org-contact file: "
org-contacts-files)))
(goto-char (point-max))
(insert (format  "\n* %s %s\n"
(org-entry-put (point) "EMAIL" email)
(org-set-tags-to (list tag))
(save-buffer))
;; update tags on existing entry
(when contact
(find-file-noselect  (cdr (assoc "FILE" (elt contact 2))))
(set-buffer (marker-buffer (elt contact 1)))
(goto-char (elt contact 1))
(org-set-tags-to (append (org-get-tags) (list tag)))))))))

(defun j-insert-emails ()
(interactive)

(helm :sources (((name . "Email address candidates")
(candidates . ,(append
;; my aliases
email-groups
;; org-contacts
(loop for contact in (org-contacts-db)
collect
(cons (format
"%s %s %s <%s> org-contact"
(cdr (assoc "FIRSTNAME" (elt contact 2)))
(cdr (assoc "LASTNAME" (elt contact 2)))
(cdr (assoc "TAGS" (elt contact 2)))
(cdr (assoc "EMAIL" (elt contact 2))))
(cdr (assoc "EMAIL" (elt contact 2)))))
;; mu contacts
(loop for contact in mu4e~contacts-for-completion
collect (cons contact contact))))
;; only action is to insert string at point.
(action . (("insert" . (lambda (x)
(insert
(mapconcat
'identity
(helm-marked-candidates)
","))))
("open" . org-contacts-open-from-email)
("tag"  . org-contacts-tag-selection)))))))

;; Finally, let us bind this to something probably convenient. I use c-c ] for
;; citations. Lets try that in compose mode.
(define-key mu4e-compose-mode-map "\C-c]" 'j-insert-emails)

j-insert-emails


Now, I have a sweet helm interface with nearly 13,000 email candidates (there is a decent amount of duplication in this list, and some garbage emails from spam, but helm is so fast, this does not bother me). I can pretty quickly narrow to any tagged set of emails from org-contacts with a search that looks like :phd: for example, or [^phd]:group: to get org-contacts tagged group, but not phd. I can narrow the selection on first name, lastname, parts of email addresses, tags in org-contacts, etc… I can open a contact, or tag contacts, even add new contacts to org-contacts. I have been using this for a few weeks, and so far I like it. Occasionally I find mu4e~contacts-for-completion is empty, and then I only get my org-contacts emails, but that seems to only happen when I first open emacs. Since Emacs is usually open for days at a time, this has not been an issue very often.

org-mode source

Org-mode version = 8.2.10

## Helm at the Emacs

| categories: | tags:

I have written several (intro , multiple args , prefix args) times about using Helm in Emacs so far. Today, I want to share a way I use helm to get me where I want to be in Emacs for my daily activities. This came out of a desire to have single command that would give me a lot of options to open exactly the buffer/file I wanted when I need it. I call the command hotspots, and it is bound to f9 for me, so when I press f9 I get a helm buffer to select what I want from.

So, what kinds of things do I want. First, I want to be able to open my mail, calendar, News feed or agenda from this command. Second, I have a list of hotspots I developed using the code at http://ergoemacs.org/emacs/emacs_hotkey_open_file_fast.html , which I want easy access to. Third, I want to be able to open any org-file in my agenda list. Fourth, any bookmark I have, or to set a bookmark. Fifth, I want recent files as candidates. There is certainly some redundancy in their, but that is ok, it gets me where I want to be.

Here is the code that does that for me. There are six helm sources that provide candidates and actions.

(defun hotspots ()
"helm interface to my hotspots, which includes my locations,
org-files and bookmarks"
(interactive)
(helm :sources (((name . "Mail and News")
(candidates . (("Mail" . (lambda ()
(progn
(delete-other-windows))

(mu4e))))
("Calendar" . (lambda ()  (browse-url "https://www.google.com/calendar/render")))
("Agenda" . (lambda () (org-agenda "" "w")))))
(action . (("Open" . (lambda (x) (funcall x))))))
((name . "My Locations")
(candidates . (("master" . "~/Dropbox/org-mode/master.org")
(".emacs.d" . "~/Dropbox/kitchingroup/jmax" )
("blog" . "~/blogofile-jkitchin.github.com/_blog/blog.org")
("ese" . "~/Dropbox/books/ese-book/ese.org" )
("Pycse" . "~/Dropbox/books/pycse/pycse.org")
("references" . "~/Dropbox/bibliography/references.bib")
("notes" . "~/Dropbox/bibliography/notes.org")
("journal" . "~/Dropbox/org-mode/journal.org")
(action . (("Open" . (lambda (x) (find-file x))))))

((name . "My org files")
(candidates . ,(f-entries "~/Dropbox/org-mode"))
(action . (("Open" . (lambda (x) (find-file x))))))
helm-source-recentf
helm-source-bookmarks
helm-source-bookmark-set)))
`

Interesting to me is that there are not a lot of actions in here. I mostly use this command for navigation to various places. For example, I press f9, type meet, and I can quickly get to the meetings file in my agenda list, or I can type the first few letters of a student's name and open the org-file associated with them. Or I press f9 and go down an entry to open my calendar, etc… I find this enormously helpful because it opens these files no matter where I am in Emacs, and it relieves my mind from remembering where they are, or the keystrokes/commands to get to them.