Org-mode is a great tool for organizing your todo lists, ideas, and notes. One of the pain points for me, though, is keeping all of my notes in the proper categories.

Ideally, I want these tasks to be sorted.

Normal refiling can be pretty slow. In this post, I'll show how to use the hydra package with simple macros and elisp functions to achieve fast refiling (and jumping) with just one or two keystrokes. The end result (code here) looks like this:

Why use hydras to refile?

Using hydras allow you to quickly repeat commands. Using nested hydras you can make most refile targets available in one or two keystrokes. When you're refiling a bunch of items, this goes from typing

to

How to refile with hydra

Refiling to a specified location

If you look at the documentation for org-refile, you'll see there's an optional argument RFLOC that allows you to programmatically specify a refile location. It doesn't mention what format it should be in, but a quick search brings up a Stack Exchange answer that shows the format and gives the following example of how to use it:

1
2
3
4
5
6

(defun my/refile (file headline &optional arg)
  (let ((pos (save-excursion
               (find-file file)
               (org-find-exact-headline-in-buffer headline))))
    (org-refile arg nil (list headline file nil pos)))
  (switch-to-buffer (current-buffer)))

This is great! We can use this to bind keys to expressions like (my/refile "someday-maybe.org" "Someday/Maybe"). One problem with this is that we'd have to keep pressing the hotkeys over and over again to refile a bunch of tasks in a row.

Using Hydra

So let's use a hydra to make refiling much faster.

1
2
3
4
5
6
7
8
9
10
11

(defhydra josh/org-refile-hydra (:foreign-keys run) 
  "Refile"
  ("g" (my/refile "shopping.org" "Grocery store") "Refile to Grocery store")
  ("o" (my/refile "shopping.org" "Office supplies") "Refile to Office supplies")
  ("e" (my/refile "tasks.org" "Email tasks") "Email tasks")
  ("r" (my/refile "tasks.org" "Research tasks") "Research tasks")
  ("j" org-refile-goto-last-stored "Jump to last refile")
  ("q" nil "cancel"))
;; Or whatever you want your keybinding to be
(global-set-key (kbd "<f9> r") 'josh/org-refile-hydra/body)

For each keybinding, we provide:

1. The key (e.g. "g")
2. The command which refiles to a specific location
3. A hint, which describes what the key does (e.g. "Refile to Grocery store")

I also use the :foreign-keys key to continue running the hydra, so I can move to a headline using C-n and C-p without exiting the hydra and having to call it again.

With just these commands, you can already begin to refile quickly.

Nested hydras

But what happens when you have a lot of headlines to refile to? If you keep adding headlines, you might get a giant hydra like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

(defhydra josh/org-refile-hydra (:foreign-keys run) 
  "Refile"
  ("g" (my/refile "shopping.org" "Grocery store") "Grocery store")
  ("o" (my/refile "shopping.org" "Office supplies") "Office supplies")
  ("a" (my/refile "shopping.org" "Buy on Amazon") "Buy on Amazon")
  ("e" (my/refile "tasks.org" "Email tasks") "Email tasks")
  ("r" (my/refile "tasks.org" "Research tasks") "Research tasks")
  ("c" (my/refile "tasks.org" "Calendar") "Calendar")
  ("p" (my/refile "tasks.org" "Projects") "Projects")
  ("s" (my/refile "someday-maybe.org" "Someday/Maybe") "Someday/Maybe")
  ("b" (my/refile "media.org" "Books to read") "Books to read")
  ("m" (my/refile "media.org" "Movies to watch") "Movies to watch")
  ("E" (my/refile "ideas.org" "Emacs ideas") "Emacs ideas")
  ("J" (my/refile "ideas.org" "Jokes") "Jokes")
  ("j" org-refile-goto-last-stored "Jump to last refile")
  ("q" nil "cancel"))

You might find that you run out of keybindings fairly quickly. Here we had to start using uppercase letters because lowercase keys were overlapping.

By using nested hydras we can break refiling into two keystrokes: The first for the file and the second for the headline. This moves us from using 26 keys to $26^2 = 676$ possible locations to refile to (and if you use uppercase it's 2704!).

Here's how you can do a simple nested hydra, for refiling into three files (File A, B, and C), each with two headlines (roughly named "1" and "2").

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

(defhydra josh/org-refile-hydra-file-a
 (:color blue :after-exit (josh/org-refile-hydra/body))
 "File A"
 ("1" (my/refile "file-a.org" "Headline 1") "Headline 1")
 ("2" (my/refile "file-a.org" "Headline 2") "Headline 2")
 ("q" nil "cancel"))
(defhydra josh/org-refile-hydra-file-b
 (:color blue :after-exit (josh/org-refile-hydra/body))
 "File B"
 ("1" (my/refile "file-b.org" "One") "One")
 ("2" (my/refile "file-b.org" "Two") "Two")
 ("q" nil "cancel"))
(defhydra josh/org-refile-hydra-file-c
 (:color blue :after-exit (josh/org-refile-hydra/body))
 "File C"
 ("1" (my/refile "file-c.org" "1") "1")
 ("2" (my/refile "file-c.org" "2") "2")
 ("q" nil "cancel"))
(defhydra josh/org-refile-hydra (:foreign-keys run) 
  "Refile"
  ("a" josh/org-refile-hydra-file-a/body "File A" :exit t)
  ("b" josh/org-refile-hydra-file-b/body "File B" :exit t)
  ("c" josh/org-refile-hydra-file-c/body "File C" :exit t)
  ("q" nil "cancel"))

I find the easiest way to construct a nested hydra is to use the :after-exit keyword to pop back to the parent hydra.

We can see that refiling again is pretty straightforward.

Using macros to make hydras

One problem with writing hydras this way is that it gets ugly pretty quickly, with a lot of redundant code. Notice how we repeatedly have to specify the file in each file's hydra, even though it doesn't change within it. Also note that the hydra hint is the same as the headline.

We can write an Elisp macro, to do the heavy lifting in creating our hydras, so we only have to specify a file once:

1
2
3
4
5
6
7
8

(defmacro josh/make-org-refile-hydra (hydraname file keyandheadline)
  "Make a hydra named HYDRANAME with refile targets to FILE.
KEYANDHEADLINE should be a list of cons cells of the form (\"key\" . \"headline\")"
  `(defhydra ,hydraname (:color blue :after-exit (josh/org-refile-hydra/body))
     ,file
     ,@(cl-loop for kv in keyandheadline
		collect (list (car kv) (list 'my/refile file (cdr kv)) (cdr kv)))
     ("q" nil "cancel")))

This makes defining our file hydras way simpler. Now our previous code simplifies to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

(josh/make-org-refile-hydra josh/org-refile-hydra-file-a
			    "file-a.org"
			    (("1" . "Headline 1")
			     ("2" . "Headline 2")))
(josh/make-org-refile-hydra josh/org-refile-hydra-file-b
			    "file-b.org"
			    (("1" . "One")
			     ("2" . "Two")))
(josh/make-org-refile-hydra josh/org-refile-hydra-file-c
			    "file-c.org"
			    (("1" . "1")
			     ("2" . "2")))
(defhydra josh/org-refile-hydra (:foreign-keys run) 
  "Refile"
  ("a" josh/org-refile-hydra-file-a/body "File A" :exit t)
  ("b" josh/org-refile-hydra-file-b/body "File B" :exit t)
  ("c" josh/org-refile-hydra-file-c/body "File C" :exit t)
  ("q" nil "cancel"))

Great! Now our code is a bit cleaner, and adding new refile targets is as easy as adding a ("Key" . "Headline") pair to a list.

Extra goodies

Using the previous code allows you to do most of your refiling tasks quickly and easily. I've made some modifications to the code to have the following features.

1. Be able to jump (without refiling) to a headline by using a C-u prefix
   • And quit the hydra if we jump, so we don't accidentally refile things on the next key press.
2. Be able to refile while you're using org-capture
3. Be able to refile in org-agenda
4. Deal with some bugs:
   • Don't refile when a headline doesn't exist
   • Use with-current-buffer so we don't accidentally switch to the buffer we're refiling to

All the code for this is available here, as a gist.

Note: Problems with current methods

Why did I use hydras to refile in the first place? There were a few problems I was having with org-refile. Some of these can be solved with some customizations that I wasn't aware of before.

• Selecting refile targets can be slow

  (length (org-map-entries 1 nil 'agenda)) tells me I have 5208 entries in my agenda files at this time of writing. Even using a narrowing framework like helm, it takes a long time to narrow down to the headline I want. And generally there are specific categories I do most of my refiling to. (Note: You could use the :tag setting in org-refile-targets to limit refile targets to specific headlines. This might work well)

  You can limit the :maxlevel of org-refile-targets, but unless I set it to "1" this doesn't speed things up much since most of my entries are level 1 or 2. Also, I like the ability to refile to any location when I need to.

  One option is to set org-refile-cache, so you don't have to generate all refile targets every time you refile.

  You can also use org-refile-target-verify-function to limit locations you refile to, like "non-DONE tasks", "only projects", "headlines with children", etc. This can become slow as the function needs to be run on all possible candidates. (But in combination with org-refile-cache, this can be less painful.)

• Refiling many headlines can be really slow

  If you follow Bernt Hansen's org-mode guide, you might have a "refile.org" capture bucket of things to refile later. For me, sequential headlines aren't necessarily related and can be refiled to many places. Picking places, again, takes several keystrokes, which is kind of annoying.

• Why not just use org-capture to directly refile to headlines?

  Normally, when using org-capture, I actually do capture directly to the headline I want. A problem with this, though, is that I need to have capture templates not just for each headline I want to capture to, but for each style/template I'd like to capture. I'd like to have different templates for things like "events", "events mentioned in emails" (so org-capture can automatically add the email link), "emails I need to reply to" and so forth.

  You can refile from org-capture, but again this runs into my first problem.

• Why not batch refile with org-agenda?

  I do this sometimes too. However, sometimes getting things I want to refile into the agenda in the first place is really cumbersome, and I'd like to navigate the tree structure of org mode (rather than flattening things with org-agenda).

I still use normal org-refile somewhat frequently, but refiling and jumping using hydras saves me a fair amount of keystrokes

Happy leap day!

To celebrate this leap day, I just released my first Emacs package called ace-mc which allows you to quickly and easily add as well as remove multiple-cursors mode cursors using ace-jump-mode.

It's available on MELPA now! So installing it is as easy as M-x package-install RET ace-mc

Documentation is available on the GitHub page, but here are a couple screencasts:

The main reason I made this package is because adding cursors with mc/mark-next-like-this or mc/mark-all-like-this-dwim doesn't work super well if you have a lot of potential matches. For example, if I'm trying to rename a variable "i", there's often a bunch of i's in other words that I don't want to touch. While multiple-cursors does make it possible to add multiple cursors using the mouse, this often is a bit of a hassle for me.

It seems like some people are already using it. In the time it's been out, one person already reported a bug (which is now fixed) and a couple people have already asked if I'll be adding avy support.

I eventually plan to add avy support. I first off used ace-jump-mode as it's the package I use currently. I understand, though, that many people have switched to avy. I've already messed around with adding a "add cursor" action to avy-dispatch-alist. The tricky, thing, though is how multiple-cursors deals with read-prompts. And since avy provides many types of jumping styles in separate commands, I'm not sure how best to add ace-mc support for all of them. But as I keep playing around with avy, I plan to finally switch to it.

Anyways, give ace-mc a try, and if you have any improvements or suggestions, let me know!

For a graduate neuronal networks class I took last semester at NYU, I had a project of trying to implement some kind of neuronal network. Given my interest in clustering, I sought to replicate Martí and Rinzel's (2013) feature categorization/clustering network.

Here I show an in-browser demo of their clustering network, as well as a small modification I made to it for anomaly detection. It's kind of fun to play around with (click here to jump directly to it). Most of the parameters in Martí and Rinzel's paper can be modified here to see what effects they have.

The network is an example of a continuous attractor network, which is to say that instead of imagining discrete cells connected to each other, you have an infinite number of cells connected to each other. In some sense, the math becomes easier because you use an integral instead of a sum. However, in this JavaScript demo, I use discrete cells (which can be increased to however many you like, it just makes the simulation slower).

Cells in this network are connected in a ring, and you can imagine each cell as being sensitive to a particular orientation in a line, just like certain visual neurons respond to particular orientations. What this network is going to try to cluster is different orientations that are input into it over time. For example, if the network sees a bunch of nearly flat lines (around $\theta = 0$) over a short period of time, the cells around $\theta = 0$ will begin to maintain a high firing rate. If only a few flat lines are presented though, the cells will forget it and not form a "cluster".

Nearby cells in the ring are connected to each other. With neighboring cells tending to excite each other, slightly further cells tending to inhibit each other, and even further cells having almost no influence. The general shape of the connectivity kernel is what's called a Mexican hat (here approximated with a difference of Gaussians). The general form of this kernel is

\begin{align} J(\theta) &= J_E(\theta) - J_I(\theta)\\ &= j_E \frac{\exp\left(m_E \cos(2\theta)\right)}{I_0(m_E)} - j_I \frac{\exp\left(m_I \cos(2\theta)\right)}{I_0(m_I)} \end{align}

where "$E$" is for excitation and "$I$" inhibition, with $m$ changing the narrowness and $j$ the height of the Mexican hat. "$\theta$" here is the angle difference between neighboring cells. You can mess around with the parameters in the "Kernel" section below to see the effect of different values.

There are two variables that we simulate over time, $s(\theta, t)$, which is the synaptic activation, and $r(\theta, t)$, which represents cell firing. For anomaly detection, I add a third variable $y(\theta, t)$. \begin{align} \tau \frac{\partial}{\partial t}s(\theta, t) &= -s(\theta, t) + r(\theta, t)\\ r(\theta, t) &= \Phi \left[ \frac{1}{\pi} \int_{-\pi/2}^{\pi/2} J(\theta - \theta')s(\theta', t) d\theta' + I(\theta, t)\right]\\ y(\theta, t) &= \Phi(I(\theta, t)) * (1 - s(\theta, t)) \end{align}

Here, $\Phi(x)$ is a sigmoid function for converting a current to a firing rate (between 0 and 1). Its equation is \begin{align} \Phi(x) &= \frac{1}{1 + exp(-\beta[x - x_0])} \end{align} and can be modified in the "Current-to-Rate" section.

The third section, "Excitation", describes how cells respond to stimuli. This basically shows how sensitive cells are to nearby orientations. It has two parameters, $m_s$ and $I_s$ that determine narrowness and the amount of current a cell receives to a nearby orientation. You can modify the inputs to the network by clicking directly on the graph to add more input points. Modifying the "Excitation" section changes how wide and strong each click input is.

Other parameters that can be modified include the length and number of time steps, the number of cells simulated in the network, and the time constant $\tau$.

On the x-axis of the graphs below is time. The y-axis ranges from $-\pi/2$ to $\pi/2$, and represents the orientation of different cells in the network.

This network's fairly interesting. Given the right settings it can detect a number of different clusters of different sizes. With the "anomaly detector" addition, we use the cluster detection to suppress non-novel stimuli. A couple potential problems with this network, though, are that clusters can possibly drift over time and will stay activated indefinitely (which may be problematic for anomaly detection).

Run Stop Clear inputs Example Inputs

I'm a big fan of the rapid serial visualization presentation (RSVP) method of speed-reading. The basic premise is that you present words in the same location in rapid succession, with the idea that your reading speed is largely limited by how quickly you can move your eyes. By quickly presenting words in the same place, you don't have to move your eyes, and can read much more quickly.

I've used a bunch of tools for speed reading in the past. One was "dictator" which is a standalone application. I used others before it, but dictator was the best and really the only one I can remember. I used to use it to read PDFs, but to use it this way I'd have to copy an entire page, paste into dictator, and repeat for each page. This got somewhat annoying.

Later, I found jetzt, which is a Chrome plugin that mimics Spritz, a company with a modified RSVP presentation. jetzt is great for reading webpages. The only problem with it, though, is that it doesn't work on PDFs.

To get around this, we can convert PDFs to HTML files using the fantastic pdf2htmlEX. Installing it on Debian is as easy as sudo apt-get install pdf2htmlex.

Because jetzt is a Chrome plugin, really all the magic happens in a JavaScript and CSS file. By embedding these into our converted HTML file, we can add speed reading capabilities to our PDF!

Here's an example with Alice in Wonderland from Project Gutenberg. Just press "r" on a page to speed read it. It'll look something like the image below. You can speed up or slow down the reading rate using the up and down arrow keys, respectively.

Converting PDFs is fairly straightforward with this bash script (shown below) that automatically adds a link to readPDF.js in the PDF HTML file. It'll use rawgit as a CDN for the following Gist, so you won't even need a local copy of readPDF.js.

I currently use this as a method to skim books and scientific articles. Reading without it now seems super tedious. =P