Note Management with Vim

I've tried managing my personal knowledge base in many ways over the years, using everything from pencil and paper to Evernote and OneNote to TagSpaces and Joplin (the latter two (can) keep your data locally and are well worth checking out).

The tools have always gotten in the way of my desired organization. Pen and paper is hard to search, and the others all have some combination of hierarchy and tag systems, and it's difficult to use only tags. The hierarchy plus tags required a surprising amount of work to keep in a way that I could find information again easily; I would invariably end up just searching for everything, which begs the question -- why am I even trying to structure these notes? And when limitations in the tool's search feature made things hard to find, I lost time looking for notes.

I began using vimwiki a while back, and it was almost perfect; but now I think I've found something that is working quite well with mostly-vanilla vim (plus some other tools).

What I want:

  • Plain-text notes. Text is greppable.
  • Version control.
  • Throw them all into one folder -- I'll organize by tags.
  • Be able to link notes and browse between them.
  • Store PDFs (academic papers, etc.) and be able to search them too.

Note: the system below will work with any text editor, potentially minus links between files. Vim makes it convenient.

An Overview

Most of what follows is based on or directly comes from Edwin Wenink, with some changes to fit my needs and PDF support.

I place all my notes in a notes directory; ~/notes. The information is important but the files are not, so I don't give them meaningful names. Every file is just a timestamp; e.g., "20200802183404.md". This ensures I don't try to organize anything outside of my established system, and the filename serves solely as a key to the information. I do not recommend going this far without a programmer's text editor.

I use ctags for my programming, so using the regex Edwin Wenink devised (it's always nice to be second) I created a tag format for Markdown. Combining tags with convenient grepping makes the notes easily manageable.

The Vim Configuration

Reading PDFs

I installed pdftoroff and w3m (lynx will work too, but links2 will not). Then my ~/.vim/ftdetect/pdf.vim gets:

autocmd BufRead *.pdf set filetype=pdf

if executable('w3m') && executable('pdftoroff')
    autocmd BufReadCmd *.pdf silent %!pdftoroff -w "%" | w3m -dump -T text/html
endif

The BufReadCmd (and other *Cmd) autocommands expect the specified command to perform the action; in this case, anytime I :edit a PDF document, instead of directly loading the file into vim's buffer, it will execute pdftoroff -w "<filename>" | w3m -dump -T text/html, which uses pdftoroff to convert the PDF to HTML, then w3m renders and dumps it to stdout, which feeds vim's buffer.

Instead of converting the PDF to HTML, you could convert it to roff: pdftoroff "%" | nroff. Using w3m seems to be more flexible regarding line widths, so it looks better with various window sizes.

Creating Notes

I do have one named note: "index.md". This gives me a place to link to important or often-used notes so I don't have to search for them. All other notes get a timestamp. For .vimrc:

nnoremap <Leader>ni :e $HOME/notes/index.md<CR>
nnoremap <Leader>nn
    \ :execute ":e $HOME/notes/" . strftime("%Y%m%d%H%M%S") . ".md"<CR>

strftime above is not guaranteed to be available on every platform; it is on Linux, the BSDs, macOS, and Windows. I use Space as a leader, so with these mappings <space>ni opens the index page and <space>nn creates a new note.

Finding Notes

Tags

I use Universal Ctags and the Gutentags plugin to manage my tags.

I want to be able to create tags like "@vim" and "@blog-post"; Universal Ctags uses the POSIX.2 regex engine in Glibc, which doesn't support some modern regex extensions. Luckily for me, Edwin Wenink already did the work to get this working, using multi-line pattern matching to let us stick multiple tags on a single line (he explains how it works).

Add the following to your g:gutentags_ctags_extra_args array to define the tags (or stick the options in the relevant option file if calling ctags directly):

let g:gutentags_ctags_extra_args = [
    \ '--langdef=mdtags',
    \ '--langmap=mdtags:.md',
    \ '--mline-regex-mdtags=/(^|[[:space:]])@(\w\S*)/\2/t/{mgroup=1}'
\]

I had also had Markdown files in my ctags exclusion list, so had to remove that, and I added "index.md" to my g:gutentags_project_root variable.

Next, I installed the CtrlP plugin to fuzzy search the tags:

if executable('rg')
    let g:ctrlp_user_command = 'rg %s --files --color=never --glob "*"'
else
    let g:ctrlp_user_command = 'grep -r %s'
endif

let g_ctrlp_user_caching = 0
nnoremap <Leader>t :CtrlPTag<CR>

All of my mappings begin with an "n" for "notes" except this CtrlP command, since it's a generic tag search.

I'm using ripgrep when present, grep otherwise. Ripgrep (and Ripgrep-all) is available for Windows; grep is not. Now <space>t opens the CtrlP split to search my tags.

Searching

Now I want to search my notes, including PDF documents. For this I'm using ripgrep-all -- it's ripgrep but also searches additional filetypes, including PDFs.

I'm still playing with size and positioning here -- I'll probably change the vertical split to a horizontal one:

if executable('rga')
    set grepprg=rga\ --vimgrep

    command! -nargs=1 NotesGrep execute 'silent grep! -S "<args>" "$HOME/notes/"'
        \ | redraw! | botright vertical cwindow | vertical resize 45
else
    command! -nargs=1 NotesGrep
        \ execute 'silent grep! -r -i --include="*.md" "<args>" "$HOME/notes/"'
        \ | redraw! | botright vertical cwindow | vertical resize 45
endif

nnoremap <Leader>ns :NotesGrep

Now <space>ns will search the text of all my notes. I do have the -r (recursive) flag on grep even though I'm keeping everything in an unsorted, flat directory. If I ever do add a subdirectory, this will ensure I don't lose anything in it.

Navigating Notes

My fourth bullet point hasn't been addressed ("Be able to link notes and browse between them"), but that's because Vim gives it to me for free. The gf command goes to the file whose name is under the cursor.

For example, in "index.md", I have a section for planning my meals:

## Dinner:

Monday: leftovers
Tuesday: leftovers
Wednesday: 20200801171401.md - Fried Rice
Thursday: leftovers
Friday: Mom + Dad's
Saturday: leftovers
Sunday: 20200726204306.md - Creamy Butter Chicken

I can place my cursor over either <number>.md file, press gf, and I'm at the relevant recipe. This is an advantage to using a flat file structure and a timestamp for the filename -- I don't have to escape spaces, I can add whatever description fits the context of the note I'm in, etc. The timestamps are easy to just skip while reading, so they don't even interrupt my flow (it could potentially be shortened by using a two digit year and not including seconds in the timestamp, but I don't expect there would be any gain in doing so).

Conclusion

This system now stores everything -- recipes, academic papers and my notes on those papers, book reviews and notes, random ideas, research notes, etc. Thus far, it's been pretty easy to navigate and find what I'm looking for. And the fact that I can do this with very little configuration in vim is rather impressive.

The one thing I have found that annoys me concerns tagging; in order to see what the note is about in the CtrlP split I have to place my tags on the header line, though I'd rather have them beneath it:

# I have to do this @vim @kb
# But I'd rather do this

@vim @kb

That's an easy problem to live with.

Tagged productivity, vim, knowledge-management; Posted 4 years ago by rjframe 7 Minute(s) to read