Integrated LLM features in Neovim

I've held out on using LLMs to help me write code, but recently I've found them extremely useful. They're particularly good at writing code for APIs that are simultaneously bad and popular (matplotlib I'm looking at you...). These two qualities create the perfect environment for llms to thrive. The models have no difficulties memorizing the confusing matplotlib API due to the high volume of matplotlib code out there, and no matter how many times I play with matplotlib formatting I don't think I'll ever remember the API.

Matplotlib is a great example because I think it's pretty uncontroversial. Most people agree matplotlib was a great package when it was written, but the art of API design has come a long way since then and it hasn't aged that well. Perhaps a bit more controversial is my take that the lua core library and neovim apis fall in a similar bucket. They're not quite as bad, but I still never seem to remember them - no matter how many plugins I write. Luckily for me, LLMs are actually very effective at writing neovim plugins.

Kind of a funny observation, but at this point I'm writing a neovim plugin by using an LLM, with the goal of making it easier to write neovim plugins using LLMs. I found this funny enough to make it the title of this blog.

I'll be writing this blog post in present tense - as I'm actually writing this plugin while writing the blog.

Desired Workflow

There's a few ways I could see myself using LLMs, but I'd like to start out simple: Chat. Today when I go to create a new neovim plugin, I open up some LLM in my web browser and start talking to it. Then I manually copy and paste the code it provides me with into neovim.

This isn't horrible, but it requires a few application switches which is a bit unergonimic. Ideally, I'd do it all within neovim. Then I could seamlessly navigate back and forth to my chat, my open buffers, paste code where I see fit across files, and yank the code back into the chat if I need to provide the model with updates.

Implementation

Luckily, frankroeder/parrot.nvim provides some pretty reasonable commands for interacting with LLMs. These are all written in vimscript which means they're compatible with vim, but unfortunately that does make it a bit more annoying to interact with from neovim. It still does a lot for us, so I'm going to use it as a foundation and only create custom commands/bindings when necessary.

parrot.nvim exposes a :PrtChatNew command. Luckily, I have <leader>cn unbound, so we can use it to represent [c]hat [n]ew. I'll create a llm.lua file and do this to get started:

vim.keymap.set("n", "<leader>cn", ':PrtChatNew<cr>');

Now I can start an llm chat with just a few keypresses. One thing that quickly became annoying was getting the responses. By default parrot requires you to run :PrtChatResponde to get the response, but I find this pretty tedious. There's two directions I can see myself taking this - one, automatically running this on file write, and two prompting it with the hotkey <leader>cn whenever I'm inside of a parrot chat buffr.

One concern I have is that I'm planning to keep all of the chats locally on disk so I can search through them later with telescope, and I don't want to worry about accidentally double prompting the LLMs with their own replies. As such, I'm going with the hotkey approach for now:

local function isParrotChatBuffer()
  local filename = vim.api.nvim_buf_get_name(0)
  if not filename then
    return false
  end


  local match = string.match(filename, "/chats/")

  if match ~= nil then
    return true
  else
    return false
  end
end

local function cn_binding()
  if isParrotChatBuffer() then
    vim.cmd([[:PrtChatResponde]])
  else
    vim.cmd([[:PrtChatNew]])
  end
end

vim.keymap.set("n", "<leader>cn", cn_binding);

Parrot also has a telescope plugin already builtin, so I'll just setup a keybinding for this now. All of my custom plugins use the pattern <leader>{prefix}s for their fuzzy search command (i.e. <leader>bs for blog search, <leader>ws for wiki search, etc). I'll just set this up now:

vim.keymap.set("n", "<leader>cs", ':PrtChatFinder<cr>');

Let's give this a try and put it all together:

It all works great. Fun fact, the above terminal rendering is actually derivated from the chat demo.

Syncing my chats

I have a dedicated github repo for all of my dot-files and personal wiki entries. I'd like to also sync my parrot chats, so I'm replacing the directory:

~/.share/nvim/parrot/chats

with a symlink:

if ! test -e $HOME/.local/share/nvim/parrot/chats; then
  ln -s $SCRIPT_DIR/chats/ $HOME/.local/share/nvim/parrot/chats
fi

It's a minor touch, but definitely worth it to have consistent Telescope results on my laptop, desktop, and VPSes.

Conclusions & future work

I was expecting to need to write more lua code but these three keybindings actually gave me everything I needed. I'm pretty impressed by parrot.nvim - it was trivial to setup and it works great. The fuzzy finder is a really nice touch.

In the future, it would be really nice to have a set of template prompts and topics that I could pull into my chats. For example, I might want a template explaining the structure of lukewood.xyz, or the way I structure my graphics programs, or even just some context about neovim. A telescope searcher that goes through these templates and prepopulates the chat would be pretty nice. Tat's probably the next thing I'll tackle, but we'll save that for another day.