Treesitter polling instead of parsing on each snippet trigger

[reptee]

Motivation

While attending lectures(only math) and taking notes in markdown everything starts OK.

Side note: I always start with an empty markdown file.

Neovim does not feel slow, everything expands in time and general experience is
good.

Then, closer to the middle of a lecture(~100 lines of markdown written) some snippets won't expand( and expanding takes a good amount of time, at some point it becomes unbearable.

Strangely, executing :e helps but not for long, soon enough problem repeats
and at some point, if the lecture is long enough, neovim is slow even after that
procedure.

I am sure that this is mostly due to snippets because without them neovim is
responsive when editing bigger markdown files. My skills in profiling lack so
It was decided to first write a polling mechanism before benchmarking.

Benchmarking

Two benchmarks were done for each case: with and without polling

To benchmark I used luajit's profiler
(which tbh doesn't say much in the end), a minimal neovim configuration and two tests:

1. enter a big enough markdown file, go to the very end, then type mk and hold
   down letter c, which will start expanding to many \subset's
2. Having a shell script, use dotool to enter
   a piece of markdown in the same file as above

Prerequisites

Minimal configuration used:

init.lua

local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "https://github.com/folke/lazy.nvim.git",
    "--branch=stable", -- latest stable release
    lazypath,
  })
end
vim.opt.rtp:prepend(lazypath)

vim.opt.wrap = true
vim.opt.textwidth = 80

vim.loader.enable()

local plugins = {}
local function use(plugin)
  table.insert(plugins, plugin)
end

use({
  -- "iurimateus/luasnip-latex-snippets.nvim",
  dir = "~/Projects/nvim/luasnip-latex-snippets.nvim",
  dependencies = { "L3MON4D3/LuaSnip", "nvim-treesitter/nvim-treesitter" },
  config = function()
    require("luasnip-latex-snippets").setup({
      use_treesitter = true,
      markdown_use_polling = false, -- TOGGLE THAT WHEN BENCHMARKING
    })
    local ls = require("luasnip")
    ls.config.setup({ enable_autosnippets = true })
    vim.keymap.set({ "i", "s" }, "<Tab>", function() ls.jump(1) end, { silent = true })
    vim.keymap.set({ "i", "s" }, "<S-Tab>", function() ls.jump(-1) end, { silent = true })
  end,
})

use({
  "nvim-treesitter/nvim-treesitter",
  build = ":TSUpdate",
  config = function()
    local ts_conf = require("nvim-treesitter.configs")

    ts_conf.setup({
      ensure_installed = { "markdown", "markdown_inline", "latex" },
      highlight = { enable = true },
    })
  end,
})

require("lazy").setup(plugins, {
  performance = {
    cache = { enabled = false },
  },
})

local p = require "jit.p"
vim.api.nvim_create_autocmd("ExitPre", {
  callback = function()
    p.stop()
  end,
})
p.start("-3vFm3", "profiler.log")

Here is the file which was edited(~1000 lines):

test.md

Please note that in the given dotool-###.sh files there are options that set
the keyboard layout. To execute them correctly you must set yours otherwise it
will produce gibberish(and it is actually dangerous because you don't know what
will it type) or remove them entirely if you use qwerty.

See dotool documentation

Holding the C

It shows how fast luasnip reacts with either polling or no polling.
To measure that, count how many letters c did not expand vs how many there
are \subset's at the very bottom of a resulting file(the one provided in archive at letterC/*.md.

dotool-c.sh

#!/usr/bin/env sh
export DOTOOL_XKB_LAYOUT="pl"
export DOTOOL_XKB_VARIANT="dvp"

pgrep dotoold || {
dotoold &
sleep 3
}

dotoolc <<EOF
typedelay 20
type nvim ./test.md -u ../init.lua
key enter
EOF

sleep 3

dotoolc <<EOF
key G zt o enter
type mk
EOF

sleep 1

dotoolc <<EOF
keydown c
EOF

sleep 5 && dotoolc <<EOF
keyup c
EOF

dotoolc <<EOF
key esc
type :x
key enter
EOF

• test-polling.md is result after running dotool-c.sh with polling enabling,
  and profiler-polling.md is profiler stats for that run
• test-no-polling.md is result after running dotool-c.sh with polling
  disabled, and profiler-no-polling.md is profiler stats for that run

As you can see with polling it did much better in terms how many cc's were
expanded.

Typing in a snippet

There is a file which types a snippet(works only on Linux, obviously read before
execution):

dotool-snip.sh

#!/usr/bin/env sh
export DOTOOL_XKB_LAYOUT="pl"
export DOTOOL_XKB_VARIANT="dvp"

pgrep dotoold || {
dotoold &
sleep 3
}

dotoolc <<EOF
typedelay 20
type nvim ./test.md -u ../init.lua
key enter
EOF

sleep 3

dotoolc <<EOF
typedelay 70
keydelay 100
key j j j ctrl+f ctrl+f ctrl+f
EOF

type_some_text() {
dotoolc <<EOF
key esc G zt o enter

type ## Markdown and limit
key enter enter
type What *could* be **better** than some \`markdown\` and mkttmath?
key enter enter

type - limit mkg for a
key enter
type - function with domain
key enter
type - mkA ccRR.
key enter
type - at any mkx_0 innA.

key enter enter

type dm
key backspace
type AAepsilon > 0\;
key enter
type EEdelta > 0\;
key enter
type AAx innA\;
key enter
type |x - x_0| < 0 => |f(x) - g| < epsilon
key esc
EOF
}
type_some_text

dotoolc <<EOF
key esc
type :x
key enter
EOF

Typing robotically a snippet of markdown with latex is assumed to show something
in profiler results,... doesn't tell much though. Both mechanisms did well

On PR

Inspired by #11

Libuv's timer has a delay of 100, which might be a bit too frequent. I believe
setting it to 200 or even 300 wouldn't hurt although I didn't try.

There was no field-testing as i have a pause in studies, so I actually don't
know how it behaves when editing markdown for a prolonged time.

Also if it behaves well, I would drop support of non-cached is_math and not_math.

Q: why use polling instead of triggering updates on every keypress?

A:

Note: To use the parser directly inside a |nvim_buf_attach()| Lua
callback, you must call |vim.treesitter.get_parser()| before you register
your callback. But preferably parsing shouldn't be done directly in the
change callback anyway as they will be very frequent. Rather a plugin that
does any kind of analysis on a tree should use a timer to throttle too
frequent updates.

(from https://neovim.io/doc/user/treesitter.html#lua-treesitter-languagetree)

This is a draft, and I am open to any suggestions.

Archive with the results:
ls-latex-snips-benchmark.zip

[reptee]

Forgot to mention that provided solution with some minor effort could be extended to work with latex

[iurimateus]

Thanks for looking into this. I haven't had the time to try it out yet.

I'll leave this here LazyVim/LazyVim@a5c9708 as a note for myself later (not directly related)

[iurimateus]

Sorry for the late response. Is this still necessary after cab1346? (we aren't calling :parse() anymore).

There might be some improvements on nvim 0.11 due to partial injections/incremental invalidation (PR #26827).