A very long initialization time of filters (deep + entry)

[mrmlnc]

Environment

• OS Version: macOS 10.13.3
• Node.js Version: 8.9.0

Source

• Performance loss in version 8 sindresorhus/globby#79

The problem

We have to figure out a way to reduce the initialization time of filters. Something like 14ms for the following tasks:

{ base: 'src/types/one',
  dynamic: true,
  patterns: [ 'src/types/one/*/sub/y/y.json' ],
  positive: [ 'src/types/one/*/sub/y/y.json' ],
  negative: [] }
Prepare reader options: 8.445ms
{ base: 'src/types/two',
  dynamic: true,
  patterns: [ 'src/types/two/*/sub/y/y.json' ],
  positive: [ 'src/types/two/*/sub/y/y.json' ],
  negative: [] }
Prepare reader options: 1.802ms
{ base: 'src/types/three',
  dynamic: true,
  patterns: [ 'src/types/three/*/sub/y/y.json' ],
  positive: [ 'src/types/three/*/sub/y/y.json' ],
  negative: [] }
Prepare reader options: 1.400ms
{ base: 'src/types/four',
  dynamic: true,
  patterns: [ 'src/types/four/*/sub/y/y.json' ],
  positive: [ 'src/types/four/*/sub/y/y.json' ],
  negative: [] }
Prepare reader options: 0.794ms
{ base: 'src/types/five',
  dynamic: true,
  patterns: [ 'src/types/five/*/sub/y/y.json' ],
  positive: [ 'src/types/five/*/sub/y/y.json' ],
  negative: [] }
Prepare reader options: 2.111ms

[mrmlnc]

[globby@7]: 14 ms

Tasks: 1.297ms
Prepare reader options: 22.569ms
Reader: 44.372ms
Transformer: 0.115ms
[fast-glob]: 69 ms

[jonschlinkert]

I mentioned this on the linked issue, but I want to get this fixed.

I also noticed this the other day before the referenced issue was created, then @doowb and I discussed it on a call. When I initially published v3 of micromatch, I thought it took something like 30ms.

edit: which I still was not happy with.

[jonschlinkert]

Ok, I know what's happening, and I know why it manifested recently.

Problem: On both Snapdragon and micromatch libraries, we've been upgrading dependencies to newer versions. This is causing a bunch of duplicates in the tree since some modules have not yet been upgraded.

Solution(s)

In the short term, to fix the problem:

1. we will expedite those upgrades and ensure everything is the latest.
2. I will look for every opportunity to remove deps with code that can be easily inlined

After that:

1. we are upgrading to es6 everywhere, which will also eliminate a few deps
2. I'm considering inlining the parsers/compilers instead of using snapdragon

fwiw, I would never publish code that took 90ms to load! that horrifies me. I'm an optimization freak. This will be fixed asap.

[mrmlnc]

Hello, @jonschlinkert,

Thanks for quickly response. I answered you in the previously created issue in the micromatch repository. I think we don't need to hurry.

[mrmlnc]

The original problem is fixed after migration from micromatch@3 to micromatch@4.

#154

The next slowdown problem will be fixed in the issue #156.