Skip to content

Free req.file.pathw in fs::ReadFileUtf8(). #57811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

Justin-Nietzel
Copy link
Contributor

Always call uv_fs_req_cleanup after calling uv_fs_open instead of just when uv_fs_open returns a negative result. I referenced ReadFileSync from node:js2c when making this change.

https://github.com/bnoordhuis made the same suggestion based on the PR #49691.

Fixes: #57800

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. labels Apr 9, 2025
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 10, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 10, 2025
@nodejs-github-bot
Copy link
Collaborator

@Justin-Nietzel Justin-Nietzel marked this pull request as ready for review April 10, 2025 14:39
@Justin-Nietzel
Copy link
Contributor Author

I updated the commit comments after the first CI failure with module names in the title. I think it was mistaking some variable names as modules? I moved the original title into the description. Will that description also get flagged. Is there a safe way to format the variable name without triggering the module name CI check?

Copy link
Member

@tniessen tniessen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Justin-Nietzel You need to force-push to your branch issue-57800 with an amended commit message. Since req.file.pathw is an implementation detail in an external dependency, I'd suggest a commit message title such as fs: add missing call to uv_fs_req_cleanup or something like that.

Is it safe to also unconditionally call us_fs_req_cleanup(&req) in defer_close? It seems very odd to me that defer_close will always call uv_fs_req_cleanup(&req) while conditionally reusing &req for uv_fs_close. Before #49691, there was a seperate uv_fs_t for the close operation (cc @anonrig).

Always call uv_fs_req_cleanup after calling uv_fs_open instead of just
when uv_fs_open returns a negative result. I referenced ReadFileSync
from node:js2c when making this change.

https://github.com/bnoordhuis made the same suggestion based on the
PR nodejs#49691.

Fixes: nodejs#57800
@Justin-Nietzel
Copy link
Contributor Author

ReadFileSync in js2c.cc follows a very similar workflow to ReadFileUTF8 when using opening a file by file path. It re-uses the req object and always calls uv_fs_close after calling uv_fs_req_cleanup. I can't say with any certainty whether this behavior is "correct" or not, but I found it to be a useful reference.

Copy link

codecov bot commented Apr 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.23%. Comparing base (8dce721) to head (951437d).
Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #57811      +/-   ##
==========================================
- Coverage   90.23%   90.23%   -0.01%     
==========================================
  Files         630      630              
  Lines      185465   185471       +6     
  Branches    36368    36374       +6     
==========================================
- Hits       167362   167353       -9     
+ Misses      10999    10996       -3     
- Partials     7104     7122      +18     
Files with missing lines Coverage Δ
src/node_file.cc 76.72% <100.00%> (+<0.01%) ⬆️

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2025
@mcollina
Copy link
Member

@Justin-Nietzel do you think you'd be able to add a test that verify the memory leak is fixed?

@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2025
@nodejs-github-bot
Copy link
Collaborator

@Justin-Nietzel
Copy link
Contributor Author

Justin-Nietzel commented Apr 11, 2025

I probably could add a unit test. I would reference the readFileSync test and this worker thread memory test. I just had a few questions.

  1. This leak scales with path length of the file bring read. I would probably want to make sure my test file has a path length of at least 100. Should I create a new test folder in fixtures for this issue so I'm not putting a really long file name directly in the fixtures folder? Alternatively, I could create the long file name in the temporary directory based on the current path instead.
  2. For a noticable difference I would probably be doing 100,00 plus file reads. Should I put this in the pummel folder?
  3. Based on my understanding of the documentation, each test is ran as a separate process. I don't have to worry about other tests effecting the RSS memory usage, right?

Added a unit test for testing the memory usage of readFileSync.
Test is looking specifically for the the issue caused by failing
to free the filepath buffer in fs::ReadFileUtf8(), but it will also
catch other significant memory leaks in readFileSync() as well.

Refs: nodejs#57800
@Justin-Nietzel
Copy link
Contributor Author

@mcollina I added a unit test for this fix and did my best to follow all the documentation I could find on style and testing. Let me know if you need me to make any adjustments.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 11, 2025
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 11, 2025

jasnell pushed a commit that referenced this pull request Apr 12, 2025
Always call uv_fs_req_cleanup after calling uv_fs_open instead of just
when uv_fs_open returns a negative result. I referenced ReadFileSync
from node:js2c when making this change.

https://github.com/bnoordhuis made the same suggestion based on the
PR #49691.

Fixes: #57800
PR-URL: #57811
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
jasnell pushed a commit that referenced this pull request Apr 12, 2025
Added a unit test for testing the memory usage of readFileSync.
Test is looking specifically for the the issue caused by failing
to free the filepath buffer in fs::ReadFileUtf8(), but it will also
catch other significant memory leaks in readFileSync() as well.

Refs: #57800
PR-URL: #57811
Fixes: #57800
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@jasnell
Copy link
Member

jasnell commented Apr 12, 2025

Landed in d0a5bd6...09ecd2e

@jasnell jasnell closed this Apr 12, 2025
@tniessen
Copy link
Member

On a side note for future PRs, the second commit technically does not satisfy our requirements for commit messages since the leading verb is not imperative (added -> add).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory leak in node::fs::ReadFileUtf8(const FunctionCallbackInfo<Value>& args) when args[0] is a string.
5 participants