Optimize CipherSyncData for very large vaults #3133
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As mentioned in dani-garcia#3111, using a very very large vault causes some issues. Mainly because of a SQLite limit, but, it could also cause issue on MariaDB/MySQL or PostgreSQL. It also uses a lot of memory, and memory allocations. This PR solves this by removing the need of all the cipher_uuid's just to gather the correct attachments. It will use the user_uuid and org_uuid's to get all attachments linked to both, weither the user has access to them or not. This isn't an issue, since the matching is done per cipher and the attachment data is only returned if there is a matching cipher to where the user has access to. I also modified some code to be able to use `::with_capacity(n)` where possible. This prevents re-allocations if the `Vec` increases size, which will happen a lot if there are a lot of ciphers. According to my tests measuring the time it takes to sync, it seems to have lowered the duration a bit more. Fixes dani-garcia#3111
Stackclash
referenced
this pull request
in Stackclash/home-cluster
May 25, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [vaultwarden/server](https://togithub.com/dani-garcia/vaultwarden) | minor | `1.27.0` -> `1.28.1` | --- ### Release Notes <details> <summary>dani-garcia/vaultwarden</summary> ### [`v1.28.1`](https://togithub.com/dani-garcia/vaultwarden/releases/tag/1.28.1) [Compare Source](https://togithub.com/dani-garcia/vaultwarden/compare/1.28.0...1.28.1) #### What's Changed - Decode knowndevice `X-Request-Email` as base64url with no padding by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3376](https://togithub.com/dani-garcia/vaultwarden/pull/3376) - Fix abort on password reset mail error by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3390](https://togithub.com/dani-garcia/vaultwarden/pull/3390) - support `/users/<uuid>/invite/resend` admin api by [@​nikolaevn](https://togithub.com/nikolaevn) in [https://github.com/dani-garcia/vaultwarden/pull/3397](https://togithub.com/dani-garcia/vaultwarden/pull/3397) - always return KdfMemory and KdfParallelism by [@​stefan0xC](https://togithub.com/stefan0xC) in [https://github.com/dani-garcia/vaultwarden/pull/3398](https://togithub.com/dani-garcia/vaultwarden/pull/3398) - Fix sending out multiple websocket notifications by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3405](https://togithub.com/dani-garcia/vaultwarden/pull/3405) - Revert setcap, update rust and crates by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3403](https://togithub.com/dani-garcia/vaultwarden/pull/3403) #### New Contributors - [@​nikolaevn](https://togithub.com/nikolaevn) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3397](https://togithub.com/dani-garcia/vaultwarden/pull/3397) **Full Changelog**: dani-garcia/vaultwarden@1.28.0...1.28.1 ### [`v1.28.0`](https://togithub.com/dani-garcia/vaultwarden/releases/tag/1.28.0) [Compare Source](https://togithub.com/dani-garcia/vaultwarden/compare/1.27.0...1.28.0) #### Major changes - The project has changed license to the [**AGPLv3**](https://togithub.com/dani-garcia/vaultwarden/blob/main/LICENSE.txt). If you're hosting a Vaultwarden instance, you now have a requirement to distribute the Vaultwarden source code to your users if they request it. The source code, and any changes you have made, need to be under the same AGPLv3 license. If you simply use our code without modifications, just pointing them to this repository is enough. - Added support for **Argon2** key derivation on the clients. To enable it for your account, make sure all your clients are using version v2023.2.0 or greater, then go to account settings > security > keys, and change the algorithm from PBKDF2 to Argon2id. - Added support for **Argon2** key derivation for the admin page token. To update your admin token to use it, [check the wiki](https://togithub.com/dani-garcia/vaultwarden/wiki/Enabling-admin-page#secure-the-admin_token) - New **alternative registries** for the docker images are available (In **BETA** for now): - **Github Container Registry**: https://ghcr.io/dani-garcia/vaultwarden - **Quay**: https://quay.io/vaultwarden/server #### What's Changed - Remove patched multer-rs by [@​manofthepeace](https://togithub.com/manofthepeace) in [https://github.com/dani-garcia/vaultwarden/pull/2968](https://togithub.com/dani-garcia/vaultwarden/pull/2968) - Removed unsafe-inline JS from CSP and other fixes by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3058](https://togithub.com/dani-garcia/vaultwarden/pull/3058) - Validate YUBICO_SERVER string ([#​3003](https://togithub.com/dani-garcia/vaultwarden/issues/3003)) by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3059](https://togithub.com/dani-garcia/vaultwarden/pull/3059) - Log message to stderr if LOG_FILE is not writable by [@​pjsier](https://togithub.com/pjsier) in [https://github.com/dani-garcia/vaultwarden/pull/3061](https://togithub.com/dani-garcia/vaultwarden/pull/3061) - Update WebSocket Notifications by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3076](https://togithub.com/dani-garcia/vaultwarden/pull/3076) - Optimize config loading messages by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3092](https://togithub.com/dani-garcia/vaultwarden/pull/3092) - Percent-encode org_name in links by [@​am97](https://togithub.com/am97) in [https://github.com/dani-garcia/vaultwarden/pull/3093](https://togithub.com/dani-garcia/vaultwarden/pull/3093) - Fix failing large note imports by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3087](https://togithub.com/dani-garcia/vaultwarden/pull/3087) - Change `text/plain` API responses to `application/json` by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3124](https://togithub.com/dani-garcia/vaultwarden/pull/3124) - Remove `shrink-to-fit=no` from viewport-meta-tag by [@​redwerkz](https://togithub.com/redwerkz) in [https://github.com/dani-garcia/vaultwarden/pull/3126](https://togithub.com/dani-garcia/vaultwarden/pull/3126) - Update dependencies and MSRV by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3128](https://togithub.com/dani-garcia/vaultwarden/pull/3128) - Resolve uninlined_format_args clippy warnings by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3065](https://togithub.com/dani-garcia/vaultwarden/pull/3065) - Update Rust to v1.66.1 to patch CVE by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3136](https://togithub.com/dani-garcia/vaultwarden/pull/3136) - Fix remaining inline format by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3130](https://togithub.com/dani-garcia/vaultwarden/pull/3130) - Use more modern meta tag for charset encoding by [@​redwerkz](https://togithub.com/redwerkz) in [https://github.com/dani-garcia/vaultwarden/pull/3131](https://togithub.com/dani-garcia/vaultwarden/pull/3131) - fix (2fa.directory): Allow api.2fa.directory, and remove 2fa.directory by [@​GeekCornerGH](https://togithub.com/GeekCornerGH) in [https://github.com/dani-garcia/vaultwarden/pull/3132](https://togithub.com/dani-garcia/vaultwarden/pull/3132) - Optimize CipherSyncData for very large vaults by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3133](https://togithub.com/dani-garcia/vaultwarden/pull/3133) - Add avatar color support by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3134](https://togithub.com/dani-garcia/vaultwarden/pull/3134) - Add MFA icon to org member overview by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3135](https://togithub.com/dani-garcia/vaultwarden/pull/3135) - Minor refactoring concering user.setpassword by [@​sirux88](https://togithub.com/sirux88) in [https://github.com/dani-garcia/vaultwarden/pull/3139](https://togithub.com/dani-garcia/vaultwarden/pull/3139) - Validate note sizes on key-rotation. by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3157](https://togithub.com/dani-garcia/vaultwarden/pull/3157) - Update KDF Configuration and processing by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3163](https://togithub.com/dani-garcia/vaultwarden/pull/3163) - Remove `arm32v6`-specific tag by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3164](https://togithub.com/dani-garcia/vaultwarden/pull/3164) - Re-License Vaultwarden to AGPLv3 by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/2561](https://togithub.com/dani-garcia/vaultwarden/pull/2561) - Admin password reset by [@​sirux88](https://togithub.com/sirux88) in [https://github.com/dani-garcia/vaultwarden/pull/3116](https://togithub.com/dani-garcia/vaultwarden/pull/3116) - "Spell-Jacking" mitigation ~ prevent sensitive data leak … by [@​dlehammer](https://togithub.com/dlehammer) in [https://github.com/dani-garcia/vaultwarden/pull/3145](https://togithub.com/dani-garcia/vaultwarden/pull/3145) - Allow listening on privileged ports (below 1024) as non-root by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3170](https://togithub.com/dani-garcia/vaultwarden/pull/3170) - don't nullify key when editing emergency access by [@​stefan0xC](https://togithub.com/stefan0xC) in [https://github.com/dani-garcia/vaultwarden/pull/3215](https://togithub.com/dani-garcia/vaultwarden/pull/3215) - Fix trailing slash not getting removed from domain by [@​BlockListed](https://togithub.com/BlockListed) in [https://github.com/dani-garcia/vaultwarden/pull/3228](https://togithub.com/dani-garcia/vaultwarden/pull/3228) - Generate distinct log messages for regex vs. IP blacklisting. by [@​kpfleming](https://togithub.com/kpfleming) in [https://github.com/dani-garcia/vaultwarden/pull/3231](https://togithub.com/dani-garcia/vaultwarden/pull/3231) - allow editing/unhiding by group by [@​farodin91](https://togithub.com/farodin91) in [https://github.com/dani-garcia/vaultwarden/pull/3108](https://togithub.com/dani-garcia/vaultwarden/pull/3108) - Fix Javascript issue on non sqlite databases by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3167](https://togithub.com/dani-garcia/vaultwarden/pull/3167) - add argon2 kdf fields by [@​tessus](https://togithub.com/tessus) in [https://github.com/dani-garcia/vaultwarden/pull/3210](https://togithub.com/dani-garcia/vaultwarden/pull/3210) - add support for system mta though sendmail by [@​soruh](https://togithub.com/soruh) in [https://github.com/dani-garcia/vaultwarden/pull/3147](https://togithub.com/dani-garcia/vaultwarden/pull/3147) - Updated Rust and crates by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3234](https://togithub.com/dani-garcia/vaultwarden/pull/3234) - docs: add build status badge in readme by [@​R3DRUN3](https://togithub.com/R3DRUN3) in [https://github.com/dani-garcia/vaultwarden/pull/3245](https://togithub.com/dani-garcia/vaultwarden/pull/3245) - Validate all needed fields for client API login by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3251](https://togithub.com/dani-garcia/vaultwarden/pull/3251) - Fix Organization delete when groups are configured by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3252](https://togithub.com/dani-garcia/vaultwarden/pull/3252) - Fix Collection Read Only access for groups by [@​Misterbabou](https://togithub.com/Misterbabou) in [https://github.com/dani-garcia/vaultwarden/pull/3254](https://togithub.com/dani-garcia/vaultwarden/pull/3254) - Make the admin session lifetime adjustable by [@​mittler-works](https://togithub.com/mittler-works) in [https://github.com/dani-garcia/vaultwarden/pull/3262](https://togithub.com/dani-garcia/vaultwarden/pull/3262) - Add function to fetch user by email address by [@​mittler-works](https://togithub.com/mittler-works) in [https://github.com/dani-garcia/vaultwarden/pull/3263](https://togithub.com/dani-garcia/vaultwarden/pull/3263) - Fix vault item display in org vault view by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3277](https://togithub.com/dani-garcia/vaultwarden/pull/3277) - Add confirmation for removing 2FA and deauthing sessions in admin panel by [@​JCBird1012](https://togithub.com/JCBird1012) in [https://github.com/dani-garcia/vaultwarden/pull/3282](https://togithub.com/dani-garcia/vaultwarden/pull/3282) - Some Admin Interface updates by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3288](https://togithub.com/dani-garcia/vaultwarden/pull/3288) - Fix the web-vault v2023.2.0 API calls by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3281](https://togithub.com/dani-garcia/vaultwarden/pull/3281) - Fix confirmation for removing 2FA and deauthing sessions in admin panel by [@​dpinse](https://togithub.com/dpinse) in [https://github.com/dani-garcia/vaultwarden/pull/3290](https://togithub.com/dani-garcia/vaultwarden/pull/3290) - Admin token Argon2 hashing support by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3289](https://togithub.com/dani-garcia/vaultwarden/pull/3289) - Add HEAD routes to avoid spurious error messages by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3307](https://togithub.com/dani-garcia/vaultwarden/pull/3307) - Fix web-vault Member UI show/edit/save by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3315](https://togithub.com/dani-garcia/vaultwarden/pull/3315) - Upd Crates, Rust, MSRV, GHA and remove Backtrace by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3310](https://togithub.com/dani-garcia/vaultwarden/pull/3310) - Add support for `/api/devices/knowndevice` with HTTP header params by [@​jjlin](https://togithub.com/jjlin) in [https://github.com/dani-garcia/vaultwarden/pull/3329](https://togithub.com/dani-garcia/vaultwarden/pull/3329) - Update Rust, MSRV and Crates by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3348](https://togithub.com/dani-garcia/vaultwarden/pull/3348) - Merge ClientIp with Headers. by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3332](https://togithub.com/dani-garcia/vaultwarden/pull/3332) - add endpoints to bulk delete collections/groups by [@​stefan0xC](https://togithub.com/stefan0xC) in [https://github.com/dani-garcia/vaultwarden/pull/3354](https://togithub.com/dani-garcia/vaultwarden/pull/3354) - Add support for Quay.io and GHCR.io as registries by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3363](https://togithub.com/dani-garcia/vaultwarden/pull/3363) - Some small fixes and updates by [@​BlackDex](https://togithub.com/BlackDex) in [https://github.com/dani-garcia/vaultwarden/pull/3366](https://togithub.com/dani-garcia/vaultwarden/pull/3366) - Update web vault to v2023.3.0 by [@​dani-garcia](https://togithub.com/dani-garcia) #### New Contributors - [@​manofthepeace](https://togithub.com/manofthepeace) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/2968](https://togithub.com/dani-garcia/vaultwarden/pull/2968) - [@​pjsier](https://togithub.com/pjsier) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3061](https://togithub.com/dani-garcia/vaultwarden/pull/3061) - [@​am97](https://togithub.com/am97) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3093](https://togithub.com/dani-garcia/vaultwarden/pull/3093) - [@​redwerkz](https://togithub.com/redwerkz) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3126](https://togithub.com/dani-garcia/vaultwarden/pull/3126) - [@​sirux88](https://togithub.com/sirux88) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3139](https://togithub.com/dani-garcia/vaultwarden/pull/3139) - [@​dlehammer](https://togithub.com/dlehammer) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3145](https://togithub.com/dani-garcia/vaultwarden/pull/3145) - [@​BlockListed](https://togithub.com/BlockListed) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3228](https://togithub.com/dani-garcia/vaultwarden/pull/3228) - [@​kpfleming](https://togithub.com/kpfleming) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3231](https://togithub.com/dani-garcia/vaultwarden/pull/3231) - [@​farodin91](https://togithub.com/farodin91) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3108](https://togithub.com/dani-garcia/vaultwarden/pull/3108) - [@​soruh](https://togithub.com/soruh) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3147](https://togithub.com/dani-garcia/vaultwarden/pull/3147) - [@​R3DRUN3](https://togithub.com/R3DRUN3) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3245](https://togithub.com/dani-garcia/vaultwarden/pull/3245) - [@​Misterbabou](https://togithub.com/Misterbabou) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3254](https://togithub.com/dani-garcia/vaultwarden/pull/3254) - [@​mittler-works](https://togithub.com/mittler-works) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3262](https://togithub.com/dani-garcia/vaultwarden/pull/3262) - [@​JCBird1012](https://togithub.com/JCBird1012) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3282](https://togithub.com/dani-garcia/vaultwarden/pull/3282) - [@​dpinse](https://togithub.com/dpinse) made their first contribution in [https://github.com/dani-garcia/vaultwarden/pull/3290](https://togithub.com/dani-garcia/vaultwarden/pull/3290) **Full Changelog**: dani-garcia/vaultwarden@1.27.0...1.28.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on saturday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/RickCoxDev/home-cluster). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS43OS4xIiwidXBkYXRlZEluVmVyIjoiMzUuNzkuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As mentioned in #3111, using a very very large vault causes some issues. Mainly because of a SQLite limit, but, it could also cause issue on MariaDB/MySQL or PostgreSQL. It also uses a lot of memory, and memory allocations.
This PR solves this by removing the need of all the cipher_uuid's just to gather the correct attachments.
It will use the user_uuid and org_uuid's to get all attachments linked to both, weither the user has access to them or not. This isn't an issue, since the matching is done per cipher and the attachment data is only returned if there is a matching cipher to where the user has access to.
I also modified some code to be able to use
::with_capacity(n)where possible. This prevents re-allocations if theVecincreases size, which will happen a lot if there are a lot of ciphers.According to my tests measuring the time it takes to sync, it seems to have lowered the duration a bit more.
Fixes #3111