Importing silently fails without UI error for 10k character+ secure notes #3048
Comments
|
Another similar previous discussion related to import of notes exceeding the maximum allowed value. I'm sure imposing the limit shouldn't be too difficult, but I wonder if there is any elegant way to "convert" any possible old notes that currently would be over the upstream value should Vaultwarden also honor the 10k encrypted limit to minimize breaking thing as much as possible. |
|
As a new user, also affected by this symptom I would suggest the following:
🤓 |
|
Just discovered another surprise; re-import of the same file, generates duplicate folders and items, even though the import file contains identical data to already existing items. Any tips on bulk cleaning up duplicates ? |
I just purged the vault data and reattempted. That is an upstream bug too but I think if it validates all items before actually restoring that would prevent that in first place. Not sure how feasible that is or if you can use a temp vault then nuke after it fails and then if it succeeds copy or do real import to main one. |
|
Why is there 10_000 limit anyway, and can we extend it? That means we can get 65535 characters without changing the database schema. Last pass allows a lot more characters so its impossible to import notes properly from last pass. |
|
@htunlogic see: #2937 As we want to keep as close as possible to Bitwarden, and don't want clients to break, because that would make it totally useless. |
|
@BlackDex You are correct. I haven't been really thinking about that... My only priority is to get the hell away from Lastpass :) |
|
Might i suggest to use attachments in this case? You can put attachments to these items. |
|
Well, the only thing I'm having issue with really are the GPG keys I have stored in secure notes. I'll shuffle through them, some of them are expired and old so those aren't that important, and others, I'll give it a try with attachments. Thanks a lot ! :) |
|
I wonder if we just created an attachment of the 10k+ character note in a note with same name as original one if that would be an acceptable compromise? doesn't technically deviate from upstream just makes it easier for people to get into bitwarden format without having to delete long notes re-export and try importing again |
That isn't something we can do in the background. Attachments are handled differently. Also filenames are encrypted for example, not something we can (or want) to do on the server side. Soz you can do this your self of course. |
When importing to Vaultwarden (or Bitwarden) notes larger then 10_000 encrypted characters are invalid. This because it for one isn't compatible with Bitwarden. And some clients tend to break on very large notes. We already added a check for this limit when adding a single cipher, but this caused issues during import, and could cause a partial imported vault. Bitwarden does some validations before actually running it through the import process and generates a special error message which helps the user indicate which items are invalid during the import. This PR adds that validation check and returns the same kind of error. Fixes dani-garcia#3048
When importing to Vaultwarden (or Bitwarden) notes larger then 10_000 encrypted characters are invalid. This because it for one isn't compatible with Bitwarden. And some clients tend to break on very large notes. We already added a check for this limit when adding a single cipher, but this caused issues during import, and could cause a partial imported vault. Bitwarden does some validations before actually running it through the import process and generates a special error message which helps the user indicate which items are invalid during the import. This PR adds that validation check and returns the same kind of error. Fixes dani-garcia#3048
When importing to Vaultwarden (or Bitwarden) notes larger then 10_000 encrypted characters are invalid. This because it for one isn't compatible with Bitwarden. And some clients tend to break on very large notes. We already added a check for this limit when adding a single cipher, but this caused issues during import, and could cause a partial imported vault. Bitwarden does some validations before actually running it through the import process and generates a special error message which helps the user indicate which items are invalid during the import. This PR adds that validation check and returns the same kind of error. Fixes #3048
Subject of the issue
Import errors are not handled the same as upstream for failing things. Which is confusing for people not tech savvy..
Your environment (Generated via diagnostics page)
Config (Generated via diagnostics page)
Show Running Config
Environment settings which are overridden: SIGNUPS_ALLOWED, INVITATIONS_ALLOWED, ADMIN_TOKEN
{ "_duo_akey": null, "_enable_duo": false, "_enable_email_2fa": false, "_enable_smtp": true, "_enable_yubico": true, "_icon_service_csp": "", "_icon_service_url": "", "_ip_header_enabled": true, "_smtp_img_src": "cid:", "admin_ratelimit_max_burst": 3, "admin_ratelimit_seconds": 300, "admin_token": "***", "allowed_iframe_ancestors": "", "attachments_folder": "data/attachments", "authenticator_disable_time_drift": false, "data_folder": "data", "database_conn_init": "", "database_max_conns": 10, "database_timeout": 30, "database_url": "***************", "db_connection_retries": 15, "disable_2fa_remember": false, "disable_admin_token": false, "disable_icon_download": false, "domain": "*****://*******************", "domain_origin": "*****://*******************", "domain_path": "", "domain_set": true, "duo_host": null, "duo_ikey": null, "duo_skey": null, "email_attempts_limit": 3, "email_expiration_time": 600, "email_token_size": 6, "emergency_access_allowed": true, "emergency_notification_reminder_schedule": "0 3 * * * *", "emergency_request_timeout_schedule": "0 7 * * * *", "enable_db_wal": true, "event_cleanup_schedule": "0 10 0 * * *", "events_days_retain": null, "extended_logging": true, "helo_name": null, "hibp_api_key": null, "icon_blacklist_non_global_ips": true, "icon_blacklist_regex": null, "icon_cache_folder": "data/icon_cache", "icon_cache_negttl": 259200, "icon_cache_ttl": 2592000, "icon_download_timeout": 10, "icon_redirect_code": 302, "icon_service": "internal", "incomplete_2fa_schedule": "30 * * * * *", "incomplete_2fa_time_limit": 3, "invitation_expiration_hours": 120, "invitation_org_name": "Vaultwarden", "invitations_allowed": true, "ip_header": "X-Real-IP", "job_poll_interval_ms": 30000, "log_file": null, "log_level": "Info", "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f", "login_ratelimit_max_burst": 10, "login_ratelimit_seconds": 60, "org_attachment_limit": null, "org_creation_users": "", "org_events_enabled": false, "org_groups_enabled": false, "password_hints_allowed": true, "password_iterations": 100000, "reload_templates": false, "require_device_email": false, "rsa_key_filename": "data/rsa_key", "send_purge_schedule": "0 5 * * * *", "sends_allowed": true, "sends_folder": "data/sends", "show_password_hint": false, "signups_allowed": true, "signups_domains_whitelist": "*************", "signups_verify": false, "signups_verify_resend_limit": 6, "signups_verify_resend_time": 3600, "smtp_accept_invalid_certs": false, "smtp_accept_invalid_hostnames": false, "smtp_auth_mechanism": null, "smtp_debug": false, "smtp_embed_images": true, "smtp_explicit_tls": null, "smtp_from": "********************", "smtp_from_name": "Vaultwarden", "smtp_host": "********************", "smtp_password": "***", "smtp_port": 587, "smtp_security": "starttls", "smtp_ssl": null, "smtp_timeout": 15, "smtp_username": "********************", "templates_folder": "data/templates", "tmp_folder": "data/tmp", "trash_auto_delete_days": null, "trash_purge_schedule": "0 5 0 * * *", "use_syslog": false, "user_attachment_limit": null, "web_vault_enabled": true, "web_vault_folder": "web-vault/", "websocket_address": "0.0.0.0", "websocket_enabled": true, "websocket_port": 3012, "yubico_client_id": "82595", "yubico_secret_key": "***", "yubico_server": null }Steps to reproduce
Export a 1Password account with 10,000character+ secure note as 1pux file... or use attached.
Import 1password 1pux
browse to file
click import starts loading then shows console error
Example from attached file
https://user-images.githubusercontent.com/1596188/209586637-c2158710-9e9d-487a-8808-f25370fbc611.mp4
Test 1pux file to test with. Just will need to rename to remove the .zip as github doesn't allow 1pux
1PasswordExport-4WNLQ6BLP5HXFOIOEC62DSEZPQ-20221226-170653.1pux.zip
Expected behaviour
For it to show error like it does on main bitwarden where it shows errors per bad note with the name so you know what to fix before exporting again.
Example:
Import error
There was a problem with the data you tried to import. Please resolve the errors listed below in your source file and try again.
[1] [SecureNote] "Lorem ipsum 100 paragraphs large note": The field Notes exceeds the maximum encrypted value length of 10000 characters.
Error is Related to the limit upstream just not handled the same in the UI here:
https://www.reddit.com/r/Bitwarden/comments/sklxy5/why_limit_secure_notes_to_10000_characters/
https://community.bitwarden.com/t/support-longer-notes-breaks-lastpass-import/2970
Actual behaviour
Import looks like its working, but stops and fails when it gets to long note and does not say any error in main UI.
Response raw
Troubleshooting data
The text was updated successfully, but these errors were encountered: