[Bug]: Upgrade 30.0.10 to 31.04 (dav) requires 6GB of RAM on single-user instance #52505
Open
5 of 8 tasks
Labels
0. Needs triage
Pending check for reproducibility or if it fits our roadmap
31-feedback
bug
feature: caldav
Related to CalDAV internals
feature: database
Database related DB
performance 🚀
Comments
Kunsi
added
0. Needs triage
|
Note: As of writing this bug report, the task is still ongoing. I will update the original report as soon as i get more information. |
|
Update: Yeah, after giving it 8 Gigabytes of Memory, the update went through successfully. It took 36 Minutes until the next log message appeared (note the timestamps): Everything else took <2 minutes to execute. |
|
Full updater log: |
|
cc @ChristophWurst this seems to be CalDAV related. |
|
The |
|
In December i removed a large ICS subscription, but since then i did multiple upgrades of my nextcloud instance which did all only ever take a couple of minutes. |
|
Maybe this background job can be chunked to reduce memory footprint? |
Sorry, I wrote background job but meant repair step 🙈 The repair step is about removing orphaned data. What are your thoughts on marking it as expensive to skip it on upgrades? Deletion is done in chunks of 200 ids. I think we could bump that to 1000 like we do in other places to have less expensive queries. Wrapping everything in a transaction could also help. |
|
Yeah, we can increase the chunk size. |
4 tasks
I meant like with other repair steps like this:
See also |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug description
When trying to upgrade 30.0.10 to 31.0.4, the
occ upgradecommand exits without error after trying to update the dav app. Increasingmemory_limitto ludicrous amounts of memory seems to help.Steps to reproduce
php occ upgradeon the shellExpected behavior
occ upgraderuns successfully and does not allocate multiple gigabytes of memoryNextcloud Server version
31
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.2
Web server
Nginx
Database engine version
PostgreSQL
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 31 to 32)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "warnochwas.de" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "pgsql", "version": "30.0.10.1", "overwrite.cli.url": "https:\/\/warnochwas.de", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "sendmail", "mail_sendmailmode": "smtp", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "25", "memcache.locking": "\\OC\\Memcache\\Redis", "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.local": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 6379, "dbindex": 7 }, "default_phone_region": "DE", "maintenance": true, "theme": "", "loglevel": 0, "maintenance_window_start": 0, "app_install_overwrite": [ "maps", "diary" ], "updater.secret": "***REMOVED SENSITIVE VALUE***" } }List of activated Apps
- federation: 1.21.0 - files: 2.3.1 - files_downloadlimit: 3.0.0 - files_external: 1.22.0 - files_pdfviewer: 3.0.0 - files_reminders: 1.3.0 - files_sharing: 1.22.0 - files_trashbin: 1.20.1 - files_versions: 1.23.0 - firstrunwizard: 3.0.0 - logreader: 3.0.0 - lookup_server_connector: 1.19.0 - nextcloud_announcements: 2.0.0 - notifications: 3.0.0 - oauth2: 1.19.1 - password_policy: 3.0.0 - photos: 4.0.0-dev.1 - polls: 7.4.2 - privacy: 2.0.0 - profile: true - provisioning_api: 1.20.0 - recommendations: 3.0.0 - related_resources: 1.5.0 - serverinfo: 2.0.0 - settings: 1.13.0 - sharebymail: 1.20.0 - socialsharing_email: 3.3.0 - support: 2.0.0 - survey_client: 2.0.0 - systemtags: 1.20.0 - text: 4.1.0 - theming: 2.6.0 - twofactor_backupcodes: 1.19.0 - twofactor_nextcloud_notification: 4.0.0 - twofactor_totp: 12.0.0-dev - twofactor_webauthn: 2.1.0 - updatenotification: 1.20.0 - user_status: 1.10.0 - viewer: 3.0.0 - weather_status: 1.10.0 - webhook_listeners: 1.1.0-dev - workflowengine: 2.12.0 Disabled: - admin_audit: 1.21.0 - deck: 1.14.4 (installed 1.14.4) - diary: 0.0.11 (installed 0.0.11) - encryption: 2.19.0 - files_retention: 1.19.1 (installed 1.19.1) - files_rightclick: 0.15.1 (installed 1.6.0) - maps: 1.5.0 (installed 1.5.0) - onlyoffice: 9.8.0 (installed 9.8.0) - suspicious_login: 9.0.1 - user_ldap: 1.22.0Nextcloud Signing status
Nextcloud Logs
Additional info
I have temporarily increased phps
memory_limitto8192M(8GB) - that seems to work, however my default of 512MB or even 2048MB were not enough for a successful upgrade.Offending, long-running SQL query seems to be (caution, line is cut off):
DELETE FROM "oc_calendarobjects_props" WHERE "id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $50, $51, $52, $53, $54, $55, $56, $57, $58, $59, $60, $61, $62, $63, $64, $65, $66, $67, $68, $69, $70, $71, $72, $73, $74, $75, $76, $77, $78, $79, $80, $81, $82, $83, $84, $85, $86, $87, $88, $89, $90, $91, $92, $93, $94, $95, $96, $97, $98, $99, $100, $101, $102, $103, $104, $105, $106, $107, $108, $109, $110, $111, $112, $113, $114, $115, $116, $117, $118, $119, $120, $121, $122, $123, $124, $125, $126, $127, $128, $129, $130, $131, $132, $133, $134, $135, $136, $137, $138, $139, $140, $141, $142, $143, $144, $145, $146, $147, $148, $149, $150, $151, $152, $153, $154, $155, $156, $157, $158, $159, $160, $161, $162, $163, $164, $165, $166, $167, $168, $169, $170, $171, $172, $173, $174, $175, $176, $177, $178, $179, $18and/or a parallel running
VACUUM(i'm not sure whether the updater initiates that, my server should not automatically do that)During the upgrade the
php occ upgradeprocess consumed about 6GB of RAM, while https://github.com/syncloud/nextcloud/blob/master/config/php.ini#L393 recommends 1GB./var/www/nextcloudis about 114GB in size, nextcloud database about 13GB prior to starting the upgrade.The text was updated successfully, but these errors were encountered: