Is there a tricky bug in the implementation of the profiles in the Joplin Android app?

[CoffeeHT]

Operating system

Android

Application version

2.12.3

Desktop: About dialog content

This problem does not occur with either a Joplin Windows App or a Joplin Linux App.

Both profiles (A-user and B-user) can easily be connected to the same Joplin Windows App. Switching between profiles is possible without any problems.

Both profiles (A-user and B-user) can easily be connected to the same Joplin Linux App. Switching between profiles is possible without any problems.

Current behaviour

In order to be able to reproduce the problem that has existed for months without jeopardising my productive environment and the database of my productive Joplin profiles, I have commissioned a separate Managed NextCloud instance from my provider for the reproduction of the problem and possible further tests. I also brought forward my investment in a new Android mobile phone so as not to expose my Android mobile phone, which is currently in productive use, to any risk.

A WebDAV connection to this managed Nextcloud 100, version 26.0.6 of the provider hosting.de is used.
When I asked if my NextCloud runs in / with an nginx webserver I got the answer
» yes, we use the official nginx configuration as a basis:
NGINX configuration — Nextcloud latest Administration Manual latest documentation .
«

Two user IDs have been created on the NextCloud instance I have temporarily commissioned for this purpose:
A-User
B-User

For each of these user IDs, a new profile was successfully set up in a Joplin Windows app and a Joplin Linux app. The access data including the WebDAV path to the user's Joplin directory were documented in a password safe, which was also synchronised with the Android test device. This ensured that the identical access data was used in every case.

Android - test device is a Samsung Galaxy S23 Ultra with Android 13 and a Joplin Android app "Joplin 2.12.3" newly installed from the Google Play Store before each of the tests.

Both profiles ( A-User and B-User) can be connected in a newly installed Joplin Android app if you connect them as the first profile.
The second profile does not connect but runs into this strange error.
It doesn't matter whether you connect A-User or B-User first.

Expected behaviour

Both profiles (A-user and B-user) can be easily connected to the same Joplin Android app. Switching between the profiles is possible without any problems.

Logs

I have already provided log information at

https://discourse.joplinapp.org/t/is-there-a-tricky-bug-in-the-implementation-of-the-profiles-in-the-joplin-android-app/32864/41?u=coffee

and

https://discourse.joplinapp.org/t/is-there-a-tricky-bug-in-the-implementation-of-the-profiles-in-the-joplin-android-app/32864/42?u=coffee

[CoffeeHT]

This problem may be related to the description I found during my research on https://trac.nginx.org/nginx/ticket/1966.

Description

Some WebDAV clients that create collections do that without including the trailing slash. The logs offer no further explanation, and to a casual user the 409 error is confusing (as it indicates that the parent collection is absent).

RFC4918 recommends them to have a trailing slash, but given that several applications don't send a trailing slash, it stands to reason that it's common for servers to accept it. Section 5.2 even gives concrete guidance that in such a case, the server should accept and send a Content-Location header ("There is a standing convention").

Example clients that don't are webdav-js (​GitHub - dom111/webdav-js: A simple WebDAV client written in JS for use as a bookmarklet, or integration into a web server.) and Stratospherix FileBrowser for Business; some examples are around for other clients that employ workarounds (eg. ​Does not play nice with nginx WebDAV · Issue #523 · laurent22/joplin · GitHub).

[laurent22]

I tried to replicate with Nextcloud but couldn't, each separate profile synchronises independently without any error.

Are you able to replicate this error with your test hosting.de account? If so could you provide the credentials via PM on the forum?

[CoffeeHT]

Hello Laurent,
I have never sent private messages there.
Could you please check whether you received my first test correctly? If so, I'll send the credentials the same way

[laurent22]

I'm getting an error with user B "WebDAV directory not found", so I think it just needs to be created as described here:

https://joplinapp.org/help/apps/sync/nextcloud

[CoffeeHT]

I think it just needs to be created as described here

I do not agree.

You have received the access data for both users.
If you had also used the access data to check whether this directory exists or if you had connected user B as the first profile with a newly installed Android Joplin app, you probably wouldn't have had these thoughts.

[laurent22]

Ok I've tried again and here's what I found:

This request:

curl -v -X PROPFIND -H "Depth: 0" -H "Authorization: Basic xxxxxxxxxx" -H "Content-Type: text/xml" -H "Cache-Control: no-store" -H "If-None-Match: JoplinIgnore-32401" -H "User-Agent: Joplin/1.0" --data '<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
<d:prop xmlns:oc="http://owncloud.org/ns">
<d:getlastmodified/><d:resourcetype/>
</d:prop>
</d:propfind>' https://xxxxxxxxx/remote.php/dav/files/B-User/Joplin

Fails with an unambiguous error message:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>File not found: Joplin in 'B-User'</s:message>
</d:error>

Except that "Joplin" exists for "B-User". The request above is definitely valid and the response shows that the server receives the correct username and folder, but somehow responds that it can't find it. Unfortunately I'm out of idea about what could be done. If the server is saying that something that exists doesn't, we can't really go past this.

I don't think it's related to profiles because clearly the request is using the correct credentials, associated with B-User (otherwise there would be an auth error), and the server clearly responds that it's dealing with B-User too, so there's no mix up. So why exactly it doesn't work, unfortunately I have no idea.

[CoffeeHT]

This problem may be related to the description I found during my research on https://trac.nginx.org/nginx/ticket/1966.

Description

Some WebDAV clients that create collections do that without including the trailing slash. The logs offer no further explanation, and to a casual user the 409 error is confusing (as it indicates that the parent collection is absent).

RFC4918 recommends them to have a trailing slash, but given that several applications don't send a trailing slash, it stands to reason that it's common for servers to accept it. Section 5.2 even gives concrete guidance that in such a case, the server should accept and send a Content-Location header ("There is a standing convention").

Example clients that don't are webdav-js (​GitHub - dom111/webdav-js: A simple WebDAV client written in JS for use as a bookmarklet, or integration into a web server.) and Stratospherix FileBrowser for Business; some examples are around for other clients that employ workarounds (eg. ​Does not play nice with nginx WebDAV · Issue #523 · laurent22/joplin · GitHub).

What about this idea ?

[laurent22]

Yes I thought that could be related so I've tried with and without trailing slash, and same result

[CoffeeHT]

Unfortunately, I can't write any programme code and probably can't read it in any meaningful way either.

I could imagine that there is an error somewhere in the code that controls the data for the profiles. Everything works with the first profile.

From the point in the code marked with the + symbol under Profiles, there could be a problem hidden somewhere that leads to some kind of error when creating the variables for the next profile.

Or this function that creates the variables for the new profile is clean and the problem starts the moment you want to change the profile and are asked to restart the application.

Or the problem starts the moment you use the second profile and want to enter and use the data of the second profile.

It could theoretically be that for some reason the exact credentials of the second profile are not transferred to the server, but something else or slightly changed.
Either the path is being transferred incorrectly somehow, or the user name or password. Perhaps there is an unprintable character somewhere. Maybe there is a place somewhere in the code that does not transmit a trailing slash.

You probably have a code that can somehow be instrumentalised so that the credentials are copied somewhere before and after they are transferred to the server. Perhaps you can recognise a problem from the credential values documented in this way.

These are just amateur theories, but that's all I have...

[CoffeeHT]

the response shows that the server receives the correct username and folder

Hmmm ... What about the password and the name of the JSON file?

[CoffeeHT]

File not found: Joplin in 'B-User'

This refers to the JSON file, doesn't it?
Could an incorrect file name be requested for whatever reason?

[CoffeeHT]

correct credentials, associated with B-User (otherwise there would be an auth error), and the server clearly responds that it's dealing with B-User too, so there's no mix up. So why exactly it doesn't work, unfortunately I have no idea.

Ooops ... only the file name of the JSON file remains from my assumptions, right ?

[laurent22]

There is not JSON file - essentially what the code above does is check that https://xxxxxxxxx/remote.php/dav/files/B-User/Joplin exists. It does, but the server says it doesn't.

Hmmm ... What about the password

The password is correct, otherwise the server will respond with an authentication error.

So there's no indication anywhere that profile info is being mixed up - it's using the correct username, password an URL to connect to WebDAV. It's confirmed by my debugging and by the server response. I never just assume that my code is right and the server is wrong, and I spent many hours getting things working with various WebDAV server. In this particular case however the only thing that doesn't make sense is the server response.

[CoffeeHT]

essentially what the code above does is check that https://xxxxxxxxx/remote.php/dav/files/B-User/Joplin exists. It does, but the server says it doesn't.

Does this mean that we would need the programmers of the NextCloud software to solve this problem?
My hosting provider itself probably can't do that

[CoffeeHT]

and why does the problem only occur in the Android implementation, why not in the Windows or Linux implementation?

[CoffeeHT]

I tried to replicate with Nextcloud but couldn't, each separate profile synchronises independently without any error.

Which NextCloud version exactly did you test this with?

[laurent22]

Which NextCloud version exactly did you test this with?

27.1.0

[CoffeeHT]

Thank you very much for your persistent and patient dialogue with me, Laurent.

I think I need to sleep on it first.
When I analysed the NextCloud log files, I noticed a few little things when I compared the successful accesses of the Windows and Linux implementations with the accesses of the Android implementation.

I'll pick them out and try to describe them well. But I'll have to be more rested and have more time than tonight to do that

[laurent22]

I'll pick them out and try to describe them well.

Rather than describe them, simply attach them here if you can

[CoffeeHT]

27.1.0

Well, my environment is Version
26.0.6

[laurent22]

I doubt it's of any help but today I tried again and sync was now working for B-User, but when I switched to A-User, that one wasn't working anymore. So it looks like it's going to work for one account but not both.

What I wanted to double-check is that each profile use the correct auth information, and they do, so again it's still not clear what's happening.

[CoffeeHT]

I doubt it's of any help but today I tried again and sync was now working for B-User, but when I switched to A-User, that one wasn't working anymore. So it looks like it's going to work for one account but not both.

What I wanted to double-check is that each profile use the correct auth information, and they do, so again it's still not clear what's happening.

Thank you very much for the new test and the results report.
I've had this situation several times before.
It has almost driven me mad each time because it cannot be reliably reproduced and clearly described in a few words.

This strange part of this repeatedly strange problem then gave me the idea that it could be some kind of problem with incorrectly assigned or transmitted credential data, which is also so difficult to grasp because several procedures could be involved, which are run through in different sequences for different reasons in a way that is completely opaque to me.

I could only help myself by starting this test environment and buying a new smartphone and limiting all my problem descriptions to the sequence reinstall Android app, log in user-A immediately afterwards, log in user-B immediately afterwards.

This was the only way I could establish initial credibility.

[CoffeeHT]

Last Friday, I received a message from my provider that my NextClouds had been automatically updated to 26.0.8. They had previously been running on 26.0.6 since 09.10.2023. So I checked to see if the strange error was still there. This was no longer the case this morning ( Is it Christmas already ;-) ? ) In the corresponding ChangeLog I found the following fix, which is probably related to my previous problem:
nextcloud/server#40798
[stable26] fix(user): Log affected user of app token login name mismatch #40798
nextcloud/server#40795
fix(user): Log affected user of app token login name mismatch #40795

[CoffeeHT]

I can only really believe it when my planned migrations now work without any problems. That will still take some time. I'll get back to you ...

[CoffeeHT]

The problem has only changed in its behaviour, but is still present. In the test environment it was gone for both test users, but in the production environment it is still there for the second productive user. I have now migrated my own productive Joplin data to the hosting environment thinking that the issue was resolved, but for my second productive user I am now unable to log in from either of my Android devices.

[CoffeeHT]

27.1.0

My provider has just written to me that they are currently preparing the rollout of Nextcloud 27.x ... Maybe that will solve the problem.

[CoffeeHT]

My provider has just written to me that they are currently preparing the rollout of Nextcloud 27.x ... Maybe that will solve the problem.

Update to 27.1.4 did not fix the problem

[CoffeeHT]

Update to 27.1.4 did not fix the problem

@laurent22

My provider asks if you can reproduce the error again in my test environment.
In my opinion, the behaviour has not changed with the new version of NextCloud - at least at first glance.

[CoffeeHT]

My provider asks if you can reproduce the error again in my test environment.

@laurent22

[laurent22]

I did the same tests as you and could see it indeed happened. So if it's still happening for you it will still happen for me

[CoffeeHT]

I did the same tests as you and could see it indeed happened. So if it's still happening for you it will still happen for me

Thank you

[CoffeeHT]

Update to 27.1.4 did not fix the problem

Update to NextCloud 27.1.5 did not fix the problem

[leachimus]

Hello everyone?

I'm glad I found this post. The error is still there.

On two iOS devices, several profiles are synchronized without problems when switching. Tested with two iPhones and two Android devices.

With the Android app it stops working as soon as a second profile is connected with completely different user data, which can never get in each other's way.

I very much hope that there will soon be a solution to the problem.

[CoffeeHT]

I am now convinced that it is not a problem in Joplin, but a problem in the NextClouds that I use.
Since I don't have any more time to report a problem to NextCloud, I work around this problem by spending money and using suitable JoplinCloud accounts. So far I haven't had any problems with multiple profiles in Android. So that I can have several profiles in JoplinCloud, I don't use my own email address, but redirect email addresses set up specifically for this purpose. One for each profile plus one for controlling a JoplinCloud team that can't have its own profile.

Translated with DeepL.com (free version)

[CoffeeHT]

Update to NextCloud 27.1.5 did not fix the problem

Update to NextCloud 27.1.9 did not fix the problem

[leachimus]

I don't think it's Nextcloud, because there are no problems on iOS. Multiple profiles are also synchronized in the desktop app without any problems.

It's a bug in the Android app and I'm surprised that so few people are interested in this problem.

I would like to see the problem addressed.

[CoffeeHT]

I would like to see the problem addressed.

That also applies to me. My next attempt will start there:
https://discourse.joplinapp.org/t/nextcloud-sync-fails-on-all-but-the-default-profile/38515/9?u=coffee

[leachimus]

Thanks for the post, but even there I don't see that the problem is understood.

It is not a Nextcloud problem. Apps on iOS do not have the described problem with multiple profiles on the same Nextcloud. Where Android does not synchronize and shows the error already mentioned here, it works fine on iOS.

Have you tested this with the Joplin server?

[laurent22]

Hello I'm closing this issue as it's unclear what it is about. If you still have an issue please create a new one and provide the requested information