[REGRESSION] Suggestions are not diacritics insensitive anymore

[DerK777]

Hello. I Have some problems with kiwix. Software version 2.2.1 Windows Desktop Russian. File Wikipedia russian 04.2022. But all older versions had the same problems. Please help to find a solution.

1. When I write in the search line, for example, "Тайфун" (Typhoon), it didn't find it. It offers "полнотекстовый поиск" (full-text search) and then I get link to "Тайфун (значения)" (Typhoon (meanings)), and only then I can get to article "Тайфун" (Typhoon).
   
   
   But it works really good with the word "Ураган" (Hurricane), for example.
   
2. Search didn't work at all if I try to do it at second and other tabs. There is no any links, there is no "полнотекстовый поиск" (full-text search), nothing
   
   It didn't depend on language, it didn't find "Hurricane" too.
   

[kelson42]

@DerK777 Thank you for your bug report. In the future, please open one ticket per bug. The part (2) of your bug report is a know bug/limitation, see kiwix/kiwix-desktop#499.

[kelson42]

@DerK777 I confirm your first bug. It seems to be a problem with suggestions an diacritics. Hopefully as soon as we will have moved mwoffliner to the latest libzim7, this bug will simply vanish. See openzim/mwoffliner#1606

[DerK777]

suggestions an diacritics

Thanks, I got the point. I must write "таифун" instead of "тайфун".

[kelson42]

@mgautierfr @veloman-yunkan Suggestions seems not diacritics insensitive anymore indeed. I search for example été on the Wikipedia in French and get no suggestion as result, but if I type ete then I get them. There is exactly the same problem on kiwix-serve. This is a major regression AND I wonder how this can go through the unit test without been noticed! We have to fix this before releasing the next maintenance/patched release of libkiwix.

[veloman-yunkan]

This is a major regression AND I wonder how this can go through the unit test without been noticed!

@kelson42 We don't have such a unit-test for suggestions that checks handling of diacritics. Also, while trying to add one (as the first step toward resolving this issue), I ran against the problem that the ZIM files used in our libkiwix unit tests don't have any articles with diacritics in the titles.

[kelson42]

@veloman-yunkan If is it necessary to implement #601 fo get this feature properly tested then we will have to do that in TOP priority IMO. But, at the level of maturity we have now in libkiwix, such a regression is for me a bad surprise. Correlated to #722 and #742, this let me think a strong and quick push in searching features automated testing is necessary.

[veloman-yunkan]

@veloman-yunkan If is it necessary to implement #601 fo get this feature properly tested then we will have to do that in TOP priority IMO. But, at the level of maturity we have now in libkiwix, such a regression is for me a bad surprise. Correlated to #722 and #742, this let me think a strong and quick push in searching features automated testing is necessary.

The broken feature now has a unit test for the underlying component in libzim, however without proper testing on the libkiwix level we cannot be sure that it works correctly. I believe that #601 is an important task and the sooner we resolve it the better.