Subsonic 6.1.4, library scanner messes up imperfect ID3 tags

[nasko]

I have been happily using Subsonic 6.0 Server and Subsonic App for few years. In addition to pretty folders (artist/album/songs...) I have bunch of mixed-bag folders having dozens of songs with imperfect ID3-tags and Subsonic 6.0 was doing very decent job in representing my multiple-level "Folder structure" - no songs/folders were excluded, everything was properly shown to the Subsonic App Client.

This suddenly changed when I upgraded to latest Subsonic 6.1.4 - some folders disappeared (individual songs are still searchable), other folders appeared with blank name and some folders seemingly got assigned random names.
(Yes, I know - fixing the ID3-tags would be best but it's not an option for me right now)

Worsening the "library-scanning" in latest Subsonic 6.1.4 makes no sense to me - whatever benefit was introduced, it might not be worth it given the pain/havoc new scanning caused to many users having folder-oriented libraries. And no - it doesn't have to be one or the other - Subsonic 6.0 proved that "ID3-tag oriented" libraries and "folder-structure" ones can properly coexist.

I am not sure if it's the Subsonic Server to blame or Subsonic App for Android - it all started when I upgraded the server.
On the other hand, the web interface of the server (the Index page) do show my folders/songs correctly. (DSub app is having the same issue as Subsonic App by t he way)
Is anybody else having these issues ? I doubt I am the only one.
I would hate to rollback to old version and switch to another server altogether.

I hope Sandre pays attention to this and make sure scanner-quality goes up, not down from one version to another.
If necessary, I could create a username for Sandre to connect his tools and see the problem firsthand - might be much more efficient than copy/pasting log files.

Help appreciated.

[nasko]

Ok, guys, did some more debugging:
I have 2 environments: old Subsonic 6.0 and new one 6.1.4 - both pointed to the same music folders. Then I issued a "getIndexes" API calls to compare:
Turns out that on bunch of "mixed-bag" folders:
6.0 Scanner properly assigns the folder-name as an "artist name" whereas
6.1.4 Scanner overagressively assigns a random "artist name" (inferred from some file in the folder). In other cases "artist name" is outright missing.

So, I guess best would be if the Scanner somehow detects that a giver folder is a "mixed-bag" folder and in this case assigns the folder-name as an "artist name" in the "getIndexes" API. If 6.0 can do it, 6.1.4 should be able to do it too, right ?

The Scanner can infer that this is a "mixed-bag" folder:
a) if #(distinct artists in the folder) > 2 (or whatever other configurable number)
OR
b) by coming across special-file ("MIXED-BAG") placed by the User in the folder.
This would eliminate most of the guesswork.

Sandre, what do you think ?
Thanks

[nasko]

Found bunch of exceptions thrown in the log. Can't make sense out of it - Sandre would have to take a look.

[sindre_mehus]

Thanks for reporting this, I will look into it!

Sindre

[sindre_mehus]

Just released 6.1.5 which should fix this bug.

Sindre

[nasko]

Upgraded to 6.1.5, did "Clean up database" and then "Scan media folders now".
Indeed, no more exceptions in the log but all other issues reported in the ticket - blank (or random) folder names in SubsonicApp) - still there.

Scan happened suspiciously fast (~8 seconds) for a library having 18,000 songs. May be library metadata has been cached from before the upgrade ?
If so, how can I force a scan from scratch ? (Removing the folder and adding it back is not very appealing since I would have to manually add it back to the 80 users currently having access to it.)

Thank you, Sindre

[sindre_mehus]

The bug was not related to the scanning or database, it was just in the API used by the Subsonic apps.

Please try to refresh the artist list in the app (or restart the app) to see if that helps.

[nasko]

Good news:
On a different machine Installed a fresh 6.1.5 (not upgrade) and all the "blank/random folder name" issues appear to be resolved.

However:
My production environment (which is an "in place upgrade to 6.1.5") appears to be stuck with library metadata from before the fix. "Cleanup database" followed by "Library scan" doesn't appear to be actually scanning the library from scratch ?
How to force a true full scan ?
Thank you Sindre
(Awesome software

[nasko]

Restarted the Subsonic6.1.5 service and tried "library scan" - again it was going at ~3000 songs per second - pretty sure it skipped over. SubsonicApp still showing blank/random folder names.

Though I am glad that at this point we only have to figure out how to force a full scan.

[sindre_mehus]

Ok, I figured it out.

The artist list is pretty aggressively cached in the API so please do the following to force a refresh in the Subsonic app:

1. Do some change in the server settings (e.g., change Settings > General > Articles to ignore)
2. Refresh the artist list in the app (e.g., for the Subsonic Android, click the refresh button in the Library tab)

[nasko]

Sindre, you nailed it !!! My folder names are back to normal on latest Subsonic6.1.5 )
Thanks !

In a summary, looks like
a) somehow my metadata got messed up when upgraded "in-place" my Subsonic6.0 (how did this happen, any clue ? just curious...)
and then
b) I got stuck with aggressively cashed artist-info (so following upgrades - 6.3,6.4,6.5 - didn't make any difference)
and then
c) your ingenious Settings-trick cleared the artist-cache

Pls consider adding "Clear artist-cache" checkbox next to "Scan media folders".