Subsonic see only 1/3 of the media library

[gogogo]

Hello I run subsonic on a raspberry with the media folder in NFS from a nas.. it was working well for many month but now
I realized that don't show all the library.. when I perform a new scan .. the result is only 1/3 of the complete library.

The nfs permissions are correct, from the terminal the raspberry correctly show all the files that I don't see in subsonic.. also I have done on all the library a chomod -R 777 and also verified a user and group ownership with a big chown to be sure.


I see logs like this

Code: Select all

[7/21/19 10:39:13 PM UTC]   ERROR   RESTFilter   Failed to write error response.
[7/21/19 11:52:00 PM UTC]   WARN   RESTFilter   Error in REST API: /media/multimedia/o3765977.jpg (No such file or directory)
[7/22/19 2:13:51 AM UTC]   WARN   RESTFilter   Error in REST API: NullPointerException
[7/22/19 11:27:44 AM UTC]   ERROR   MediaScannerService   Failed to scan media library.
[7/22/19 11:27:45 AM UTC]   INFO   PlaylistService   Starting playlist import.
[7/22/19 11:27:45 AM UTC]   INFO   PlaylistService   Completed playlist import.

.....

What I can do?

Thank you.

[mbro]

I have a similar concern... Was running Subsonic on a Linux desktop (Debain) NFS mount to a freenas server locally. My issue is slightly different in that Subsonic indexes all my music successfully, but over time it seems to "forget" about a few songs. Example: an album has 18 songs, and all 18 were there and present, but I go look right now and it shows tracks 1-10, 18. What happened to 11-17? I look on the file system, and they are present, and there are no rights differences. So they were in the library and subsonic's database, but Subsonic has seemingly forgot about them. Subsequent media scans doesn't show them, tried disabling "fast access" mode, tried cleaning up the database. No difference. If I delete the database entirely and re-index the entire library fresh, those files will be present.

Seems to be something with the Subsonic database. Anyone else experienced something like that?

Marlen.

[gogogo]

The most aggressive and fast solution is trying to delete the db and create a new one

[mbro]

Yes, I've done that before, but thats not a fix as it will happen again. Also, when you delete and recreate the database, you loose all the history about what has been played, popular in your library, etc.

The solution is to tell the database not to randomly forget about individual tracks, or for the database to re-present those tracks again. First question, I guess, would be "why does it think that track doesn't exist when it absolutely does"...

[gogogo]

there is a backup directory that you can restore

[gogogo]

and .. after years of using subsonic..the best way is run it as LXC container under KVM hypervisor with automatic snapshot at the night and every day you will have a backup of all the system for restore in a click in any time...

raspberry etc.. are not good solutions.. can works for some months.. but will not reach one year of uptime and you will lose the DB...

is better to buy a little fanless pc x64 with proxmox installed.. or a windows 10 using the free embed virtualization

[mbro]

Interesting thought, but I think I can prove you wrong. I was running an old PC with Debian and Subsonic on a flash disk. Uptime was nearing 2 years when I finally experienced a power bump. The database on that one was also sporadically losing tracks that were still present, so I don't think its about the stability of the platform running subsonic, or how often it crashes (which is almost never). On that platform, I did also delete the database and recreate it, and experienced the same thing again over time. Seems to be albums that get more play time where that happens.

Still no solid explanation... And no good solution - backup/restore the database is nice in the event of a catastrophe, but the database shouldn't lose sight of tracks at all.

Thoughts on how to troubleshoot this?

[gogogo]

I don't know where you have the file system, but raspberry it's not stable for applications.
Better VPS. Life is better

[3Dscrewer]

Running Subsonic from a Raspberry is not the best idea, I was using a Raspberry 3B+ and it was no choice for me ....

A few suggestions
First of all, if your Database is on a Harddrive the Timeout of the USB Port might be to short. Crate and/or add to „/boot/config.txt“ the string „program_usb_timeout=1
One Shot one ...

Code: Select all

echo program_usb_timeout=1 | sudo tee -a /boot/config.txt

One Shot two

Code: Select all

sudo touch /boot/timeout

Next time Raspberry is booting it is adding the timeout himself.

Another suggestion and imho the best way is Berryboot. With Berryboot you need the sdcard only for booting and then it is running fully at the HDD or USB-Stick
But use also the suggestion above, because the startup of some HDD is simply to long. That's causing some problems
Have a look here ....
https://www.berryterminal.com/doku.php/berryboot

[mbro]

Those are great suggestions, however the core issue I was experiencing on the Raspberry Pi with Dietpi and Subsonic, I was also experiencing when it was running on a flash disk on a dedicated PC.

Root issue: why do tracks sometimes disappear from the database, and what do I need to do differently to ensure they don't...

[3Dscrewer]

I didn't say it clear, my opinion is the sdcard. I f you read the groups you can claerly assume that sdcards still causing problems. A remote diagnosis is still a problem too, but sdcards have a special behavior and a databank is not the first choice for sdcards. Most of them have a block size of 512bytes and written sequential to the card. Writing and deleting will cause a lots of gaps in the structure. In one word it is total different to a HDD. The Raspberry IO Chip has also a special design, doing a lot of things at the same time, USB and Network and so on. Another problem is the power source, I have had with Kodi a lot of squares at the screen showing that the power is not high enough, replacing the power source was very helpful, but during scan it was also a heat problem that was causing a freeze.
If you read the forum, your problem is not a common problem, so it feels like a problem of your installation and no one can say what the problem is.

[mbro]

You were clear, and makes sense. Why then, did I experience the same issue on a PC running Debian on an SSD. I completely understand what you're saying, but I've experienced the same issue on two COMPLETELY different platforms, one of which was a PC running Debian on an SSD, and should have had no issues at all.

Bottom line, thanks and I appreciate the points you raise, very valid, however the issue I have experienced on two different platforms with completely different technology has not been explained. Respectfully, if it was a problem with my installation (which is "dpkg -i subsonicxx.deb"), then maybe that would be a possibility, but I'm sure others have done the same with no issues. Could also be others have experienced a similar issue, just they haven't noticed it thus far... I think I got the installation right. How did you do yours?

[3Dscrewer]

I'm using subsonic at Windows 10 Pro 24/7 that is rock stable. I have so far 260,000 Titles and a lots of Videos with a total of nearly 10TB. I'm working every day at my database to have it clean, renaming the Filenames to my needs, adding the cover to the Tags, removing some parts of the Tags I don't need. Adding new Albums, so I have always heavy load at the database, without any problems.

I'm also using Airsonic, that is pretty the same as subsonic at an earlier stage with some modifications, there I have only the music base imported. There it is Ubuntu 18.04 LTS headless complex installation with apache2, SSL and reverse proxy, that is running from 8:00am to midnight. Also without problems.

I have also had airsonic running in Apache Tomcat at windows base, but the installation is still the same at Linux. Short steps install java, install tomcat, place the subsonic.war file into desired folder, start tomcat, call tomcat in browser and start subsonic.
BTW also SSD's are acting like large SDcards, simplified said But that should not be the issue, I'm using SSD's too.

EDIT: Some times it is usefull to set the time stamp of an album, folder and all files to actual date and rescan. I found the issue while editing some albums and it was still showing the old version in database, but after that it is not always showing the album as new. The reason is "getAlbumList2.view" and the same without 2 one is listing by Tag and the other by folder. So subsonic recognizes the changes but if the Tags are still the same, it is not new ...

[mbro]

A little update on the observation that I was seeing... I added some new material to my library, all was discovered. While I was playing some of that new material, the indexer kicked off, and took a good long while (indexing more new material). The album that had like 12 tracks, now only shows 3 of those tracks in Subsonic. All the files are still in the file system. I feel the indexing process is doing something in the database while a song is being played. I'm no DBA, so I couldn't outline exactly whats happening there, but this is the closest approximation to an explanation that I can come up with, because of this coincidental timing.

The subsonic.log shows this at the end of each indexing run:

[2021-02-19 10:54:24,312] INFO MediaScannerService - Marking non-present files.
[2021-02-19 10:54:42,189] INFO MediaScannerService - Marking non-present artists.
[2021-02-19 10:54:42,244] INFO MediaScannerService - Marking non-present albums.

How do I view the contents of what has been marked related to these entries? And, if something gets marked as "non-present", how do I poke it to say "hey, that file is there - look again"...