Tracks are flagged non-present at the end of scan

[dkl]

Hello,

I have a strange issue in Subsonic. I'm using subsonic 6.1.6 (6.1.6 (build 0cfa60) – November 10, 2019) on a RPI4 since a (long) while without major issues.

Database is MariaDB.

I have recently discovered that, for an unknown reason, some Tracks (flac files) do not appear anymore in the Web UI. They exist in the DB and in the file system. I was able to play them in the past, but today, the Album appears empty in the UI.

This is true for all Tracks of the album, but the "cover" (jpg file) is visible.
I have "touched" the flac files, but this does not change anything.
All the albums (15 CD's) of the same directory have the same issue.


Trying to reach these tracks through the API raises the same issue: I cannot play them. This is thus not just a display issue.

Here is an example.

In the DB (I show only the 2 first tracks of the CD). It's a bit difficult to display long lines....

Code: Select all

SELECT * FROM media_file WHERE path like '/MNT/Music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1/%';
+--------+-------------------------------------------------------------------------------------------------+-----------------------+-------+--------+---------+-------+---------+--------------+-------------+--------------+------+-------+----------+-------------------+------------------+-----------+-------+--------+----------------+------------------------------------------------------------------------------------+------------+---------------------+---------+---------------------+---------------------+---------------------+-----------------------+---------+---------+
| id     | path                                                                                            | folder                | type  | format | title   | album | artist  | album_artist | disc_number | track_number | year | genre | bit_rate | variable_bit_rate | duration_seconds | file_size | width | height | cover_art_path | parent_path                                                                        | play_count | last_played         | comment | created             | changed             | last_scanned        | children_last_updated | present | version |
+--------+-------------------------------------------------------------------------------------------------+-----------------------+-------+--------+---------+-------+---------+--------------+-------------+--------------+------+-------+----------+-------------------+------------------+-----------+-------+--------+----------------+------------------------------------------------------------------------------------+------------+---------------------+---------+---------------------+---------------------+---------------------+-----------------------+---------+---------+
| 176788 | /MNT/Music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1/Track01.flac | /MNT/Music/Music/flac | MUSIC | flac   | Track01 | Orgue | JS Bach | JS Bach      |        NULL |         NULL | NULL | NULL  |      520 |                 1 |               52 |   3402154 |  NULL |   NULL | NULL           | /MNT/Music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1 |          2 | 2022-06-01 22:22:03 | NULL    | 2019-01-03 22:17:15 | 2022-06-01 22:00:43 | 2022-06-02 06:51:54 | 1970-01-01 01:00:00   |       0 |       5 |
| 176768 | /MNT/Music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1/Track02.flac | /MNT/Music/Music/flac | MUSIC | flac   | Track02 | Orgue | JS Bach | JS Bach      |        NULL |         NULL | NULL | NULL  |      585 |                 1 |              118 |   8628766 |  NULL |   NULL | NULL           | /MNT/Music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1 |          1 | 2022-06-01 22:22:50 | NULL    | 2019-01-03 22:17:15 | 2022-06-01 22:00:43 | 2022-06-02 06:51:54 | 1970-01-01 01:00:00   |       0 |       5 |
.....


In the file system, same "Track01" with its parent drectories:

Code: Select all

/media/music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1 $ ls -l
total 292168
-rwxr-xr-x 1 pi pi  3402154 Jun  1 22:00 Track01.flac
-rwxr-xr-x 1 pi pi  8628766 Jun  1 22:00 Track02.flac
.....

pi@raspberrypi:/media/music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet $ ls -l
total 4
drwxr-xr-x 17 pi pi 4096 Jul 29  2010 Orgue


pi@raspberrypi:/media/music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue $ ls -l
total 60
drwxr-xr-x 2 pi pi 4096 May  8  2015 CD1
drwxr-xr-x 2 pi pi 4096 May  8  2015 CD2
drwxr-xr-x 2 pi pi 4096 May  8  2015 CD3
.....


/media/music/Music/flac/Classical/03-Baroque/JS Bach/Orgue/Olivier Viernet/Orgue/CD1 $ ls -l
total 292168
-rwxr-xr-x 1 pi pi  3402154 Jun  1 22:00 Track01.flac
-rwxr-xr-x 1 pi pi  8628766 Jun  1 22:00 Track02.flac
.....


Maybe recreating everything would solve the issue, but any help would be appreciated to understand what is wrong here. And maybe how to solve this.

Denis

[dkl]

It looks like the "present" field is set to 0 for these Tracks. Shouldn't it be set to "1"?

[dkl]

Better. Performing a "touch" on the tracks AND on all parent directories seems to solve the issue (after rescan, of course). I have to make more tests to be sure....

I seems to me that a LOT of my flack files (approx 30%?) in the DB have the "present" field set to 0. And thus cannot be played nor displayed.

I have absolutely no idea why this is so.

Is there any way to set this back to 1 using SQL?

Or should I rebuild completely my DB?

[dkl]

Hi again...

Going further.... I have decided to rebuild the full DB (ie the Music Folder DB). This will take some time....

It's currently doing this, and after having scanned 45000 files (25% approx of the full DB), the result is encouraging:

Code: Select all

MariaDB [subsonic]> SELECT path FROM media_file WHERE present=0;
Empty set (0.001 sec)

MariaDB [subsonic]>



which means there are for the moment no files with "present=0". I had loads of them before.

So I'm waiting for the scan to finish, and i'll make more tests.

What I understand is that, for an unknown reason, a lot of music files (flac files) were tagged "not present" in the DB. Like if these files were not in the file structure, although I'm sure they were. And the corresponding cover files are not impacted.

In the file structure, these files were never corrupted, as far as I know.

I know Subsonic is an abandonware, I'm not waiting for any answer from an non exisitng dev team..... I'm just posting to help any other people who could see this issue in the future.

And if anybody could give me a hint and explain in which conditions this issue can occur, this would of course be great. Maybe it's ony a bug in Subsonic.

The fact is that I have changed recently my way of updating the DB. Instead of running a weekly update, I do it know only on purpose, when changes have been made to the file structures (mostly when new albums are entered), And I do it through a python API (pysonic). I don't know if this can be the cause of these troubles.

Denis

[dkl]

Bad surprise. At the end of the scan, I have finally again loads of flac files which cannot be played. Apparently, all of them have the "present" field equal to 0 (instead of 1).

The files exist in the file structure. They exist in the DB. But cannot be played.

It seems they are marked "non present" at the end of the scan:

Code: Select all

....
[6/6/22 8:41:06 PM CEST]   INFO   MediaScannerService   Scanned media library with 142000 entries.
[6/6/22 8:41:20 PM CEST]   INFO   MediaScannerService   Scanned media library with 142230 entries.
[6/6/22 8:41:20 PM CEST]   INFO   MediaScannerService   Marking non-present files.
[6/6/22 8:42:10 PM CEST]   INFO   MediaScannerService   Marking non-present artists.
[6/6/22 8:42:10 PM CEST]   INFO   MediaScannerService   Marking non-present albums.
[6/6/22 8:42:13 PM CEST]   INFO   MediaScannerService   Completed media library scan.
....


It looks to me that, at the end of scan, something is done (from [6/6/22 8:41:20 PM CEST] to [6/6/22 8:42:10 PM CEST]) during 50 seconds...

Most of the log traces I can see elsewhere show also this step "Marking non-present files", but with no duration.

Moreover, it seems that, each time, the full album is missing, except the cover file. I don't see any "single file" missing. It's always the full album. There must be a good reason somewhere....

Any help or hint appreciated.

[dkl]

Hi again,

Looks the issue comes from the DB implementation (MariaDB) or the integration with subsonic.
I switched back to the internal DB of subsonic, and it looks much better now. It's a bit less easy to make SQL queries and so on (specially write results of queries into files), but at least. all my music files are back, it seems....

I'll post here if I find one day the root cause of the problem.

Denis