Subsonic is not playing well wtih S3FS

[AccidentallyTheCable]

I created an S3 bucket, and setup S3FS, I am able to mount the bucket, and work with it, even as the subsonic user (su - subsonic -s /bin/bash; /etc/passwd has the user as /bin/false). The server can even 'see' all of the files during scans, but at the end of the scan, theres nothing in my library.

I havent figured out how to turn on debug logging in the new version (I wasnt using s3 on the old one, I moved hosts, and in the process upgraded to the latest subsonic)

Code: Select all

[2017-03-11 18:20:19,816] INFO MediaScannerService - Scanned media library with 25250 entries.
[2017-03-11 18:20:23,091] INFO MediaScannerService - Scanned media library with 25500 entries.
[2017-03-11 18:20:26,050] INFO MediaScannerService - Scanned media library with 25750 entries.
[2017-03-11 18:20:29,950] INFO MediaScannerService - Scanned media library with 26000 entries.
[2017-03-11 18:20:34,181] INFO MediaScannerService - Scanned media library with 26250 entries.
[2017-03-11 18:20:38,344] INFO MediaScannerService - Scanned media library with 26500 entries.
[2017-03-11 18:20:42,447] INFO MediaScannerService - Scanned media library with 26750 entries.
-[2017-03-11 18:20:45,109] INFO MediaScannerService - Scanned media library with 27000 entries.
[2017-03-11 18:20:48,074] INFO MediaScannerService - Scanned media library with 27250 entries.
[2017-03-11 18:20:50,276] INFO MediaScannerService - Scanned media library with 27432 entries.
[2017-03-11 18:20:50,276] INFO MediaScannerService - Marking non-present files.
[2017-03-11 18:20:51,078] INFO MediaScannerService - Marking non-present artists.
[2017-03-11 18:20:51,080] INFO MediaScannerService - Marking non-present albums.
[2017-03-11 18:20:51,124] INFO MediaScannerService - Completed media library scan.
[2017-03-11 18:20:51,462] INFO PlaylistService - Starting playlist import.
[2017-03-11 18:20:51,462] INFO PlaylistService - Completed playlist import.
[2017-03-11 20:18:53,962] INFO PodcastService - Starting scheduled Podcast refresh.
[2017-03-11 20:18:53,963] INFO PodcastService - Completed scheduled Podcast refresh.


When I look in the library within subsonic, nothing, theres no songs or artists, it does however show the (correct) totals on the help page:

Code: Select all

Media collection
   736 artists
   2,403 albums
   24,654 songs
   275.30 GB
   2,529 hours


I suspect that this has something to do with the fact that S3 does not actually make directories with a 'size', so when you do an ls -alh on the s3fs mount, all directories show as 0 size, though they have data in them:

root@media01:/srv/s3-media/music# groups subsonic
subsonic : storage
root@media01:/srv/s3-media/music# tree -hugp Sexy Kill Device/
Sexy Kill Device/
└── [drwsrws--- root storage 0] Electric Dandies
├── [-rw-rw---- root storage 6.1M] 01 - Ono - Sendai Cyberspace 7.mp3
├── [-rw-rw---- root storage 9.9M] 02 - Electric Daindies (Back 2 The Future).mp3
├── [-rw-rw---- root storage 10M] 03 - Netrunners (Not forgive.Not forget).mp3
├── [-rw-rw---- root storage 9.0M] 04 - Rebel 2.0 (NewGen Outlaws).mp3
├── [-rw-rw---- root storage 8.9M] 05 - G-Head (SimStim Technologies.World WarGame).mp3
├── [-rw-rw---- root storage 8.9M] 06 - Skyrider.mp3
├── [-rw-rw---- root storage 13M] 07 - Religion Plastic B3ta (Empathy Box 2.0).mp3
├── [-rw-rw---- root storage 8.5M] 08 - Small Heroes (Press Start to Continue).mp3
├── [-rw-rw---- root storage 8.7M] 09 - Neotek Lords (Console Cowboy's).mp3
├── [-rw-rw---- root storage 12M] 10 - Desert Of the Real (More Human Than Human).mp3
└── [-rw-rw---- root storage 6.2M] 11 - Wintermute.mp3

1 directory, 11 files

[alphawave7]

Not sure this is the solution, but a recent development in later versions: admin has to give himself folder/file permissions in settings. Make sure that is done, as what you describe with proper scan data yet no library displayed is a result of this setting. Off topic, but I would LOVE a tutorial on how you have achieved S3 Bucket server and the costs involved...my collection is about double the size of yours, and I access perhaps 2-3 times a week, owing to Google Music over LTE when I commute, so mostly Subsonic at home. Cheers!

[daneren2005]

I've also thought of doing that so I'm very curious what the costs involved are as well.

[AccidentallyTheCable]

OMG alphawave7 you're the man, that totally fixed it. I wasnt aware that was a thing in the new version.

Well, problem solved then!

A quick write up on my setup:

Linode 4GB
AWS S3 using S3FS-Fuse - v1.80
Subsonic 6.0

Its as easy as installing s3fs and configuring it, then pointing a media folder at that location in subsonic. Make sure your permissions match (and you may have to run: find -exec touch{} \;; before rescanning for the first time). Speed is definitely not a problem, much faster than my old physical hardware setup.

I also looked at s3backer, but I had problems getting it to work, I would definitely consider using this though as it makes a disk-as-a-file setup, so you could have one bucket and many s3 'disks', where s3fs is 1 bucket = 1 mount point

As far as costs, I havent gotten my first S3 bill yet, but based on their cost calculator, its only about $5-8/mo for 500GB storage. My first bill will probably be a bit high since I was re-scanning, and doing all kinds of stuff to try to get this to work.

I pay for subsonic premium, since I use the android app a lot while out and about, totally worth it.

<3 subsonic