Page 1 of 1

Faster scanning for content

PostPosted: Tue Dec 22, 2015 11:54 pm
by BloodyIron
I use subsonic for my music, syndicated tv, movies, and podcasts. I have a LOT of data, and I'm pretty sure I'm not alone.

When I add an album, or some other new bit of media, I have to clean the database, and then scan for new media. This is a process that really needs to be streamlined.

Also, if we could get it so that the system just picks up on new media, without having to wait for the scheduled once-a-day scan, or having to manually scan, that would be great.

I'm thinking perhaps some form of directory hashing tree might be in order? I'm thinking the scanning or updating needs to be simple-first, then get the "complex" data when it detects changes.

1) Scan JUST for new folders, this should be dead simple scanning. If it picks up nothing, goto 2
1b) If new folders are detected, scan their contents, if multimedia is found, add to catalogue, etc.
2) Scan JUST for new files, again, just scan for the presence of new files.
2b) If new files are found, add to catalogue, etc.
3) Scan for folder or file CHANGES, maybe hashing can handle name and size changes, I'm not sure.
3b) Re-catalogue changed stuff.

To re-iterate, I would like to see subsonic handle new media without any user interaction. You put it in a folder it manages, and omg it already knows a change happened, and it's starting to manage it. Then you can use it right away.

I know this will be a careful balance between resource usage, and functionality, but I think revisiting this is worthwhile.

Please? :)

Re: Faster scanning for content

PostPosted: Wed Dec 23, 2015 1:16 am
by tranceConscious
Thumbs up on that.

I have 500000 music files, and scanning takes about 4 hours to finish...
And by the time it's finished, files have already been added or changed
so an emediate rescan is needed.

Maybe a "watch folder" solution?

Re: Faster scanning for content

PostPosted: Wed Dec 23, 2015 1:54 am
by BloodyIron
Crikey that's a lot of files!

I think at that case using hashing trees is probably one of the most efficient ways to do it, as hashing is very fast for large amounts of files. Although to be fair I don't know the method used currently.


tranceConscious wrote:Thumbs up on that.

I have 500000 music files, and scanning takes about 4 hours to finish...
And by the time it's finished, files have already been added or changed
so an emediate rescan is needed.

Maybe a "watch folder" solution?