Faster scanning for content

Got an idea? Missing something? Post your feature request here.

Moderator: moderators

Faster scanning for content

Postby BloodyIron » Tue Dec 22, 2015 11:54 pm

I use subsonic for my music, syndicated tv, movies, and podcasts. I have a LOT of data, and I'm pretty sure I'm not alone.

When I add an album, or some other new bit of media, I have to clean the database, and then scan for new media. This is a process that really needs to be streamlined.

Also, if we could get it so that the system just picks up on new media, without having to wait for the scheduled once-a-day scan, or having to manually scan, that would be great.

I'm thinking perhaps some form of directory hashing tree might be in order? I'm thinking the scanning or updating needs to be simple-first, then get the "complex" data when it detects changes.

1) Scan JUST for new folders, this should be dead simple scanning. If it picks up nothing, goto 2
1b) If new folders are detected, scan their contents, if multimedia is found, add to catalogue, etc.
2) Scan JUST for new files, again, just scan for the presence of new files.
2b) If new files are found, add to catalogue, etc.
3) Scan for folder or file CHANGES, maybe hashing can handle name and size changes, I'm not sure.
3b) Re-catalogue changed stuff.

To re-iterate, I would like to see subsonic handle new media without any user interaction. You put it in a folder it manages, and omg it already knows a change happened, and it's starting to manage it. Then you can use it right away.

I know this will be a careful balance between resource usage, and functionality, but I think revisiting this is worthwhile.

Please? :)
BloodyIron
 
Posts: 49
Joined: Wed Dec 18, 2013 7:51 pm

Re: Faster scanning for content

Postby tranceConscious » Wed Dec 23, 2015 1:16 am

Thumbs up on that.

I have 500000 music files, and scanning takes about 4 hours to finish...
And by the time it's finished, files have already been added or changed
so an emediate rescan is needed.

Maybe a "watch folder" solution?
tranceConscious
 
Posts: 15
Joined: Wed Dec 02, 2015 11:17 am

Re: Faster scanning for content

Postby BloodyIron » Wed Dec 23, 2015 1:54 am

Crikey that's a lot of files!

I think at that case using hashing trees is probably one of the most efficient ways to do it, as hashing is very fast for large amounts of files. Although to be fair I don't know the method used currently.


tranceConscious wrote:Thumbs up on that.

I have 500000 music files, and scanning takes about 4 hours to finish...
And by the time it's finished, files have already been added or changed
so an emediate rescan is needed.

Maybe a "watch folder" solution?
BloodyIron
 
Posts: 49
Joined: Wed Dec 18, 2013 7:51 pm


Return to Feature Requests

Who is online

Users browsing this forum: No registered users and 10 guests