problem with updating library

General discussions.

Moderator: moderators

problem with updating library

Postby pro2oman » Fri Dec 14, 2012 1:50 am

I've been having problems with updating the library. new files are never shown or found, and yes i have run a library update/scan/refresh

from what i have found it seems that subsonic looks for directory timestamps to determine if files in a dir have been added/modified
now my problem is that i use mhddfs to pool my hard drives into one, and by doing this, my dir timestamps are not updated with i change files within a dir.
does anyone know a fix for this? other then changing the timestamps on all my files manually?

is there a way to make mhddfs update the timestamps when files are added?

edit: Possibly a way to make subsonic check the files not the folders?
pro2oman
 
Posts: 4
Joined: Tue Oct 30, 2012 1:46 pm

Re: problem with updating library

Postby GJ51 » Fri Dec 14, 2012 4:18 am

Uncheck all media folders.

Rescan with no folders selected.

Clean-up database.

Recheck media folders.

Rescan
Gary J

http://bios-mods.com
http://www.maplegrovepartners.com
http://theaverageguy.tv/category/tagpodcasts/cyberfrontiers/
User avatar
GJ51
 
Posts: 3492
Joined: Wed Oct 20, 2010 11:58 pm
Location: Western New York

Re: problem with updating library

Postby hakko » Fri Dec 14, 2012 8:46 am

I guess you could run something like find / -type f -exec touch "{}" \; to set a new file date for all files on your whole drive. :!: :!: (use with caution)

In my Subsonic mod (viewtopic.php?f=11&t=10587), I keep track of changed files by looking at file modification date + file size. If that changes, the file is re-read. It seems to work well.
MusicCabinet developer
hakko
 
Posts: 1416
Joined: Tue Apr 17, 2012 7:05 pm
Location: Sweden

Re: problem with updating library

Postby GJ51 » Fri Dec 14, 2012 9:19 am

hakko wrote:I guess you could run something like find / -type f -exec touch "{}" \; to set a new file date for all files on your whole drive. :!: :!: (use with caution)

In my Subsonic mod (viewtopic.php?f=11&t=10587), I keep track of changed files by looking at file modification date + file size. If that changes, the file is re-read. It seems to work well.


That's a possible I guess, but then you lose the "New" feature. When I use the method I outlined I don't lose much.
Gary J

http://bios-mods.com
http://www.maplegrovepartners.com
http://theaverageguy.tv/category/tagpodcasts/cyberfrontiers/
User avatar
GJ51
 
Posts: 3492
Joined: Wed Oct 20, 2010 11:58 pm
Location: Western New York

Re: problem with updating library

Postby hakko » Fri Dec 14, 2012 9:23 am

True. I think of both methods as work-arounds (if you have a massive library, you probably don't want to re-scan it like that every time you add a single song, running a script is more automated). But of course, I prefer having a Subsonic server that detects new files without user intervention...

What about playlists in Subsonic 4.7? Do you lose them when cleaning the db?
MusicCabinet developer
hakko
 
Posts: 1416
Joined: Tue Apr 17, 2012 7:05 pm
Location: Sweden

Re: problem with updating library

Postby GJ51 » Fri Dec 14, 2012 9:32 am

Yeah, I think so, but I don't really use them much. It's more important for me to see the new panel as that's what I listen to most, Everything else is easy enough to find. I've got 316GB if music plus about 4TB of video and although I know it's not the largest collection around, I've still never been to fussy about search or playlists.

Guess I'm just too simple.
Gary J

http://bios-mods.com
http://www.maplegrovepartners.com
http://theaverageguy.tv/category/tagpodcasts/cyberfrontiers/
User avatar
GJ51
 
Posts: 3492
Joined: Wed Oct 20, 2010 11:58 pm
Location: Western New York

Re: problem with updating library

Postby hakko » Fri Dec 14, 2012 9:37 am

Yeah, I don't have any video at all that I bother to stream but a lot of music that I'm very obsessed with. So I guess everyone's different! Some people have playlists they've worked endless hours on so it might be good to warn them anyway.
MusicCabinet developer
hakko
 
Posts: 1416
Joined: Tue Apr 17, 2012 7:05 pm
Location: Sweden

Re: problem with updating library

Postby pro2oman » Fri Dec 14, 2012 4:04 pm

i dont want to touch every file as that would make flexraid have to re backup every file...im going to try to use flex raid pooling instead of mhddfs, i believe it does dir timestamps correctly.. i just hope i dont mess up my data somehow..

and the clean library button never seemed to do anything... I have tried still didn't fined my new files
pro2oman
 
Posts: 4
Joined: Tue Oct 30, 2012 1:46 pm

Re: problem with updating library

Postby GJ51 » Fri Dec 14, 2012 8:57 pm

If you followed the procedure I outlined and files are still not showing, then try moving the folder to a temporary directory and then recopying back into your library. I've run into that scenario before and it seemed to fix the folders that weren't showing.

and the clean library button never seemed to do anything... I have tried still didn't fined my new files


Just hitting the clean library button is not going to work in your case. You must first uncheck the media folders, Save, rescan, clean db, rescan, recheck folders, and then rescan.

I have found that on all my sites once this has been done I have no further problems in this area as long as there are no unusual shutdowns or crashes of the host server. We add new material 2-3 times/week and then sync to a mirror library without any problems. This has been stable on our servers for several months now using regular Subsonic, Madsonic, Music Cabinet, and all other variants tested. Once you get it cleaned up properly it tends to work very well.

New additions show up after a rescan.
Gary J

http://bios-mods.com
http://www.maplegrovepartners.com
http://theaverageguy.tv/category/tagpodcasts/cyberfrontiers/
User avatar
GJ51
 
Posts: 3492
Joined: Wed Oct 20, 2010 11:58 pm
Location: Western New York

Re: problem with updating library

Postby pro2oman » Sat Dec 15, 2012 1:18 am

GJ51 wrote:If you followed the procedure I outlined and files are still not showing, then try moving the folder to a temporary directory and then recopying back into your library. I've run into that scenario before and it seemed to fix the folders that weren't showing.

and the clean library button never seemed to do anything... I have tried still didn't fined my new files


Just hitting the clean library button is not going to work in your case. You must first uncheck the media folders, Save, rescan, clean db, rescan, recheck folders, and then rescan.

I have found that on all my sites once this has been done I have no further problems in this area as long as there are no unusual shutdowns or crashes of the host server. We add new material 2-3 times/week and then sync to a mirror library without any problems. This has been stable on our servers for several months now using regular Subsonic, Madsonic, Music Cabinet, and all other variants tested. Once you get it cleaned up properly it tends to work very well.

New additions show up after a rescan.


I did do the other steps with the "clean" button, i understand its pointless with the library still there..
i ment that even deleting the folder form the library and then using the clean did nothing..
as soon as i re-added the folders to the library all the scanned files where back(only the old), so the clean did nothing..

and wouldn't moving the file be the same as using the touch command..
it would fix it now, but if the dates aren't updated then newer file added latter will still not be found latter on...
so moving everything every time i need to update the library is worse of then using "touch"
besides i don't have the room to move all the video,music ext..
pro2oman
 
Posts: 4
Joined: Tue Oct 30, 2012 1:46 pm

Re: problem with updating library

Postby bushman4 » Sat Dec 15, 2012 6:37 pm

You keep missing a step.

Remove the items from the library.

DO A LIBRARY SCAN! (this is the step you are missing) Even with NO media folders added... just do it.

THEN run the clean.

THEN re-add the media folders.

THEN do another library scan.

At the end of the library scan (with empty media folder settings) the scan process marks all files that it did not find in that scan as missing. The clean routine then actually DELETES the records that have been marked missing. Because you are not scanning in between, you have no records marked as missing.

Glenn
Glenn Sullivan
Subsonic 6.1.6 (Unraid Docker)
90 regular Subsonic Users

Library as of 2024-10-28:
4,527 artists
19,996 albums
282,151 songs
10201.40 GB
41,583 hours
User avatar
bushman4
 
Posts: 875
Joined: Thu Dec 02, 2010 1:47 pm
Location: Massachusetts, USA


Return to General

Who is online

Users browsing this forum: No registered users and 19 guests