inaccessible folders/scanning issues [SOLVED - WITH CAVEATS]

[rumremix]

I did see a few posts with similar error (TransientDataAccessResourceException), but different symptoms. So starting a new thread here as the issue may be unrelated.

Problem Description: When I try to access some folders in the web interface (via "index" tab) I get an error message (see below). Other folders that contain albums do not present an error but simply show no albums. In both cases, the folders are accessible via OS folder navigation and the albums appear as expected. Other folders appear correctly in the web interface with all the albums as expected. I upgraded to 6.1.1 (build fc853b) some time ago but I first noticed this new issue just last week.
I'm noticing one other odd thing related to library scanning but I'm not sure if it's related. (1) When I manually scan, the "scanning files: *number scanned so far*" message that appears to the right only gets up to 100+ and then vanishes a moment after I start, as though the scan is complete. I have 1.75TB of music so I'm accustomed to it taking longer. (2) I noticed that immediately after ripping a new album, the new album is available in subsonic! This is odd because I have it scheduled to scan for new files every day at 1am. Is there a new feature in the latest subsonic version that continuously scans for new music? If so, why would the schedule setting still live in settings? From the log (shown below) it looks like scanning is failing. But odd that newly added files are both visible and playable right away, even without a manual scan (o waiting for the daily scan).
Troubleshooting Steps: I stopped the subsonic server and restarted it.
Playback Application and version:Subsonic web interface (error message appears), Sonos, AVSub
Subsonic Version: 6.1.1 (build fc853b) – May 31, 2017
Server Version: jetty-6.1.x, java 1.8.0_45, HSQLDB, Windows 8 (83.7 MB / 145.0 MB)
Hardware Platform: Windows 8
Java Memory Limit: 150
Last ten log file lines:

Code: Select all

[7/24/17 8:05:51 PM PDT]   INFO   PlaylistService   Completed playlist import.
[7/24/17 8:37:50 PM PDT]   INFO   MediaScannerService   Starting to scan media library.
[7/24/17 8:37:50 PM PDT]   ERROR   MediaScannerService   Failed to scan media library.
[7/24/17 8:37:51 PM PDT]   INFO   PlaylistService   Starting playlist import.
[7/24/17 8:37:51 PM PDT]   INFO   PlaylistService   Completed playlist import.
[7/24/17 8:38:13 PM PDT]   INFO   TranscodeInputStream   Starting transcoder: c:\subsonic\transcode\ffmpeg -i M:\NewRips\Lossless\fresh\Various\Music from the Lost Provinces (Old-Time Stringbands from Ashe Cty NC, 1927-1931)\03 Ephraim Woodie & the Henpecked Husbands Last Gold Dollar [1929].flac -map 0:0 -b:a 320k -v 0 -f mp3 -
[7/24/17 8:39:21 PM PDT]   INFO   MediaScannerService   Starting to scan media library.
[7/24/17 8:39:21 PM PDT]   ERROR   MediaScannerService   Failed to scan media library.
[7/24/17 8:39:22 PM PDT]   INFO   PlaylistService   Starting playlist import.
[7/24/17 8:39:22 PM PDT]   INFO   PlaylistService   Completed playlist import.

And here is an example of an error message I'm seeing when I try to navigate to an unavailable folder in the subsonic web interface:

Code: Select all

Exception   org.springframework.dao.TransientDataAccessResourceException

Message   PreparedStatementCallback; SQL [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]; S1000 General error java.util.NoSuchElementException in statement [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]; nested exception is java.sql.SQLException: S1000 General error java.util.NoSuchElementException in statement [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]

Java version   Oracle Corporation 1.8.0_45
Operating system   Windows 8 6.2
Server   jetty-6.1.x
Memory   Used 68 of 145 MB
Stack trace   org.springframework.dao.TransientDataAccessResourceException: PreparedStatementCallback; SQL [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]; S1000 General error java.util.NoSuchElementException in statement [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]; nested exception is java.sql.SQLException: S1000 General error java.util.NoSuchElementException in statement [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]
   at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:106)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
   at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:607)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:641)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:670)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:678)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:710)
   at net.sourceforge.subsonic.dao.AbstractDao.query(AbstractDao.java:84)
   at net.sourceforge.subsonic.dao.MediaFileDao.getChildrenOf(MediaFileDao.java:68)
   at net.sourceforge.subsonic.service.MediaFileService.getChildrenOf(MediaFileService.java:187)
   at net.sourceforge.subsonic.service.MediaFileService.getChildrenOf(MediaFileService.java:164)
   at net.sourceforge.subsonic.controller.MainController.handleRequestInternal(MainController.java:73)
   at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
   at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
   at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
   at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
   at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
   at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
   at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:25)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.ParameterDecodingFilter.doFilter(ParameterDecodingFilter.java:36)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.BootstrapVerificationFilter.doFilter(BootstrapVerificationFilter.java:41)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
   at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
   at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
   at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
   at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
   at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
   at org.mortbay.jetty.Server.handle(Server.java:313)
   at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
   at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
   at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514)
   at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
   at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
   at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
   at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: java.sql.SQLException: S1000 General error java.util.NoSuchElementException in statement [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where parent_path=? and present]
   at org.hsqldb.jdbc.Util.throwError(Unknown Source)
   at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(Unknown Source)
   at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:648)
   at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:591)
   ... 62 more

Any suggestions on the best way to proceed would be greatly appreciated. Thanks!

[Exrace]

Have you moved files around recently?
Some have reported if the dir date doesn't change but the dir moves the scan looks to have a problem with updating existing db records for the same files.
You can try creating a new dir and move the files there or use a command line dos or powershell to "touch" the file.
Goggle "windows equivalent to touch command" for some ideas.

[rumremix]

Thanks for the ideas, Exrace.

Yes, I do move files around. I do this regularly as I preview newly ripped or downloaded music before I move it into my primary index. I've been doing this routinely for many years. If this is the cause it seems odd that it would not have created any problem before last week.

I can report that I have not recently moved any files within two specific directories that are now inaccessible from the web interface. Nor have I moved the directories themselves. So it seems unlikely that the moving of files or directories would have caused the issue that arose last week in those cases.

I tried your suggestion of creating a new directory (new sibling of a directory that is inaccessible) and moving some files from the inaccessible directory into the new one. Then I manually ran a scan (which aborted after just a moment, consistent with recent behavior). The result is that now the parent directory is inaccessible. Whereas before there were four accessible and two inaccessible folders within this parent directory. Now this parent directory is no longer accessible so all of the child folders are unavailable. Again, I have not moved any files within this parent directory in a very long time (aside from this test a moment ago).

Does this observation help illuminate the problem?

I also noticed some issues with port forwarding during the night (see log below). Seems that it failed every other hour. But I wouldn't expect this to be related.

Code: Select all

[7/25/17 1:41:30 AM PDT]   WARN   NetworkService   Failed to create port forwarding.
[7/25/17 2:41:30 AM PDT]   INFO   NetworkService   Successfully forwarding port 4040.
[7/25/17 3:41:38 AM PDT]   WARN   NetworkService   Failed to create port forwarding.
[7/25/17 4:41:38 AM PDT]   INFO   NetworkService   Successfully forwarding port 4040.
[7/25/17 5:41:38 AM PDT]   WARN   NetworkService   Failed to create port forwarding.
[7/25/17 6:41:38 AM PDT]   INFO   NetworkService   Successfully forwarding port 4040.
[7/25/17 7:41:38 AM PDT]   WARN   NetworkService   Failed to create port forwarding.
[7/25/17 8:41:38 AM PDT]   INFO   NetworkService   Successfully forwarding port 4040.
[7/25/17 9:41:38 AM PDT]   WARN   NetworkService   Failed to create port forwarding.


[rumremix]

Update to my last report...

Once again, four of the six folders in the noted parent directory are accessible again, and two are not (presenting the error message/stack trace instead). The newly created folder into which I moved files from one inaccessible folder is simply not appearing at all. (This is likely due to the failed scanning but does beg the question as to why some freshly ripped albums are available in subsonic with no scanning at all -- as mentioned in original post.)

Any further fix or troubleshooting suggestions greatly appreciated.

[rumremix]

For the sake of being thorough and precise, I should point out that I just ripped a new CD and it did *not* become immediately available in subsonic as did the ones I ripped and reported on yesterday (direct folder siblings of this new album). Manual scanning did not help (as expected). interesting that the two albums ripped yesterday became available even without either manual or daily scheduled scan but this one did not.

Again, I appreciate any advice in resolving these issues.

[rumremix]

I've still not resolved this issue but have attempted additional troubleshooting so wanted to report what I've since tried.

(1) I tried stopping the subsonic service, deleting the contents of my subsonic/db folder, then restoring a backed up subsonic.script file to that folder, and restarting subsonic (as per this instruction: https://wiki.cementhorizon.com/display/ ... c+database).

That method has worked for me in the past to resolve corrupted db issues but this time, after restarting subsonic I get a 503 error when I try to access the web interface. Apparently, this method doesn't work for version 6.1. Luckily I backed up all of the db folder contents to be safe. When I stopped the service again, restored that folder to its original state and restarted, I ended up right back where I was before the experiment. Issue still persists.

(2) I deleted the contents of the subsonic/db folder and resinstalled subsonic. I then opened the web interface (using default first-time login credentials) and tried to run a scan. Same result, scan aborts after just a moment. Not sure if deleting the contents of the db folder and re-installing subsonic is a proper fresh install, but it does sure look like the issue is not that my database is corrupted (this is not evidence that my data is not corrupted only that corrupted db is not the cause of my scanning problem).

Very much appreciate any other suggestions. When subsonic is unhappy, I am unhappy.

[rumremix]

I have solved the scanning issue by completely uninstalling and re-installing subsonic (rather than just re-installing). I can now do a full manual scan and all of the folders that were inaccessible via the web interface before (resulting in error noted above) are now once again accessible.

This was not the most elegant solution though because I had to reconfigure all of my settings and lost all my playlists and "starred" songs and albums that I have been accumulating for years. I did save my old data files but transferring them to the new installation is not as simple as I had hoped. I will ask this forum for suggestions on a better way to do that in a new thread (viewtopic.php?f=2&t=17519) as I'm hoping there is a faster way than exporting each playlist one at a time.

[Exrace]

Wonder if the cleandb function in web interface would fix this issue?

[rumremix]

That didn't help remove the error, unfortunately.