New Install - Java "out of memory" Error [Resolved]

[phrend]

I just installed Subsonic a couple days ago, and it's still performing it's initial media scan (network share - it's up to 79k files thus far). I got the following error when clicking on, "Recently Added" in the web interface... does that mean that I need to dedicate more RAM to Subsonic? I'm planning on trying again, once the initial scan is complete, but figured I'd ask now. (Not sure if the scan takes up lots of memory, or what?)

Thanks in advance!

Code: Select all

Exception   org.springframework.dao.TransientDataAccessResourceException
Message   PreparedStatementCallback; SQL [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where type=? and present order by created desc limit ? offset ?]; out of memory; nested exception is java.sql.SQLException: out of memory
Java version   Sun Microsystems Inc. 1.6.0_27
Operating system   Windows 7 6.1
Server   jetty-6.1.x
Memory   Used 144 of 145 MB
Stack trace   org.springframework.dao.TransientDataAccessResourceException: PreparedStatementCallback; SQL [select id, path, folder, type, format, title, album, artist, album_artist, disc_number, track_number, year, genre, bit_rate, variable_bit_rate, duration_seconds, file_size, width, height, cover_art_path, parent_path, play_count, last_played, comment, created, changed, last_scanned, children_last_updated, present, version from media_file where type=? and present order by created desc limit ? offset ?]; out of memory; nested exception is java.sql.SQLException: out of memory
   at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:106)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
   at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:607)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:641)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:670)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:678)
   at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:710)
   at net.sourceforge.subsonic.dao.AbstractDao.query(AbstractDao.java:83)
   at net.sourceforge.subsonic.dao.MediaFileDao.getNewestAlbums(MediaFileDao.java:214)
   at net.sourceforge.subsonic.service.MediaFileService.getNewestAlbums(MediaFileService.java:271)
   at net.sourceforge.subsonic.controller.HomeController.getNewest(HomeController.java:159)
   at net.sourceforge.subsonic.controller.HomeController.handleRequestInternal(HomeController.java:95)
   at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
   at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
   at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
   at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
   at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
   at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
   at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.ResponseHeaderFilter.doFilter(ResponseHeaderFilter.java:47)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:43)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.ParameterDecodingFilter.doFilter(ParameterDecodingFilter.java:54)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.BootstrapVerificationFilter.doFilter(BootstrapVerificationFilter.java:54)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
   at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
   at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
   at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
   at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
   at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
   at org.mortbay.jetty.Server.handle(Server.java:313)
   at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
   at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
   at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514)
   at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
   at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
   at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:227)
   at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:626)
   at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Caused by: java.sql.SQLException: out of memory
   at org.hsqldb.jdbc.Util.throwError(Unknown Source)
   at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(Unknown Source)
   at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:648)
   at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:591)
   ... 65 more

[phrend]

For the record, I installed:
4.7.beta2 (build 3012) – June 9, 2012
on a Windows 7 box.

The initial scan is complete now, but it's reporting that it found:
658,753 songs
3654.61 GB

Those values can't be correct! I've only got one media folder configured (my iTunes/iTunes Media/Music folder), and according to iTunes, there are 60,798 music files and 62,304 total files in my library. (...and the media is just under 350GB) There are 66,283 files according to, "find iTunes/iTunes\ Media/Music -type f -print | wc -l" - it looks like some of the folders contain jpg's and other crap like desktop.ini, etc., which accounts for the difference.

I don't know if the numbers Subsonic is reporting are a bug specific to the beta version, or if something else is going on?

At any rate - after finishing the scan, I brought up the Subsonic control window and noticed that the memory use was spiking up to around 135MB, so I increased the max memory up to 200MB to give it a little extra headroom - hope that does the trick!

[BKKKPewsey]

Sindre has found some problems with the beta regarding the scan and the database (now been fixed but not released).
There does appear to be something very wrong with your db - I would suggest installing the last stable release (4.6)
Do a clean install (delete c:\subsonic first) and rescan your media.
Let us know if the files found are still "silly" (note 4.6 is much quicker on scanning so you will not have too long to wait)
If that goes ok I would then wait for next 4.7 update before upgrading but that's your call.

[phrend]

Thanks for your suggestions - I wiped out my installation and installed 4.6. It finished the initial scan in a matter of minutes! I can already see some of the 4.7 features that are missing, and am looking forward to it coming out of beta!

[phrend]

Update - it looks like the scan didn't actually finish... the list of artists did show up on the left side of the screen (web browser), so I thought it was done, but there was no total song count, etc. (not sure if that even exists in 4.6?)... anyway, it didn't appear to have created a search index (I wasn't able to search for anything yet), so I went to Settings > Search > Update search index now. and went to sleep... it has been at least 12 hours and I'm still getting, "The media folder is now being scanned. Some features are not available." at the top of the screen... maybe it doesn't like something in my iTunes music folder and it getting hung up?

The log in the About section shows:

Code: Select all

[7/14/12 9:54:22 PM PDT]    INFO   SearchService   Created search index with 229000 entries.
[7/14/12 9:57:28 PM PDT]    INFO   SearchService   Created search index with 230000 entries.
[7/14/12 10:03:51 PM PDT]    INFO   SearchService   Created search index with 231000 entries.
[7/14/12 10:08:40 PM PDT]    INFO   SearchService   Created search index with 232000 entries.
[7/14/12 10:12:15 PM PDT]    INFO   SearchService   Created search index with 233000 entries.
[7/14/12 10:16:20 PM PDT]    INFO   SearchService   Created search index with 234000 entries.
[7/14/12 10:19:58 PM PDT]    INFO   SearchService   Created search index with 235000 entries.
[7/14/12 10:23:28 PM PDT]    INFO   SearchService   Created search index with 236000 entries.
[7/14/12 10:27:19 PM PDT]    INFO   SearchService   Created search index with 237000 entries.
[7/14/12 10:30:38 PM PDT]    INFO   SearchService   Created search index with 238000 entries.
[7/14/12 10:34:10 PM PDT]    INFO   SearchService   Created search index with 239000 entries.
[7/14/12 10:37:30 PM PDT]    INFO   SearchService   Created search index with 240000 entries.
[7/14/12 10:41:38 PM PDT]    INFO   SearchService   Created search index with 241000 entries.
[7/14/12 10:45:18 PM PDT]    INFO   SearchService   Created search index with 242000 entries.
[7/14/12 10:48:55 PM PDT]    INFO   SearchService   Created search index with 243000 entries.
[7/14/12 10:52:22 PM PDT]    INFO   SearchService   Created search index with 244000 entries.
[7/14/12 10:55:53 PM PDT]    INFO   SearchService   Created search index with 245000 entries.
[7/14/12 10:59:56 PM PDT]    INFO   SearchService   Created search index with 246000 entries.
[7/14/12 11:03:45 PM PDT]    INFO   SearchService   Created search index with 247000 entries.
[7/14/12 11:08:10 PM PDT]    INFO   SearchService   Created search index with 248000 entries.
[7/14/12 11:11:13 PM PDT]    INFO   SearchService   Created search index with 249000 entries.
[7/14/12 11:15:01 PM PDT]    INFO   SearchService   Created search index with 250000 entries.
[7/14/12 11:18:52 PM PDT]    INFO   SearchService   Created search index with 251000 entries.
[7/14/12 11:23:09 PM PDT]    INFO   SearchService   Created search index with 252000 entries.
[7/14/12 11:26:22 PM PDT]    INFO   SearchService   Created search index with 253000 entries.
[7/14/12 11:29:54 PM PDT]    INFO   SearchService   Created search index with 254000 entries.
[7/14/12 11:33:38 PM PDT]    INFO   SearchService   Created search index with 255000 entries.
[7/14/12 11:37:24 PM PDT]    INFO   SearchService   Created search index with 256000 entries.
[7/14/12 11:40:42 PM PDT]    INFO   SearchService   Created search index with 257000 entries.
[7/14/12 11:44:13 PM PDT]    INFO   SearchService   Created search index with 258000 entries.
[7/14/12 11:47:39 PM PDT]    INFO   SearchService   Created search index with 259000 entries.
[7/14/12 11:51:30 PM PDT]    INFO   SearchService   Created search index with 260000 entries.
[7/14/12 11:54:56 PM PDT]    INFO   SearchService   Created search index with 261000 entries.
[7/14/12 11:58:32 PM PDT]    INFO   SearchService   Created search index with 262000 entries.
[7/15/12 12:02:12 AM PDT]    INFO   SearchService   Created search index with 263000 entries.
[7/15/12 12:05:35 AM PDT]    INFO   SearchService   Created search index with 264000 entries.
[7/15/12 12:09:02 AM PDT]    INFO   SearchService   Created search index with 265000 entries.
[7/15/12 12:13:02 AM PDT]    INFO   SearchService   Created search index with 266000 entries.
[7/15/12 12:16:38 AM PDT]    INFO   SearchService   Created search index with 267000 entries.
[7/15/12 12:20:08 AM PDT]    INFO   SearchService   Created search index with 268000 entries.
[7/15/12 12:24:12 AM PDT]    INFO   SearchService   Created search index with 269000 entries.
[7/15/12 12:28:06 AM PDT]    INFO   SearchService   Created search index with 270000 entries.
[7/15/12 12:31:46 AM PDT]    INFO   SearchService   Created search index with 271000 entries.
[7/15/12 12:35:13 AM PDT]    INFO   SearchService   Created search index with 272000 entries.
[7/15/12 12:38:53 AM PDT]    INFO   SearchService   Created search index with 273000 entries.
[7/15/12 12:42:17 AM PDT]    INFO   SearchService   Created search index with 274000 entries.
[7/15/12 12:46:13 AM PDT]    INFO   SearchService   Created search index with 275000 entries.
[7/15/12 12:50:06 AM PDT]    INFO   SearchService   Created search index with 276000 entries.
[7/15/12 12:52:11 AM PDT]    INFO   PlayerDao   Created player 4.
[7/15/12 12:57:53 AM PDT]    INFO   SearchService   Created search index with 277000 entries.

I don't know exactly what that means, but if each entry is supposed to be a media file (song), then something is wrong... as I mentioned before, I've only got around 61,000 music files.

I did get a java update message when I was installing 4.6 last night, and ended up installing java 7 - which may be another problem, since I just noticed a thread about Subsonic not being compatible with it... bleh - this is turning out to be a lot more trouble that I thought it would be.

[BKKKPewsey]

Check your media folder destination - it looks like SS is possibly trying to scan your whole drive?
I do not use iTunes but is there anything in the folders eg cache thumbnails etc that might be causing SS problems?
60k songs I would expect to be scanned in about 2 hrs for an initial scan and about 20 mins for a re-scan so something is certainly not right there.

[phrend]

iTunes does have other directories that have a bunch of cached cover-art, etc., but I pointed Subsonic directly to the folder that contains just the music (and a few folder.jpg and desktop.ini files left over from Windows, doing it's thing):
\\NAS\Media\iTunes\iTunes Media\Music

According to OS X's/GNU's "find" utility there are 62,304 total files in the directory. I'll poke around and make sure there aren't any simlinks or anything else funny in there... but I don't know if Samba would even support them, if they were in there... and iTunes shouldn't be doing anything like that anyway.

I also pointed Subsonic at an empty directory on local disk to be used for uploads:
e:\Subsonic_Uploads

I can remove that to narrow things down a bit.

Is there a setting somewhere to increase the log level so I can get more details on what's happening?

[phrend]

Don't ask me how it got there, but there was exactly one simlink nested in there, and it pointed back to the root "Music" directory (so it was circular)...

I'll kill everything off again and start over - if all goes well, I'll try the beta version again!

Sorry for the trouble!

[phrend]

Just to close the loop on all this... I uninstalled Subsonic, blew away c:\subsonic, and re-installed 4.6, allowed it to scan my library, and there were still problems... the page wouldn't fully load for minutes (mostly it seemed like the bar on the left was lagging). So, I decided to give it another shot tonight... I removed Subsonic 4.6 and Java, and re-installed 4.7b2 and allowed it to install Java 6.x. I bumped up the max memory to 200MB and restarted the service. I configured my media source and forced a scan... a couple hours later - scan is complete, and the page loads properly now... Android client is also working. Not sure what fixed it at this point, but it seems happy now. (Service is currently using 138MB of memory.) If it continues to work, I may actually register!