Lucene search index update crashes subsonic

[karvanapa]

Hello,

I've been lately having issues with subsonic stability. It seems that when new music is added or folders are renamed, updating lucene search index (manual or scheduled) causes subsonic to crash.

Sometimes I get java.lang.OutOfMemoryError errors to subsonic.log. I've tried to increase memory size and clearing lucene database (removed whole lucene db folder). But still no help. I have also tried reinstalling subsonic.

Latest crash was from today when I added couple of new artists and updated search index manually. Subsonic freezed when "INFO SearchService - Updating Lucene search index. " line appeared to the log. And couple of minutes after that I got "java.lang.OutOfMemoryError: Java heap space" error.

Subsonic Version:
4.4 (build 2169) – February 6, 2011

Server Version:
jetty-6.1.x, java 1.6.0_20, Linux (45.0 MB / 83.2 MB)

Hardware Platform:
Asrock ION 330 Running Ubuntu linux 10.04

Java Memory Limit:
SUBSONIC_MAX_MEMORY=100
I increased memory limit from 50 to 100 when subsonic started crashing.

Logs from latest crash:

Code: Select all

[2011-07-17 20:21:11,442] INFO SearchService - Created search index with 29000 entries.
[2011-07-17 20:21:20,469] DEBUG SearchService - Logically deleting info for album /data/Music/Deadmau5/2010 - At Play 3. Not found on disk.
[2011-07-17 20:21:20,484] INFO SearchService - Updating Lucene search index.
[2011-07-17 20:23:11,908] WARN JaudiotaggerParser - Failed to find cover art tag in /data/Music/Beach House/2010 - Teen Dream/01 Zebra.mp3
java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
        at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
        at org.jaudiotagger.audio.mp3.MP3File.readV2Tag(MP3File.java:185)
        at org.jaudiotagger.audio.mp3.MP3File.<init>(MP3File.java:391)
        at org.jaudiotagger.audio.mp3.MP3FileReader.read(MP3FileReader.java:39)
        at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:285)
        at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:148)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.getArtwork(JaudiotaggerParser.java:274)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.isImageAvailable(JaudiotaggerParser.java:251)
        at net.sourceforge.subsonic.service.MusicFileService.getBestCoverArt(MusicFileService.java:211)
        at net.sourceforge.subsonic.service.MusicFileService.listCoverArtRecursively(MusicFileService.java:191)
        at net.sourceforge.subsonic.service.MusicFileService.getCoverArt(MusicFileService.java:120)
        at net.sourceforge.subsonic.service.MusicFileService.getCoverArt(MusicFileService.java:92)
        at net.sourceforge.subsonic.ajax.NowPlayingService.convert(NowPlayingService.java:94)
        at net.sourceforge.subsonic.ajax.NowPlayingService.getNowPlaying(NowPlayingService.java:73)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:740)
        at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:744)
        at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:593)
        at org.directwebremoting.dwrp.BaseCallHandler.handle(BaseCallHandler.java:90)
        at org.directwebremoting.servlet.UrlProcessor.handle(UrlProcessor.java:120)
        at org.directwebremoting.servlet.DwrServlet.doPost(DwrServlet.java:141)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
        at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
        at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
        at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)


Sometimes there isn't much in the log when subsonic crashes:

Code: Select all

[2011-06-26 03:04:28,822] ERROR LuceneSearchService - Failed to create Lucene search index.
java.lang.OutOfMemoryError: Java heap space
[2011-06-26 03:32:00,832] INFO SearchService - Created search index with 28732 entries.

Does anyone have any suggestions how I can debug my subsonic installation further?

[karvanapa]

Still lucene update crashes subsonic. Any ideas anyone?

[sindre_mehus]

Does it always crash on the same file (/data/Music/Beach House/2010 - Teen Dream/01 Zebra.mp3)?

If not, try increasing the memory limit even more.

[karvanapa]

Does it always crash on the same file (/data/Music/Beach House/2010 - Teen Dream/01 Zebra.mp3)?

If not, try increasing the memory limit even more.

No, it's always different file. I increased memory limit to 250 MB. I will post here if this resolves the issue.

[karvanapa]

Now it seems that subsonic is stable. Lucene update seems to use very much memory. Sometimes memory usage is over 200MB for subsonic.

Now I'm having another strange issue after the lucene update. When I login I get error message popup "error". Log is full of these entries:

Code: Select all

[8/8/11 9:09:03 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:09 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:13 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:19 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:23 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:29 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:33 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:39 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:43 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx
[8/8/11 9:09:49 AM EEST]    WARN   FileUtil   Failed to list children for /data/Music/The xx/xx

This is one album I renamed before search index update. I can remove the error message by restarting subsonic, but who I can get subsonic to "forget" that old folder? Could this error also have something to do with the high memory usage?

[xaloy]

How do i increase the memory limit on debian for subsonic?

[BKKKPewsey]

I don't use Debian but does not the installation instructions tell you?

eg

Advanced configuration

To change the port number, Java memory settings or other startup parameters, edit the SUBSONIC_ARGS variable in /etc/default/subsonic. To restart Subsonic, do sudo service subsonic restart