Tracks missing from search index

[ajkessel]

When searching Subsonic (either from the web interface or iPhone iSub client), lots of tracks don't show up in search results. I can navigate to them through the folders in the left pane; they play fine; and the metadata is fine. But they aren't being indexed.

I've found no pattern to which tracks are indexed and which are not, but the ones that are missing seem to remain missing.

Any suggestions for how to troubleshoot?

Running 4.4 (build 2169) / jetty-6.1.x, java 1.6.0_22, Linux (53.9 MB / 96.7 MB). (Debian Stable).

[GJ51]

Have you run a manual indexing?

Settings/Search - Update search index now.

[ajkessel]

Yes, I've done a manual reindex several times.

[Kirk]

I thought this was happening a few times on my Subsonic server too... but then I realized the metadata actually was screwed up for the tracks. lol.

Are you 100% sure?

Have you checked your Subsonic logs while the search index is being created, to look for any possibly relevant errors during the process?

Cheers,
Kirk

[ajkessel]

I noticed after 20+ times reindexing, some of the missing tracks appear or re-appear. So it does seem to be some problem with the indexing. I think the metadata should be fine, though -- the tracks look and play okay in both iTunes and Mediamonkey.

This might be the cause, from subsonic_sh.log. The machine has 1GB RAM, which I would think should be enough, although maybe this is a JVM specific issue:

Code: Select all

50905303 [btpool0-64] INFO org.apache.http.impl.client.DefaultHttpClient - Retrying request
Exception in thread "Search Index Generator" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3209)
        at java.lang.String.<init>(String.java:215)
        at java.lang.StringBuffer.toString(StringBuffer.java:585)
        at java.io.BufferedReader.readLine(BufferedReader.java:334)
        at java.io.BufferedReader.readLine(BufferedReader.java:362)
        at net.sourceforge.subsonic.service.SearchService.getIndex(SearchService.java:545)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:151)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)


There are also some suspicious errors in subsonic.log, e.g.:

Code: Select all

[2011-06-06 03:14:18,843] WARN JaudiotaggerParser - Error when parsing tags in [path to file.m4a]
org.jaudiotagger.audio.exceptions.CannotReadException: [path to file.m4a]:Index: 2, Size: 2
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:117)
        at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:285)
        at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:148)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.getRawMetaData(JaudiotaggerParser.java:71)
        at net.sourceforge.subsonic.service.metadata.MetaDataParser.getMetaData(MetaDataParser.java:43)
        at net.sourceforge.subsonic.domain.MusicFile.getMetaData(MusicFile.java:242)
        at net.sourceforge.subsonic.service.SearchService$Line.forFile(SearchService.java:717)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:813)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:139)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)
Caused by: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.RangeCheck(ArrayList.java:547)
        at java.util.ArrayList.get(ArrayList.java:322)
        at org.jaudiotagger.tag.mp4.field.Mp4TrackField.build(Mp4TrackField.java:143)
        at org.jaudiotagger.tag.mp4.Mp4TagField.<init>(Mp4TagField.java:86)
        at org.jaudiotagger.tag.mp4.field.Mp4TagTextField.<init>(Mp4TagTextField.java:65)
        at org.jaudiotagger.tag.mp4.field.Mp4TagTextNumberField.<init>(Mp4TagTextNumberField.java:57)
        at org.jaudiotagger.tag.mp4.field.Mp4TrackField.<init>(Mp4TrackField.java:127)
        at org.jaudiotagger.audio.mp4.Mp4TagReader.createMp4Field(Mp4TagReader.java:216)
        at org.jaudiotagger.audio.mp4.Mp4TagReader.read(Mp4TagReader.java:161)
        at org.jaudiotagger.audio.mp4.Mp4FileReader.getTag(Mp4FileReader.java:46)
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:106)
        ... 14 more


Another one:

Code: Select all


[2011-06-06 03:12:14,634] WARN JaudiotaggerParser - Error when parsing tags in [path to file.ogg]
org.jaudiotagger.audio.exceptions.CannotReadException: [path to file.ogg]:/ by zero
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:117)
        at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:285)
        at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:148)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.getRawMetaData(JaudiotaggerParser.java:71)
        at net.sourceforge.subsonic.service.metadata.MetaDataParser.getMetaData(MetaDataParser.java:43)
        at net.sourceforge.subsonic.domain.MusicFile.getMetaData(MusicFile.java:242)
        at net.sourceforge.subsonic.service.SearchService$Line.forFile(SearchService.java:717)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:813)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:139)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)
Caused by: java.lang.ArithmeticException: / by zero
        at org.jaudiotagger.audio.ogg.util.OggInfoReader.computeBitrate(OggInfoReader.java:132)
        at org.jaudiotagger.audio.ogg.util.OggInfoReader.read(OggInfoReader.java:123)
        at org.jaudiotagger.audio.ogg.OggFileReader.getEncodingInfo(OggFileReader.java:62)
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:104)
        ... 14 more


[Ivanslav1]

I am having the same problem as well.

Album art shows up but track do not. When I look under my Media Monkey, the tracks are there. Don't know why it's not being indexed.

Ran manual indexing twice. No luck.

No warnings in log file either

[ajkessel]

It appears my index is not updating at all anymore. I've upgraded to the latest release. Here's the error I'm seeing; any suggestions for a workaround?

Code: Select all

[2011-08-26 14:58:40,642] ERROR SearchService - Failed to create search index.
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at java.lang.String.getChars(String.java:854)
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:391)
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        at net.sourceforge.subsonic.service.SearchService$Line.toString(SearchService.java:794)
        at java.lang.String.valueOf(String.java:2826)
        at java.io.PrintWriter.println(PrintWriter.java:710)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:816)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:141)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:62)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:115)


[ajkessel]

Now the index does seem to update, but it's really hit or miss what appears in search results. Can anyone help with a solution?

[BKKKPewsey]

What is your memory setting? This is common problem if SS (Java) is running low on memory.
Remember the larger your music collection the larger your memory limit needs to be, so after time if you add to music collection (which most people do) you may need to increase the SS memory allocation.

[ajkessel]

Thanks for the tip. I just bumped it up from the default (100M) to 300M. I have about ~60K tracks. It seems to be returning results more reliably now, but there is still something odd about it. For example, if I search for "Beatles," I see lots of results under albums (and songs), but no "artist" results. Other searches give only one album when I knew there are many, but the songs list tracks from many albums. Is this how the search results are supposed to behave?

[BKKKPewsey]

Yep the search results can be a bit strange because it appears to use a mix of both ID tags and folder (directory) names.
If you have your music stored as artist/album/tracks as recommended then its is usually ok but can get a bit weird for compilation albums (various artists).
Dont forget it only shows the first five results and you need to click on "more" to see the rest

[ajkessel]

But if I have a folder /The Beatles/Abbey Road/(filename for first track), shouldn't "The Beatles" show up as an artist in the search result? I am getting "Abbey Road" in the album results, just no results at all for artist.

Although maybe this is something odd about this artist in particular -- there is an album result "The Beatles," and if I select that, I actually get all Beatles albums.

But as another example, I have /Phish/(Album Name)... If I search on "Phish," I do see "Phish" as an artist result, and only one album as an album result. If I then click on "Phish" as the artist result, I see 27 albums. But then if I search arbitrarily on some of those albums, I get no results for those album names.

[BKKKPewsey]

This is what I get



and if I click more on the album list



My folders are organised as music/1st artist character/Artist/album/tracks
eg music/B/Beatles/Album/tracks
Notice that my folder is called "Beatles" but the ID tags are "The Beatles"
hence why they are shown without the "THE" in the artist field but that shouldn't make any difference

[ajkessel]

Although my folders are organized /Artist/Album/Track, I don't see "Beatles" in the result. I.e., the folders are /The Beatles/With The Beatles/(filename), but see below (no artist result):

2011-09-18_093639.jpg

Any idea why this might be?

[BKKKPewsey]

No idea sorry

I am assuming that the left pane artist list is showing "The Beatles"
If so it is one of Subsonics little mysteries