Page 1 of 3

Tracks missing from search index

PostPosted: Mon May 30, 2011 2:52 pm
by ajkessel
When searching Subsonic (either from the web interface or iPhone iSub client), lots of tracks don't show up in search results. I can navigate to them through the folders in the left pane; they play fine; and the metadata is fine. But they aren't being indexed.

I've found no pattern to which tracks are indexed and which are not, but the ones that are missing seem to remain missing.

Any suggestions for how to troubleshoot?

Running 4.4 (build 2169) / jetty-6.1.x, java 1.6.0_22, Linux (53.9 MB / 96.7 MB). (Debian Stable).

PostPosted: Mon May 30, 2011 9:38 pm
by GJ51
Have you run a manual indexing?

Settings/Search - Update search index now.

Manual re-index

PostPosted: Mon May 30, 2011 9:41 pm
by ajkessel
Yes, I've done a manual reindex several times.

PostPosted: Mon Jun 06, 2011 5:11 am
by Kirk
I thought this was happening a few times on my Subsonic server too... but then I realized the metadata actually was screwed up for the tracks. lol.

Are you 100% sure?

Have you checked your Subsonic logs while the search index is being created, to look for any possibly relevant errors during the process?

Cheers,
Kirk

Works now

PostPosted: Mon Jun 06, 2011 2:01 pm
by ajkessel
I noticed after 20+ times reindexing, some of the missing tracks appear or re-appear. So it does seem to be some problem with the indexing. I think the metadata should be fine, though -- the tracks look and play okay in both iTunes and Mediamonkey.

This might be the cause, from subsonic_sh.log. The machine has 1GB RAM, which I would think should be enough, although maybe this is a JVM specific issue:

Code: Select all
50905303 [btpool0-64] INFO org.apache.http.impl.client.DefaultHttpClient - Retrying request
Exception in thread "Search Index Generator" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3209)
        at java.lang.String.<init>(String.java:215)
        at java.lang.StringBuffer.toString(StringBuffer.java:585)
        at java.io.BufferedReader.readLine(BufferedReader.java:334)
        at java.io.BufferedReader.readLine(BufferedReader.java:362)
        at net.sourceforge.subsonic.service.SearchService.getIndex(SearchService.java:545)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:151)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)


There are also some suspicious errors in subsonic.log, e.g.:

Code: Select all
[2011-06-06 03:14:18,843] WARN JaudiotaggerParser - Error when parsing tags in [path to file.m4a]
org.jaudiotagger.audio.exceptions.CannotReadException: [path to file.m4a]:Index: 2, Size: 2
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:117)
        at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:285)
        at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:148)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.getRawMetaData(JaudiotaggerParser.java:71)
        at net.sourceforge.subsonic.service.metadata.MetaDataParser.getMetaData(MetaDataParser.java:43)
        at net.sourceforge.subsonic.domain.MusicFile.getMetaData(MusicFile.java:242)
        at net.sourceforge.subsonic.service.SearchService$Line.forFile(SearchService.java:717)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:813)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:139)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)
Caused by: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.RangeCheck(ArrayList.java:547)
        at java.util.ArrayList.get(ArrayList.java:322)
        at org.jaudiotagger.tag.mp4.field.Mp4TrackField.build(Mp4TrackField.java:143)
        at org.jaudiotagger.tag.mp4.Mp4TagField.<init>(Mp4TagField.java:86)
        at org.jaudiotagger.tag.mp4.field.Mp4TagTextField.<init>(Mp4TagTextField.java:65)
        at org.jaudiotagger.tag.mp4.field.Mp4TagTextNumberField.<init>(Mp4TagTextNumberField.java:57)
        at org.jaudiotagger.tag.mp4.field.Mp4TrackField.<init>(Mp4TrackField.java:127)
        at org.jaudiotagger.audio.mp4.Mp4TagReader.createMp4Field(Mp4TagReader.java:216)
        at org.jaudiotagger.audio.mp4.Mp4TagReader.read(Mp4TagReader.java:161)
        at org.jaudiotagger.audio.mp4.Mp4FileReader.getTag(Mp4FileReader.java:46)
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:106)
        ... 14 more


Another one:

Code: Select all

[2011-06-06 03:12:14,634] WARN JaudiotaggerParser - Error when parsing tags in [path to file.ogg]
org.jaudiotagger.audio.exceptions.CannotReadException: [path to file.ogg]:/ by zero
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:117)
        at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:285)
        at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:148)
        at net.sourceforge.subsonic.service.metadata.JaudiotaggerParser.getRawMetaData(JaudiotaggerParser.java:71)
        at net.sourceforge.subsonic.service.metadata.MetaDataParser.getMetaData(MetaDataParser.java:43)
        at net.sourceforge.subsonic.domain.MusicFile.getMetaData(MusicFile.java:242)
        at net.sourceforge.subsonic.service.SearchService$Line.forFile(SearchService.java:717)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:813)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:139)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:60)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:113)
Caused by: java.lang.ArithmeticException: / by zero
        at org.jaudiotagger.audio.ogg.util.OggInfoReader.computeBitrate(OggInfoReader.java:132)
        at org.jaudiotagger.audio.ogg.util.OggInfoReader.read(OggInfoReader.java:123)
        at org.jaudiotagger.audio.ogg.OggFileReader.getEncodingInfo(OggFileReader.java:62)
        at org.jaudiotagger.audio.generic.AudioFileReader.read(AudioFileReader.java:104)
        ... 14 more

missing tracks

PostPosted: Wed Jun 08, 2011 1:52 pm
by Ivanslav1
I am having the same problem as well.

Album art shows up but track do not. When I look under my Media Monkey, the tracks are there. Don't know why it's not being indexed.

Ran manual indexing twice. No luck.

No warnings in log file either

Re: Tracks missing from search index

PostPosted: Fri Aug 26, 2011 7:12 pm
by ajkessel
It appears my index is not updating at all anymore. I've upgraded to the latest release. Here's the error I'm seeing; any suggestions for a workaround?

Code: Select all
[2011-08-26 14:58:40,642] ERROR SearchService - Failed to create search index.
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at java.lang.String.getChars(String.java:854)
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:391)
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        at net.sourceforge.subsonic.service.SearchService$Line.toString(SearchService.java:794)
        at java.lang.String.valueOf(String.java:2826)
        at java.io.PrintWriter.println(PrintWriter.java:710)
        at net.sourceforge.subsonic.service.SearchService$Scanner.visit(SearchService.java:816)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:348)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.domain.MusicFile.accept(MusicFile.java:355)
        at net.sourceforge.subsonic.service.SearchService.doCreateIndex(SearchService.java:141)
        at net.sourceforge.subsonic.service.SearchService.access$000(SearchService.java:62)
        at net.sourceforge.subsonic.service.SearchService$1.run(SearchService.java:115)

Re: Tracks missing from search index

PostPosted: Sat Sep 17, 2011 10:08 pm
by ajkessel
Now the index does seem to update, but it's really hit or miss what appears in search results. Can anyone help with a solution?

Re: Tracks missing from search index

PostPosted: Sat Sep 17, 2011 10:45 pm
by BKKKPewsey
What is your memory setting? This is common problem if SS (Java) is running low on memory.
Remember the larger your music collection the larger your memory limit needs to be, so after time if you add to music collection (which most people do) you may need to increase the SS memory allocation.

:mrgreen:

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 12:35 am
by ajkessel
Thanks for the tip. I just bumped it up from the default (100M) to 300M. I have about ~60K tracks. It seems to be returning results more reliably now, but there is still something odd about it. For example, if I search for "Beatles," I see lots of results under albums (and songs), but no "artist" results. Other searches give only one album when I knew there are many, but the songs list tracks from many albums. Is this how the search results are supposed to behave?

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 12:54 am
by BKKKPewsey
Yep the search results can be a bit strange because it appears to use a mix of both ID tags and folder (directory) names.
If you have your music stored as artist/album/tracks as recommended then its is usually ok but can get a bit weird for compilation albums (various artists).
Dont forget it only shows the first five results and you need to click on "more" to see the rest 8)

:mrgreen:

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 1:31 am
by ajkessel
But if I have a folder /The Beatles/Abbey Road/(filename for first track), shouldn't "The Beatles" show up as an artist in the search result? I am getting "Abbey Road" in the album results, just no results at all for artist.

Although maybe this is something odd about this artist in particular -- there is an album result "The Beatles," and if I select that, I actually get all Beatles albums.

But as another example, I have /Phish/(Album Name)... If I search on "Phish," I do see "Phish" as an artist result, and only one album as an album result. If I then click on "Phish" as the artist result, I see 27 albums. But then if I search arbitrarily on some of those albums, I get no results for those album names.

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 2:01 am
by BKKKPewsey
This is what I get

Image

and if I click more on the album list

Image

My folders are organised as music/1st artist character/Artist/album/tracks
eg music/B/Beatles/Album/tracks
Notice that my folder is called "Beatles" but the ID tags are "The Beatles"
hence why they are shown without the "THE" in the artist field but that shouldn't make any difference :?

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 1:39 pm
by ajkessel
Although my folders are organized /Artist/Album/Track, I don't see "Beatles" in the result. I.e., the folders are /The Beatles/With The Beatles/(filename), but see below (no artist result):

2011-09-18_093639.jpg


Any idea why this might be?

Re: Tracks missing from search index

PostPosted: Sun Sep 18, 2011 2:15 pm
by BKKKPewsey
:? No idea sorry

I am assuming that the left pane artist list is showing "The Beatles"
If so it is one of Subsonics little mysteries :roll: