Items not found when searching but ARE when browsing...

[scottherzig]

It is with great reluctance that I post this issue. I say that because I have seen so many posts similar to it, I am likely missing the answer somewhere, but I truly have searched through this forum. I have spent some time over the last few days grazing the forum and have found a lot of great information about the issues around searching and the index.

First things first...... YES I UPDATED THE SEARCH INDEX. I have it set to update at 2300 (and I do it manually whenever I rip a few more CDs throughout the day). I see the index file cleared, and rebuilt over a few minutes. I have checked the permissions on the folders in my library. I have renamed items in the albums I can't find and the folder too. I have sprinkled the blood of a dead chicken on the computer. I can NOT find some items with the search. My example today is "Wiz Khalifa" and the album is "Rolling Papers." If I use the artist browser on the left - no problem; I find my artist and my album. Happy Day.

So I search for the artist.... this artist does not come up. I search for the album, it doesn't come up. I search for a song in the album (and the profanity in some of the song titles makes it easy to narrow it down in a hurry), nothing comes up.

I use iTunes as the input method for my music. This is not a large music database, only about 50K titles. The index file is just shy of 15MB.

Is this possibly an issue with tags not being in the right format? I see NO errors in the log file (except when thumbnails are not available).

Any help is appreciated.

-SH

[BKKKPewsey]

I have sprinkled the blood of a dead chicken on the computer

Try using a live chicken next time whilst standing in a pentagram
Assuming you have the albums in question in the normal (for SS) format of Artist/Album/tracks then I would suggest checking your ID3 tags.
Make sure that Album, Artist & Title tags are correct for ALL the tracks on the album.

If they seem ok rebuild the database from scratch - Stop SS and then delete the lucene search folders (subsonic/lucene/delete what's here)
Restart SS and then update search index.

[scottherzig]

Thanks BKKK for your reply. My folder structure is generally in the format you recommend. The compilations folder adds an extra layer to that - but it seems to be ok.

I will look for the answer, but in case a reply comes forth first...in what format is SS reading tags? Mine seem to be in ID3v2.x according an external program. I am going to spend some time looking at the tags.

If I don't get anywhere, then I will clear the folder you recommend and go from there.

For now, I am off to the butcher...just in case.

-SH

[scottherzig]

I wanted to follow up and confirm that the solution seems to be to delete the lucene sub-folders. It seems to me that the nightly index should be able to handle this. I am *CONSTANTLY* updating my music library. There is a change, addition, removal of a duplicate entry....something going on *EVERY* day. I have been adding files on a daily basis and have noticed this phenomenon of some items missing from search results for quite some time now. If the fix is to shutdown the server, delete the sub-folders in the lucene directory, restart the service, then index again - then this is what should be happening automatically when you schedule the index to be scanned through the web-UI.

Also I think the index/rebuild option(s) should be on the control panel. Personally I would like more "controls" added (not moved) to the "control" panel to save me from having to go into the browser. I would also like the ability to browse by more than just the artist....album, genre, rating, etc.

-- Thanks for a great product and a great forum. This changes nothing in my usage or loyalty of the product...this call for help has merely transformed into a Request For Enhancement.

Cheers,
SH

[bushman4]

Now that you have reset the index, do new additions show up after an index rebuild?

I would suspect that your fix was to fix a larger problem with the index files, and is not needed daily. At least I have never noticed that new stuff is not searchable, and I also add/move/change daily.

Just curious,

Glenn

[scottherzig]

Glenn,

As the saying goes - its better than it was. The search is returning much more meaningful and reliable results, which speaks to your theory of corruption during the initial build. I obviously had some dirty files or folders (Noooooooooo! Not THAT kind of dirty) in there somewhere.

So those of you that happen along here with similar issues consider the fix to be:
1) Yes, it is your fault.
2) Stop SS thru the control panel.
3) Delete folders within the lucene directory in SS folder tree.
3a) Go find your music folders and find the files that somehow have Chinese characters and delete them...yes really.
3b) Consider boosting the memory allocation. I set mine at 256. Unless you have multiple users, this should be plenty.
4) Restart SS thru control panel.
5) Index.

[chadalan]

i have been running rather extensive test scenarios on this topic and can confirm SOMETHING is indeed wrong; just can't discern what is causing the issue.

FIRST CONFIG:
Media Folder path: M:\media
SS DB shows ~45k entries culled from across ~3600 subfolders
My "test" artist originally had their 9 albums stored in 9 different subfolders randomly occurring across the 3600 total folders
An Artist search would only find 8 of the 9 albums
An Album Title search for the missing album finds nothing
BUT - A search for any select song title from the missing album WOULD return a valid result
AND - browsing in subsonic web or android app to the missing albums folder path successfully found the album (and displayed and played it correctly)

SECOND CONFIG:
Media Folder path: M:\media
SS DB shows ~45k entries culled from across ~3600 subfolders
On second config, I moved ALL 9 of my "test" artist's album folders into a newly created parent folder (M:\media\NEWTESTFOLDER\then album 1, album 2, etc.)
An Artist search would STILL only find 8 of the 9 albums
An Album Title search for the missing album STILL finds nothing
BUT - A search for any select song title from the missing album WOULD return a valid result
AND - browsing in subsonic web or android app to the missing albums folder path successfully found the album and displayed and played it correctly

So, something must be wrong with the ONE particular album's tags, right? But, then I tried this:

THIRD CONFIG:
NEW Media Folder path: M:\media\NEWTESTFOLDER\ (the folder i created in config #2 above that only holds the 9 albums from my test artist.)
SS DB shows 110 entries culled from across 9 subfolders
An Artist search NOW FINDS ALL 9 albums
An Album Title search for the (previously) missing album NOW FINDS THE ALBUM
AND, needlless to say, my search for any select song title from the missing album STILL returned a valid result
AND - browsing in subsonic web or android app to the missing albums folder path STILL successfully found the album and displayed and played it correctly

Any clue what could be causing this? I've run these same tests in the latest versions of winamp, win media player, and mediamonkey -- all three of those apps find all 9 albums no matter where I have them stored. only SS 4.6 (build 2583) loses one of the albums in two of my three config tests.

(FYI - SS cites no errors in the indexing log. Have also tried deleting and rebuilding the search index between all three tests. Both ID3v1 and v2 tags are properly applied to all mp3s.No special characters or font encodings exist on the album that likes to go missing in SS.)

thanks for any help on this issue. (and thanks Sindre for the absolutely wonderful program that is Subsonic! not complaining about this issue, just trying to figure out exactly what is going wrong so as to get the issue resolved.)

[BKKKPewsey]

Because of the strange way that SS acquires the artist and album names, it is mostly probably a problem with the directory structure.
This is especially true for any multi cd albums - SS doesn't like artist/album/cd#/tracks
You end up with an album called cd# and an artist with the album name.
Until SS starts using the tag info for all the search info and not just the title tag then there not much can be done
Agreed it is rather frustrating for those of us who have proper tagged files.

[chadalan]

i'm not seeing the issue coming from misnamed tags OR file/folders -- like my fellow commenters in this thread (and many others in the SS forums) -- i am NOT able to SEARCH on certain albums that i KNOW are there; but i CAN browse to them.

i am sure the data is in the lucene index file AND i can successfully browse to "missing" albums in SS. for example- in my lucene database i have these two abums:


A ixYxi 1231649856118 ixYxi 1231649856118 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono ixYxi 0 ixYxi SPOON ixYxi TELEPHONO ixYxi ixYxi ixYxi
F ixYxi 1332853153265 ixYxi 1332853153265 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\01 Don't Buy The Realistic.mp3 ixYxi 6618694 ixYxi SPOON ixYxi TELEPHONO ixYxi DON'T BUY THE REALISTIC ixYxi 1996 ixYxi
F ixYxi 1332853153266 ixYxi 1332853153266 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\02 Not Turning Off.mp3 ixYxi 5312418 ixYxi SPOON ixYxi TELEPHONO ixYxi NOT TURNING OFF ixYxi 1996 ixYxi
F ixYxi 1332853153267 ixYxi 1332853153267 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\03 All The Negatives Have Been Destr.mp3 ixYxi 4525590 ixYxi SPOON ixYxi TELEPHONO ixYxi ALL THE NEGATIVES HAVE BEEN DESTROYED ixYxi 1996 ixYxi
F ixYxi 1332853153268 ixYxi 1332853153268 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\04 Cvantez.mp3 ixYxi 4706284 ixYxi SPOON ixYxi TELEPHONO ixYxi CVANTEZ ixYxi 1996 ixYxi
F ixYxi 1332853153269 ixYxi 1332853153269 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\05 Nefarious.mp3 ixYxi 4929188 ixYxi SPOON ixYxi TELEPHONO ixYxi NEFARIOUS ixYxi 1996 ixYxi
F ixYxi 1332853153270 ixYxi 1332853153270 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\06 Claws Tracking.mp3 ixYxi 4783532 ixYxi SPOON ixYxi TELEPHONO ixYxi CLAWS TRACKING ixYxi 1996 ixYxi
F ixYxi 1332853153271 ixYxi 1332853153271 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\07 Dismember.mp3 ixYxi 3042990 ixYxi SPOON ixYxi TELEPHONO ixYxi DISMEMBER ixYxi 1996 ixYxi
F ixYxi 1332853153272 ixYxi 1332853153272 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\08 Idiot Driver.mp3 ixYxi 2766400 ixYxi SPOON ixYxi TELEPHONO ixYxi IDIOT DRIVER ixYxi 1996 ixYxi
F ixYxi 1332853153273 ixYxi 1332853153273 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\09 Towner.mp3 ixYxi 5380228 ixYxi SPOON ixYxi TELEPHONO ixYxi TOWNER ixYxi 1996 ixYxi
F ixYxi 1332853153274 ixYxi 1332853153274 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\10 Wanted To Be Your.mp3 ixYxi 3083945 ixYxi SPOON ixYxi TELEPHONO ixYxi WANTED TO BE YOUR ixYxi 1996 ixYxi
F ixYxi 1332853153275 ixYxi 1332853153275 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\11 Theme To Wendel Stivers.mp3 ixYxi 3301961 ixYxi SPOON ixYxi TELEPHONO ixYxi THEME TO WENDEL STIVERS ixYxi 1996 ixYxi
F ixYxi 1332853153276 ixYxi 1332853153276 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\12 Primary.mp3 ixYxi 2427274 ixYxi SPOON ixYxi TELEPHONO ixYxi PRIMARY ixYxi 1996 ixYxi
F ixYxi 1332853153277 ixYxi 1332853153277 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\13 The Government Darling.mp3 ixYxi 4230902 ixYxi SPOON ixYxi TELEPHONO ixYxi THE GOVERNMENT DARLING ixYxi 1996 ixYxi
F ixYxi 1332853153278 ixYxi 1332853153278 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Telephono\14 Plastic Mylar.mp3 ixYxi 5832532 ixYxi SPOON ixYxi TELEPHONO ixYxi PLASTIC MYLAR ixYxi 1996 ixYxi
A ixYxi 1323835050459 ixYxi 1323835050459 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference ixYxi 0 ixYxi SPOON ixYxi TRANSFERENCE ixYxi ixYxi ixYxi
F ixYxi 1332876831876 ixYxi 1332876831876 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\01 - Before Destruction.mp3 ixYxi 5610727 ixYxi SPOON ixYxi TRANSFERENCE ixYxi BEFORE DESTRUCTION ixYxi 2010 ixYxi
F ixYxi 1332876831877 ixYxi 1332876831877 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\02 - Is Love Forever.mp3 ixYxi 4255907 ixYxi SPOON ixYxi TRANSFERENCE ixYxi IS LOVE FOREVER? ixYxi 2010 ixYxi
F ixYxi 1332876831879 ixYxi 1332876831879 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\03 - The Mystery Zone.mp3 ixYxi 9918612 ixYxi SPOON ixYxi TRANSFERENCE ixYxi THE MYSTERY ZONE ixYxi 2010 ixYxi
F ixYxi 1332876831881 ixYxi 1332876831881 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\04 - Who Makes Your Money.mp3 ixYxi 6241578 ixYxi SPOON ixYxi TRANSFERENCE ixYxi WHO MAKES YOUR MONEY ixYxi 2010 ixYxi
F ixYxi 1332876831882 ixYxi 1332876831882 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\05 - Written In Reverse.mp3 ixYxi 7669717 ixYxi SPOON ixYxi TRANSFERENCE ixYxi WRITTEN IN REVERSE ixYxi 2010 ixYxi
F ixYxi 1332876831884 ixYxi 1332876831884 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\06 - I Saw The Light.mp3 ixYxi 10315148 ixYxi SPOON ixYxi TRANSFERENCE ixYxi I SAW THE LIGHT ixYxi 2010 ixYxi
F ixYxi 1332876831886 ixYxi 1332876831886 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\07 - Trouble Comes Running.mp3 ixYxi 6706792 ixYxi SPOON ixYxi TRANSFERENCE ixYxi TROUBLE COMES RUNNING ixYxi 2010 ixYxi
F ixYxi 1332876831888 ixYxi 1332876831888 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\08 - Goodnight Laura.mp3 ixYxi 3882024 ixYxi SPOON ixYxi TRANSFERENCE ixYxi GOODNIGHT LAURA ixYxi 2010 ixYxi
F ixYxi 1332876831889 ixYxi 1332876831889 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\09 - Out Go The Lights.mp3 ixYxi 7477485 ixYxi SPOON ixYxi TRANSFERENCE ixYxi OUT GO THE LIGHTS ixYxi 2010 ixYxi
F ixYxi 1332876831891 ixYxi 1332876831891 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\10 - Got Nuffin.mp3 ixYxi 7001492 ixYxi SPOON ixYxi TRANSFERENCE ixYxi GOT NUFFIN ixYxi 2010 ixYxi
F ixYxi 1332876831893 ixYxi 1332876831893 ixYxi M:\MEDIA\Spoon - Current Discog\Spoon -Transference\11 - Nobody Gets Me But You.mp3 ixYxi 8749192 ixYxi SPOON ixYxi TRANSFERENCE ixYxi NOBODY GETS ME BUT YOU ixYxi 2010 ixYxi

why is it i can search on the artist SPOON and find the first album (Telephono ); but not the second (Transference)?
why is it i can search on the album title Telephono and find it; but search on Transference and NOT find it?
BUT - why is it i can search on a song title (like Trouble Comes Running) FROM the 'missing' Transference album and yet I do find it?!?

again; to my untrained eyes, the tags look right, the file/folder structure looks right, the index file looks right, browsing to the 'missing' album WORKS right, only in this particular example, one particular album title search and artist name search are not returning valid results.

i can replicate this same result with MANY different entries in my collection.

can any devs out there see a clear-cut issue with the examples above? thanks again!

[chadalan]

just f.y.i. as a follow-up to the post above:

if i perform wildcard searches i get the same results:

spoo* -- i find the telephono but NOT the transference album (when returning artists pattern matching the name spoo(*)n)
trans* -- i DO NOT find the transference album (even though i should)
trou* -- i DO find the song trouble comes running; which appears on the transference album (even though i can't find that album with either of the searches above)