Problem with Search Index-creation

Need help? Post your questions here.

Moderator: moderators

Problem with Search Index-creation

Postby wannabe_nerd » Sun Jul 24, 2011 8:29 am

Hi Guys,

as the topic says, i have a problem with my search index. I know there are several post about problems with the search index already in this forum, but i think my problem is a little bit different. But first my situation:

Everytime i try to create a new search index the logs says the creation stopps at 13000 entries. However if i go to the main-page or try to search something i get the message, that the search index creation is running...

here is the log:

Code: Select all
[2011-07-24 10:10:26,621] INFO SearchService - Starting to create search index.
[2011-07-24 10:10:27,199] INFO SearchService - Created search index with 1000 entries.
[2011-07-24 10:10:27,604] INFO SearchService - Created search index with 2000 entries.
[2011-07-24 10:10:27,994] INFO SearchService - Created search index with 3000 entries.
[2011-07-24 10:10:28,353] INFO SearchService - Created search index with 4000 entries.
[2011-07-24 10:10:28,743] INFO SearchService - Created search index with 5000 entries.
[2011-07-24 10:10:29,117] INFO SearchService - Created search index with 6000 entries.
[2011-07-24 10:10:29,523] INFO SearchService - Created search index with 7000 entries.
[2011-07-24 10:10:29,897] INFO SearchService - Created search index with 8000 entries.
[2011-07-24 10:10:30,303] INFO SearchService - Created search index with 9000 entries.
[2011-07-24 10:10:30,677] INFO SearchService - Created search index with 10000 entries.
[2011-07-24 10:10:31,036] INFO SearchService - Created search index with 11000 entries.
[2011-07-24 10:10:31,426] INFO SearchService - Created search index with 12000 entries.
[2011-07-24 10:10:31,769] INFO SearchService - Created search index with 13000 entries.


at this point the search index creation stopps.
One thing is interessating: when i looked at the index itself i found that the index stopps right in the middle of one certain index-entry:

Code: Select all
F ixYxi 1292207916697 ixYxi 1292207916697 ixYxi F:\Videos von F&F\Filme\Millenium-Trillogie\Vergebung.Teil.3.2009.German.AC3.720p.BluRay.x264-QRC\vergebung.avi ixYxi 6146582856 ixYxi MILLENIUM-TRILLOGIE ixYxi VERGEBUNG.TEIL.3.2009.GERMAN.AC3.720P.BLURAY.X264-QRC ixYxi VERGEBUNG ixYxi  ixYxi
A ixYxi 1305468561439 ixYxi 1305468561439 ixYxi F:\Videos von F&F\Filme\Minority Report ixYxi 0 ixYxi FILME ixYxi MINORITY REPORT ixYxi  ixYxi  ixYxi
F ixYxi 1201892096000 ixYxi 1201892096000 ixYxi F:\Videos von F&F\Filme\Minority Rep


This also is the entry which often causes errors in the subsonic log:

Code: Select all
[2011-07-24 10:03:19,616] ERROR SearchService - An error occurred while reading index entry 'F ixYxi 1201892096000 ixYxi 1201892096000 ixYxi F:\Videos von F&F\Filme\Minority Rep'.
java.lang.ArrayIndexOutOfBoundsException: 5
   at net.sourceforge.subsonic.service.SearchService$Line.parse(SearchService.java:684)
   at net.sourceforge.subsonic.service.SearchService.getIndex(SearchService.java:549)
   at net.sourceforge.subsonic.service.SearchService.getRandomAlbums(SearchService.java:425)
   at net.sourceforge.subsonic.controller.HomeController.getRandom(HomeController.java:167)
   at net.sourceforge.subsonic.controller.HomeController.handleRequestInternal(HomeController.java:95)
   at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
   at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
   at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
   at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
   at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
   at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
   at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
   at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
   at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
   at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
   at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.ResponseHeaderFilter.doFilter(ResponseHeaderFilter.java:47)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:43)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.ParameterDecodingFilter.doFilter(ParameterDecodingFilter.java:54)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at net.sourceforge.subsonic.filter.BootstrapVerificationFilter.doFilter(BootstrapVerificationFilter.java:54)
   at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
   at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
   at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
   at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
   at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
   at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
   at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
   at org.mortbay.jetty.Server.handle(Server.java:313)
   at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
   at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
   at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514)
   at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
   at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
   at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
   at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)


I waited all night yesterday but it would't go on. The only solution to use the search and the main-page services (random music etc.) is to restart subsonic. This problem also occures if the automatic index creation starts at 3.00am every night. To use Subsonic at the next day a had to write a small cmd-file which resarts subsonic every night at 3.05 am.

I already upped the memorylimit for subsonic from 50mb to 1000mb (i have more than enough ram for this xD ), but this didn't work.

Of course because of this problem the search-index does not work correctly - some songs are missing although there a playable and findable in the left index-frame. But if i search them i get no results.

My System is a normal Windows 7 64bit, 8gb ram, subsonic instatalled on a 60gb ssd which is my system-drive too.

Do you have any ideas? ^^

greets wannabe_nerd
[/quote]
Cloud Computing .... i made my own cloud !
wannabe_nerd
 
Posts: 36
Joined: Wed May 11, 2011 9:06 am

Postby BKKKPewsey » Sun Jul 24, 2011 10:20 am

I expect you may have tried this but have you tried moving that particular film out of an indexed folder and then trying to update the search index?
That film may have a bad header which is causing the indexer problems :roll:
Everyone is entitled to be stupid, Image but some abuse the privilege!

Due to the confusion from too many genres of music, we have decided to put both country music and rap music into the genre of Crap music.
User avatar
BKKKPewsey
 
Posts: 2080
Joined: Mon May 23, 2011 12:16 pm
Location: United Kingdom

Postby wannabe_nerd » Sun Jul 24, 2011 10:37 am

That was a good Tip and i tried this out a few minuetes ago ... There is no effect but I noticed that the index stopped at a other point. The interesting thing is that the index-file now and before I removed the movie(minorityreport) is exactly 3,151mb big. Could that be an useful information?
Cloud Computing .... i made my own cloud !
wannabe_nerd
 
Posts: 36
Joined: Wed May 11, 2011 9:06 am

Postby BKKKPewsey » Sun Jul 24, 2011 11:41 am

wannabe_nerd wrote:That was a good Tip and i tried this out a few minuetes ago ... There is no effect but I noticed that the index stopped at a other point. The interesting thing is that the index-file now and before I removed the movie(minorityreport) is exactly 3,151mb big. Could that be an useful information?


3,151mb? are you sure 3Gb seems to be a bit big. I am indexing 330Gb of music and vids and my index is only 13,000 KB in size :?
Everyone is entitled to be stupid, Image but some abuse the privilege!

Due to the confusion from too many genres of music, we have decided to put both country music and rap music into the genre of Crap music.
User avatar
BKKKPewsey
 
Posts: 2080
Joined: Mon May 23, 2011 12:16 pm
Location: United Kingdom

Postby wannabe_nerd » Sun Jul 24, 2011 12:20 pm

no, not 3gb
sry i meant 3 mb + 152kb ^^

my music / film-libary is about 1,3 Terabyte. 1,2 TB Movies and videos and about 100gb Music and stuff
Cloud Computing .... i made my own cloud !
wannabe_nerd
 
Posts: 36
Joined: Wed May 11, 2011 9:06 am

Problem solved!

Postby wannabe_nerd » Mon Jul 25, 2011 6:31 am

I found the solution for my problem:
i simply temporaly disabled the video-dicretory-path in subsonic and created manually the new search index without my video-files. And it worked perfectly oO

It seems as if Subsonic has problems with my videofile-metadata.

Now if i want to update my search index i just have to disable my video-paths.

Thanks a lot for your help BKKKPewsey!

greetz wannabe_nerd
Cloud Computing .... i made my own cloud !
wannabe_nerd
 
Posts: 36
Joined: Wed May 11, 2011 9:06 am


Return to Help

Who is online

Users browsing this forum: No registered users and 33 guests