Error browsing into non-latin subfolders

[TheMaestro42]

I have a problem when I try to browse to an album without plain english (in this case Japanese) characters in its title. I get an internal error:

Big Block Begin --------------------------------

Error

Subsonic encountered an internal error. You can report this error in the Subsonic Forum. Please include the information below.
Exception java.lang.NullPointerException
Message null
Java version Sun Microsystems Inc. 1.5.0_08
Operating system Windows XP 5.1
Tomcat home C:\Program Files\Apache Software Foundation\Tomcat 5.5
Stack trace java.lang.NullPointerException at net.sourceforge.subsonic.domain.MusicFile.getChildren(MusicFile.java:248) at net.sourceforge.subsonic.controller.MainController.handleRequestInternal(MainController.java:31) at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:45) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:796) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:727) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:396) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:350) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:108) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:25) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:425) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:452) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1285) at java.lang.Thread.run(Unknown Source)

Big Block End -----------------------------

It only happens when I try to browse into an album with (Japanese) characters. I can play the songs normally if I click the "Play All" or "Add All" text in the artist directory, even if the songs have Japanese Characters in them, although they only appear as periods in the bottom playlist window.

Thanks for any help in advance.

[sindre_mehus]

Hi,

I'm trying to reproduce the error, but I can't even create a directory with Japanese characters
I tried "日本", but no luck on my English Win XP. What XP version do you have? Can you give an example of a directory name which gives this error?

Thanks,
Sindre

[TheMaestro42]

Sorry for the late response. Haven't been focused on the issue too much since I can work around it.

But, for an exact example, I have an artist folder, "Kumi Koda" and in that artist folder, I have an album folder "BEST～second session～ (flac)" The tilde are the Japanese character, not the ASCII '~'.

I can of course browse to the artist folder, but when I click on the Album folder, I get the above error message.

I have Windows XP Media Center Edition (English), and I have the asian language pack added through the regional options in the control panel.

If I change the marks to an ASCII ~, I no longer receive that error, and even though there are some japanese characters in the actual song titles, I can play them just fine.

I have several directories, all that produce the same problem, but that's the easiest example I can provide.

Thanks

[Jof]

I have also a problem with german letters like öäü, songs with this letters in their name are not shown in subsonic.
So i added in the server config of tomcat following entry: URIEnconding="UTF-8"
Now all songs with öäu are shown, but when i click at them i get following error:

Exception java.lang.NullPointerException
Message null
Java version Sun Microsystems Inc. 1.5.0_08
Operating system Linux 2.6.17-10-386
Tomcat home /usr/share/tomcat5.5
Stack trace java.lang.NullPointerException at net.sourceforge.subsonic.domain.MusicFile.getChildren(MusicFile.java:223) at net.sourceforge.subsonic.controller.MainController.handleRequestInternal(MainController.java:32) at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:45) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:806) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:736) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:396) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:350) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:108) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:25) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595)

[TheMaestro42]

I've read elsewhere on the forum that you can set the environment variable LC_ALL=en_US for Linux, but I couldn't find any sort of equivalent in Windows XP, or even if it's the same issue.

What I'm actually having a problem with is when the directories have non English characters in them, not so much with the songs themselves. (The songs will play and transcode)

And of course, I have Windows XP, not Linux.

Also, I can go into an artist folder "Björk" or "Blue Öyster Cult" but I can't browse into an artist folder "Test 日本" it fails. I wonder if it has anything to do with the number of bytes in the UTF-8 encoding. (i.e. 2 bytes works, but 3 bytes does not)

But I know nothing. Help!

edit: I created the test artist using the IME build in to Windows XP, if that helps. All my filesystems are NTFS as well.

[Jof]

I have the solution!

You have to export the Language Variables in the tomcat start script.
For Example in my case de_dE.UTF-8

[TheMaestro42]

Now all I need to do is figure out how to do that in Windows XP.

Time to dig!

Update: Still no luck, I don't think that's my problem... Oh well.

Yet another update: I checked the Tomcat log and came up with:

java.io.IOException: Failed to create thumbnail for E:\Audio\Library\HIGH and MIGHTY COLOR\???? (mp3)\cover.jpg. E:\Audio\Library\HIGH and MIGHTY COLOR\???? (mp3)\cover.jpg (The filename, directory name, or volume label syntax is incorrect)

As in it's ignoring the japanese characters and replacing them with ????

(This was the cover that doesn't load when I browse into the folder)

[TheMaestro42]

A list of things I've tried:

Tomcat 6.0 (Didn't fix the problem, but works )

Adding Character encoding changes to the JVM in the Tomcat Service Manager:

-Dfile.encoding=UTF-8
-Duser.language=jp
-Duser.country=JP

All sorts of combinations thereof.

I've tried setting environment variables in Windows (In the System Properties)

LC_ALL=jp_JP
LANG=jp_JP

And various combinations there above.

I've also tried setting the default language, region settings, and country to Japanese in Windows, but that also did not solve the issue.

I'm almost confident this is an encoding issue somewhere in the Java, but I just don't know enough about it to isolate the issue.


What I'm currently using:

Windows XP Media Center Edition (With all updates)
Tomcat 6.0 beta 2
Apache 2.2.3 (With mod_jk 1.2.19 connector, it still fails when I access Tomcat directly)
Subsonic 2.9
Java JRE 1.5 rel 9

I have the Asian Language pack installed in Windows XP (Under Regional options) and I currently have Windows Firewall on, but have port 80 unblocked (for outside use). Port 8080 is blocked on the outside, but still works locally. And I have a direct connection to the Internet.

I wish I had a spare machine so I could just put Linux on it and fix this silly problem, but I don't, and don't like dual booting.

All files are stored on an NTFS formatted 320GB drive.

Any additional information I can give if you ask.

[d2master]

I also have the same problem when trying to access my sub-folders
which are in Hebrew characters.

I use WinXP + Tomcat 5.5 + Java 1.5

and i get the following error message:

----------------------------------------------------------------------------
java.lang.NullPointerException at net.sourceforge.subsonic.domain.MusicFile.getChildren(MusicFile.java:223) at net.sourceforge.subsonic.controller.MainController.handleRequestInternal(MainController.java:32) at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:45) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:806) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:736) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:396) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:350) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107) at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:181) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:216) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:108) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:195) at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274) at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148) at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:90) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.sourceforge.subsonic.filter.RequestEncodingFilter.doFilter(RequestEncodingFilter.java:25) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Unknown Source)

----------------------------------------------------------------------------------

i would appriciate any help on the subject
________
grape ape pictures

[sindre_mehus]

Please download the latest beta version (3.0.beta1). This bug has been fixed there. Please let us know if that works for you.

Cheers,
Sindre

[d2master]

i downloaded the beta version.
It does fix the browsing into directories with non-latin letters.
thanks.

but there are other problems:
1. Many of the ID3 tags of the mp3 files are showing Gibrish instead of the text (still few are showing properly).
if i delete the tag then i can see the original file name O.K. (but i don't want to delete the ID3 tags of my files).

2. I still can't save playlists with files that are in Non-Latin directories

3. The option - "rating/Clear rating" of the music gives the same error as above.

4. (a small bug) some directories are showing on the wrong letter index:
The World Of Irish Folk - Show under "W"
The Verve - Shows under "V"
The Sunclub - Shows under "S"

they should've all apear uner "T" ofcourse.

But thanks for the new version it is great.
________
Warren Transmission

[sindre_mehus]

1. Many of the ID3 tags of the mp3 files are showing Gibrish instead of the text (still few are showing properly).
if i delete the tag then i can see the original file name O.K. (but i don't want to delete the ID3 tags of my files).

Is it a problem with Subsonic, or is it a problem with the tags? Does it show properly in other players (Winamp etc)?

2. I still can't save playlists with files that are in Non-Latin directories
3. The option - "rating/Clear rating" of the music gives the same error as above.

You're right. There's a lot of stuff that doesn't work with non-latin directories. Thanks for reporting it. I've found the bug and will fix it.

4. (a small bug) some directories are showing on the wrong letter index:
The World Of Irish Folk - Show under "W"
The Verve - Shows under "V"
The Sunclub - Shows under "S"

they should've all apear uner "T" ofcourse.

This is entirely intentional, but if you want to change it, please go to Settings > General > Articles to ignore

Sindre

[d2master]

The ID3 tags are showing propely in Winamp & Windows Media Player.
so i guess it is a problem with subsonic.

thanks for the advise about the "articles to ignore".



p.s
why everytime i try to access the subsonic stream over http, it creates a new "player" instead of streaming me the player that is already playing?

becuase each time i need to reload the playlist and make it start playing,
is there a way around it ? (i am logging with a different IP each time).

thanks for the great help and continue your great work on this product.

________
Ipad Cases

[sindre_mehus]

If possible, can you email me an example mp3 file with the id3 problem?

How do you "access the subsonic stream over http"? Do you use the web interface, or do you manually type in the stream url in your player?

Sindre

[d2master]

I've sent you an email (sindre_mehus@users.sourceforge.net)
with 2 mp3 example files (hope this helps).

As for the streaming, i'm accessing by entering the URL to the media player.
which means that i first need to try to connect with WMP (so a New player is created in subsonic) , then go into the web interface and load a playlist in the new created player.

a more intuitive approach would allow me to first start a playlist and then
open the url threw WMP (and connecting to one of the already running players).

thanks for the help.
________
Caprice