network time out on large folders

[teknojnky]

Hello,

Using android subsonic app (sprint evo 4g, custom rom) and subsonic server 4.2 (readynas/linux).

Whenever I attempt to browse a folder with a large amount of subfolders, I get network timeouts and can not navigate any further down that folder path.

Other folders and general connectivity is fine, I can download/stream fine otherwise.

Is there an option on the client or server to either increase the timeout on folder listings or send the folder tree in smaller sections (ie 100 max at a time) ?

I am sure the obvious answer would be to divide up the folder so it contains less folders, but that is not something I want to do as I have my library organized specifically and in an semi-automated manner. Dividing it up further would greatly complicate things.

[GJ51]

Sounds a bit more like connections issue than a size issue. I've connected my EVO 4G to some pretty big collections without a problem. One I remember was over 600GB, but then someone yesterday was talking about 6TB collection size, so I guess unless you specify how big we're talking, it's all relative.

Have you tested on other comparably sized servers? Do you get same results using Wifi and 3G or maybe 4G? I've got about 2000 folders in my primary library and they load in a matter of a few seconds over 3G.

[teknojnky]

I was using 4g.

But I am trying wifi now and it is doing the same thing. (wifi/nas/phone all same network)

I am monitoring the nas via istat on a ipod touch, and whenever I attempt to go to a large folder, I see the cpu max out.


there is nothing going in the log while this happens, here is the last couple entries from a recent search rescan:

Code: Select all

[2011-01-05 22:05:06,067] INFO SearchService - Updating Lucene search index.
[2011-01-05 22:05:32,404] INFO SearchService - Created search index with 213422 entries.

the library is very large as you might infer from the search index, approx 1.2tb of mp3 (and some art) only.

Smaller folders, of about 300 work, but around 450+ starts throwing retry errors and failing.

The largest folder I have contains about 700 sub folders.

exact version info:

Code: Select all

Version   4.2 (build 1944) – November 21, 2010
Server   jetty-6.1.x, java 1.5.0_14, Linux (179.6 MB / 572.2 MB

thanks for any further insight

[alphawave7]

Sounds like you've overloaded the NAS' RAM..how much is it supposed to have?

[teknojnky]

it has 1 gig ram 1 gig swap, atom single core with hyperthreading.

I can watch memory use via istat, and while mem usage does increase, it doesn't max out (no swap used), and besides there are no out of memory errors from subsonic.

fwiw, subsonic max memory = 800

[alphawave7]

Interesting..typically a spiked processor is accompanied by packed RAM on the NAS devices I've tried...and I agree, I almost never saw memory errors, just a slammed, unresponsive NAS.

[teknojnky]

the (sustained 100%) cpu spikes that occur with the large folders, idle out after 10-30 seconds from the network error on the client.

cpu usage for less large folders typically only lasts a couple seconds and only hits 50% cpu or so.

for example, my 'a' folder contains ~300 folders, cpu spikes @ 50-60% for about 3 seconds.

my 's' folder contains ~460 folders, cpu starts @50-60 then either spikes to 100% for a few seconds and returns the folder list to the client (some times with retries, sometimes without).

my 'soundtracks' folder contains ~650 folders, cpu starts @ 50-60, then maxes out 100% for entire time that the client is retrying, then idles out about 20-30 seconds later.

here is a 'top' of the cpu during the attempt to list soundtrack folder

Code: Select all

top - 23:24:01 up 26 min,  1 user,  load average: 2.72, 1.94, 1.13
Tasks: 124 total,   2 running, 122 sleeping,   0 stopped,   0 zombie
Cpu0  : 70.5%us, 10.8%sy,  0.0%ni,  1.3%id, 16.1%wa,  0.3%hi,  1.0%si,  0.0%st
Cpu1  : 66.0%us, 12.9%sy,  0.0%ni,  1.7%id, 18.8%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:   1014128k total,   926096k used,    88032k free,    31212k buffers
Swap:  1048440k total,        8k used,  1048432k free,   123168k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
2948 root      20   0 1069m 420m 7380 S  151 42.5  12:10.35 java
2674 admin     20   0 97836  38m 3860 R    4  3.9   0:05.09 apache-ssl
2634 root      20   0 32608 2876 1864 S    3  0.3   0:25.40 leafp2p
  315 root      20   0     0    0    0 D    1  0.0   0:02.88 kswapd0
4125 root      20   0  2308 1132  832 R    1  0.1   0:06.01 top


[ShadowVlican]

this is what subsonic developers should be fixing...

it takes far too much resources and time to load a folder that contains many subfolders

this problem extends to large playlists as well

[teknojnky]

some more info

via istat, I can also see the network bandwidth used.

when I was first investigating I had other stuff going on network-wise on the nas, so I could not see what/how much bandwidth was being used by subsonic.

so I was checking now, with nothing else going on the nas network, I can verify that it is not related to any network connection or bandwidth issues, since I can there is essentially nothing being uploaded/downloaded while subsonic tries to process the folder list.

when browsing a working folder, I see subsonic do its processing, then a nearly instant peak of about ~400kbs (which is near my my upload cap @ 512) when the client recieves the folder list.

I get no upload peak on the folders that fail to load, indicating to me that either the server is failing to send (my guess) or the client has already closed the connection and won't accept the late upload (possible).

[BuddyCasino]

I have the same problem. Server is running on a Core i5. One folder has about 2000 files of .aac music, and it always times out when using the Android client. One core is maxed out until the timeout comes.

[zombie]

I am having the same problem on ubantu its eating up my hyper thread p4 and not using much RAM. When I was running SS on Windows I don't think I was having the same problem. An error in the code?? Maybe...

[ShadowVlican]

not error in code, since it is eventually able to load (if you have patience to keep trying lol)

i'd say this is either inefficient coding or he has implemented an inefficient way to grab folders/playlist content

[steve2045]

i'm having this issue as well on android

any solution?

[allroy1975]

I have the same kind of issue. I have a playlist with a few thousand songs in it...it's a 163k file. It takes a minute or so to load, I usually see a "Subsonic isn't responding - Force Close - Wait - Cancel" message. Clicking wait works, but it's a hassle.

It's funny to me (not funny..but you know..) that as soon as the 163k playlist loads the songs sarts downloading like 1.5mb a second over the WAP.

The way I do things on my set up is I have a Shuffled Favorites m3u where once a day @ 3AM the playlist gets re-shuffled via a scheduled command line. In the morning while I'm getting dressed I load that playlist up and set the songs to cache to Unlimited and start downloading songs over my Wireless router. That way I've got a good head start for the drive in and my day's music and the playlist is different every day and I use less 3g and less battery. My EVO is pretty good with battery life, but probably because I do things like this to help it.

[nibato]

I've been having this issue as well on my gentoo nas box. It seems it doesn't matter how much memory i allocate for it, it always uses 100% cpu and times out on my large music folders.