[Solved] subsonic-service.exe running at near 100% CPU

[lovebags]

I am finding the subsonic-service.exe running at near 100% CPU under what normally should be a relatively low user load on the server. Although today there have been 105 users who have logged in to the server (there are now 850 players listed in total) there would only be around 10-20 listening at any one time. I recently had an extra CPU to the VM when I saw this happen the first time. As shown in the screenshot the "service CPU usage" graph shows it below 50% but CPU 0 and CPU 1 peaking at close to 100%. (RAM usage on the VM is at 1339MB with 683MB available). I wonder if this is anything to do with having 244 threads running?

I have seen much lower CPU usage when threads is at around 40 and there is a similar low amount of disk activity going on. For example on an identical test server with one CPU I have simultaneously streamed over 30 songs at once and the CPU barely hit 20%. However this was played all from the one computer.

Subsonic Version: 4.5 (build 2385)
Server Version: jetty-6.1.x, java 1.6.0_18, Windows Server 2008 R2 (200.3 MB / 247.5 MB)
Hardware Platform: (Running on a VM in our data centre) Windows Server 2008 R2 Enterprise, Processor Intel Xeon CPU X7550 @ 2.00Ghz (2 Processors) Installed memory (RAM): 2 GB
Java Memory Limit: 256MB

Last ten log file lines:

Code: Select all

[5/20/12 9:23:09 PM EST]   INFO   StreamController   Got range: Range[714156,9223372036854775807]
[5/20/12 9:23:11 PM EST]   INFO   PlaylistInputStream   user1 listening to "Week 05\19 Blue Yodel No. 8 (Mule Skinner Blues).mp3"
[5/20/12 9:23:25 PM EST]   INFO   StreamController   Got range: Range[714156,9223372036854775807]
[5/20/12 9:23:28 PM EST]   INFO   PlaylistInputStream   user1 listening to "Week 05\19 Blue Yodel No. 8 (Mule Skinner Blues).mp3"
[5/20/12 9:23:43 PM EST]   INFO   StreamController   Got range: Range[714156,9223372036854775807]
[5/20/12 9:23:45 PM EST]   INFO   PlaylistInputStream   user1 listening to "Week 05\25 Midnight Special.mp3"
[5/20/12 9:23:47 PM EST]   INFO   PlaylistInputStream   user1 listening to "Week 05\19 Blue Yodel No. 8 (Mule Skinner Blues).mp3"
[5/20/12 9:23:48 PM EST]   INFO   PlaylistInputStream   user2 listening to "Week 11\1-02 Don't Think Twice, It's All Right.mp3"
[5/20/12 9:23:50 PM EST]   INFO   PlaylistInputStream   user1 listening to "Week 05\25 Midnight Special.mp3"
[5/20/12 9:23:56 PM EST]   INFO   PlaylistInputStream   user3 listening to "Week 08\08 Jesu Meine Freude (So Aber Christus In Euch Ist).mp3"
[5/20/12 9:24:13 PM EST]   INFO   StreamController   Got range: Range[3700108,9223372036854775807]

[lovebags]

I did a software upgrade to 4.6 and seemed to solve the issue instantly. The other thing I didn't check was the ffmpeg process in the resource monitor so I'm not sure what that was doing at the time. But now with around 40 users logged in at once and an around of 10Mb/s outbound, the CPU is hovering at an average of 45%.

[lovebags]

Interestingly after another recent night of heavy usage the CPU once again got pegged at near to 100% with around 25-30 users logged in at once over a few hours. Once they logged out, things went back down to low level CPU usage so I'm not sure if this issue is really resolved at all. My other option could be to go to 4 CPU's to cope with demand. We only get around 2 massive spikes in usage during the year for a week around exam time when students do all their listening online. The other thing I have tested is that if one user from one computer opens 30 browser windows and plays music on loop from all windows, the CPU doesn't really skip a beat and peaks at about 15%. I'm wondering if there is something about unique users and different OS's being logged in a at once that causes issues with the software/hardware?