I've been testing video with Subsonic for several weeks now. In order to get satisfactory results, you need to have an understanding of what works best in each environment and you have to have realistic expectations for the hardware you're running. As much as it would be nice, no one is going to be watching a ripped Blue Ray on their cell phone anytime soon. On the other side of the equation, you can definitely get good results watching more than a low res You Tube video on a gigabit lan network.
Several factors come into play and you need to develop sensible strategies for getting the best results in your target environment. Connection speed, server CPU power, memory, and the recieving device are all important factors. No matter what you watch, on the fly transcoding will slow things down, so if you have the capability to store your media into a JW Player native format you can bypass transcoding and get better results with less demand on the server, which can be very important if more than one user is demanding server resources at the same time.
I've had great success ripping Blue Ray movies to MP4 format. I then create a SS player for Movies that has the MP4>flv transcode box UNCHECKED. This way the video is sent right to JW in full resolution without any transcoding. Video playback done this way plays smooth, without pause, at full HD quality, 1920X1080. This plays without a hitch on my internal network which is gigabit lan and wireless N. On a recent road trip over last weekend I was able to test playback over internet. Both my home server and the location I was at, 400 miles away, were connected over verizon FIOS connections. My network consistently achieves upload speeds of 20 - 30 Mbps, and where I was I was getting 25 Mbps download speeds. I wasn't able to just get a steady playback from start to finish with HD quality video, but if I started the video and paused it for 10 - 15 min. I could then start the movie and get it top play through without any problems - that is on my daughter's Dell XPS 15 Core i5 with 1080P screen. Trying to do the same on my old 2003 Pentium 4 Toshiba revealed that my trusty old laptop appears to finally be reaching the end of its productive life, as the graphics just weren't strong enough to keep up with the HD video and stuttering and pauses were evident. The bottom line lesson - hardware really matters when assessing just what is practical with video playback. With the right hardware and a fast enough connection, Subsonic can render a beautiful HD video without any problems.
At the other end, my EVO can do a pretty good job of video playback, even when it's transcoded video using Wifi at home. Over wireless N I can get standard DVD stored as MP4's to play well using the default MP4>flv transcoding. For reasons I don't know yet, the desktop JW Player will play an MP4 directly, but the player on Android still need to be an flv. I will most likely be transcoding and storing video for mobile to flv format and storing in a dedicated folder for mobile playback. I realistically think that over 3G video playback will most likely be limited to short home videos, pre-transcoded to low resolution flv format.
If you're not getting the results you think you should be getting, it's not Subsonic that's the problem. You can't move the contents of a six bedroom house accross the country in a Corvette. Try different setting and combinations to find what works with your particular hardware configuration. In most cases, if you want better results, your going to have to get better hardware. In any event, you should be able to achieve results that are on a par with what you typically get from You Tube, assuming you store the files as MP4 or flv so you don't have to transcode on the fly, and you have decent upload speed.
YMMV - Good Luck!
Edit: Here are the transcoding settings I'm currently using: ffmpeg -ss %o -i %s -b %bk -s %wx%h -ar 44100 -ac 2 -v 0 -f flv -
I've got a few videos up on my demo site in the "Videos" folder:
http://maplegrove.subsonic.org/login.vi ... word=guest