Web interface unresponsive with many songs in the playlist

[andvaranaut]

Hi all

I am running the latest stable version of Subsonic (4.6) in Jukebox mode, and I use Chromium (and Firefox) to access it.

I have noticed that, when a large playlist is used (I noticed the problem with 1395 songs, scattered over 30 albums), whenever the song changes the web interface becomes unresponsive for several seconds. The "now playing" song is updated more or less immediately in the "now playing" area in the right, but the rest of the interface freezes up, until the "playing" icon (the loudspeaker) moves to the correct position in the playlist frame. This is specially noticeable with Chromium, which freezes for nearly a minute; Firefox also pops up the "unresponsive script" alert every once in a while.

The problem does not lie in the server itself, which keeps playing fine - it's exclusively a problem with the web interface.

I have used the Chromium profiling facility to see if I could find any clues, and it is spending a disproportionate amount of time in dwr.util.removeAllRows, called from within playlistCallback (it's over 90% of the total time spent in the callback, which, as I gather from the source, means that it's the emptying of the table which takes a long time - the contents are recreated almost immediately). I have seen that the dwr.util.removeAllRows call includes a filter which makes it remove all rows but one; perhaps it's the filtering that makes the operation so slow?

Direct inspection of the DOM tree with the Elements view of Chromium also shows the removeAllRows operation taking a long time (it is easy to see each individual row being removed from the DOM, each taking a fraction of a second, whereas the creation of all the new rows is almost immediate).

I have no experience with Prototype or DWR util.js (I'm more of a jQuery evangelist), but the (simplistic) approach I would take would be to store a copy of the "special" row, then replace the whole contents of the table with that special row (this would be easy to do in jQuery using selectors and .html()). The end result would be the same as with removeAllRows, but the need to iterate over all rows to remove them would be removed.

Best regards

[GJ51]

Ver 4.7 beta 1 is available and the database has been restructured for better efficiency. It may handle your problem much better. I have 4.7 b1 installed on two servers and like it much better than 4.6 It may be worth trying for you. Just be sure to save your C:\Subsonic directory incase you want to go back.

[andvaranaut]

Thanks for the suggestion, I'll give it a try. However, the problem apparently does not lie in the database itself, but in the web interface (ie. it's a Javascript/browser problem, not a database/server problem).

Best regards

[hakko]

Very good bug report! Would you be interested in a general overlook of the playlist frame? I think there are a few appealing ideas, like HTML5 drag'n'drop support, selecting multiple lines in the playlist, dragging them around.. just like a normal desktop program would behave. Or do you rather hope that Sindre/someone else will look into this particular issue and solve it? I'm always a bit unsure whether it's worth polishing the current UI, or if the aim should be for a re-written one.

To me, the playlist frame is currently the weakest point and I'd like to improve it. However, I'm more of a back-end developer so I'd need to team up with someone with graphic skills.

[andvaranaut]

Thanks!

I would be happy to help improve Subsonic, with two catches... first, I'm not going to have much free time until at least 2 weeks from now; and second, I do not have previous experience with Prototype, which seems to be the framework of choice for Subsonic. So for the time being I'm hoping that somebody else will rise up to the challenge I have not been able to locate the Prototype equivalent of jQuery's .html() method, but I suppose that it should not be too difficult to replicate.

The desired functionality would be a two liner in jQuery:

var a = $('#pattern').clone();
a.appendTo($('#playlistBody').html(''));

I agree with you in that a redesign along the lines you mention would be nice, but I would need to study the inner behavior of the playlist frame more carefully before being able to make more practical suggestions about how to do it.

Best regards

[hakko]

Oh, it's so easy getting stuck in thinking of how it's done today and then adding small tweaks, rather than doing a bigger overhaul. My impression is that Sindre specializes more in server development and that the web interface isn't really bleeding edge as of 2012. (No offense meant - I'm using his code every day and I donated to this project.) But I think one shouldn't stick to Prototype just because it's there already. It might be tempting to achieve small goals quickly, but in the long run, it might be smarter to replace it with jQuery now already.

I'm not striving to have a new playlist frame tomorrow. I keep it a long term goal though, creating something better, and then it's maybe a good idea to build from the ground using the libraries that are available today, rather than adding to something from five years ago.

If you do look into it when you have some spare time, please post on the forum! I'd be happy to discuss it and integrate it into the mod I'm writing: viewtopic.php?t=9504

[GJ51]

I love the conversation here as I agree that it's probably getting harder and harder for Sindre to keep up with the back end, the front end, and the app development at the same time. I think it would be great if Sindre did some extensive communication with a few of you that have the time and skills necessary to improve this great platform so that it can evolve more quickly, yet still be a high quality bug free product.

I've been begging for some front end improvements, like resizable frames (and I think your ideas regarding drag and drop would be welcome as well), a Photo album viewing/slideshow mode, getting rid of the white flash effect when the main screen redraws - and I guess I'll leave it ther for now.

I think it would be great if this evolved into a more guided effort from Sindre rather than what appears to be happening now where good mods are just eventually rolled in. Although this is helpful, I've been around since ver 3.8 and I have learned that it's just too much work going with the custom mods as you currently have no idea how much of a hassle it will be when the next ver release is rolled out.

A coordinated effort might be more efficient.

[andvaranaut]

I haven't yet delved into the source code, and I don't know whether there is some kind of API document for the web interface... But I agree that a fresher look for the default interface sounds like an exciting project (I would go for a frameless design with a more advanced JS framework to make animation, interaction, etc. easier).

Although right now I am not in the position to contribute much time, this might change in the near future. Since 4.7 is about to go live, perhaps it would be a nice project to tackle for 3.8. The interface is, in essence, a collection of HTML files, so it should be possible to have an option to switch to the 'experimental' interface and keep the current one around for those who prefer it.

Best regards

[hakko]

GJ51: It doesn't really seem like Sindre wants other developers to join though, does it? I really like the platform he built, but I think my project is going in a different direction than his. You can't ask the everyday user to install and run Postgres (like I do) and also, the license under which I fetch data from last.fm requires the project to be not-for-profit and I don't think Sindre is gonna give up on donations. So I think there's room for an open source mod that is more of a collaborative project, that people use without actually planning to go back to Sindre's new releases.

andvaranaut: If you happen to get some free time to play around with it, I'd suggest looking quickly at http://www.subsonic.org/pages/api.jsp and then check viewtopic.php?f=8&t=7553 thoroughly!

Sorry for hi-jacking this thread for a completely different topic btw.

[andvaranaut]

hakko, I think that an interface redesign needs to go far beyond what is offered in the REST API. Things like switching players or accessing the currently active playlist in each of them, for instance, don't seem to be supported in the API. Then there's the problem of accessing logs, changing setup options, things like that. I would expect a full interface redesign, even being able to reuse pieces (the current frame juggling can be converted to AJAX calls, while leaving most of the logic intact) to be quite a project. And mantaining two interfaces in parallel would be a nightmare, so to "graduate" to mainstream Subsonic, the newly built interface must replicate all the functionality of the old.

I'm not trying to rain in anybody's parade, just making the point that there's probably much more work involved in an interface redesign that in creating a standalone player.

[hakko]

What approach would you recommend from an architect point of view? Keeping the Spring MVC and the JSP it produces (but try to fit the different pieces produced into a single page using some AJAX magic), or extending the REST API to expose all functionality that's being used from the user interface today and build an interface built around that? Or some third method?

It will be a lot of work, I'm not of a different opinion. But since I spend so much time with my music player, I think it's worth putting time into it, to get the product I actually want myself (cause nobody else seems to have built it yet).

[andvaranaut]

I'm not much of an architect from a Subsonic standpoint, since I know next to nothing about Java or JSP (I'm a PHP fanboy at heart). However, from what I have seen (still tentative), I think that either course of action would work. A significant chunk of the current interface either uses AJAX calls to the REST API (eg. the chat/now playing frame makes a request every 10 seconds) or could be trivially reproduced by using this (eg. the main index list of random albums). In fact, since Subsonic is able to return the REST results as JSON, integrating a lightweight JSON templating method (which takes ~ 20 lines of Javascript code to implement) might make for cleaner and more concise code.

Given that, in an ideal world I would personally favor expanding the REST API and having the web interface use exclusively the API, as would any other application. In addition to cleanliness (the Subsonic interface would be another completely separate application; you could even serve it from any other webserver and customize it to your heart's content), this would expose a lot more functionality for third party apps, which they probably can profit from. However, this does require a lot of work in the Subsonic core and good knowledge of its inner workings.

Examples of REST API categories to expose in order to fully replicate the existing functionality would be settings, changing listeners on the fly, podcasts, reading/setting the current playlist for a player, accessing logs, accessing statistics (it would be awesome to be able to make complex queries to the database to gather more detailed statistics, but it would be difficult to implement it without it becoming a huge security hole)... There are quite a few loose ends which the web interface provides and which are not yet available via the API.

We have a saying in Spanish which, more or less, goes as follows: "best" is an enemy of "good enough". In this case, "best" would be the API expansion and "good enough" would be reusing most of the current code and JS logic, but revamping the appearance by using more modern techniques and turning most of the current HTML/CSS/presentation JS upside down.