Subsonic 4.7.beta3 released

[tigro]

Hi


Got a database issue on 4.7b3:

[2012-08-15 17:51:53,623] ERROR MediaScannerService - Failed to scan media library.
org.springframework.dao.TransientDataAccessResourceException: PreparedStatementCallback; SQL [update artist set present=?, last_scanned=? where name=?]; java.io.IOException: S1000 Data file size limit is reached in statement [update artist set present=?, last_scanned=? where name=?]; nested exception is java.sql.SQLException: java.io.IOException: S1000 Data file size limit is reached in statement [update artist set present=?, last_scanned=? where name=?]


java.sql.SQLException: java.io.IOException: S1000 Data file size limit is reached in statement

Full detail here:
http://pastebin.com/C5fyupDM

Is there a way to know the problem ? This file collection indexes fine with the 4.6 release

Thanks

[hakko]

I don't think that an SSL connection is so appealing. Partly, I have to admit, this is a user interface issue. Using the self signed certificates (and how many Subsonic users know how to issue a valid certificate for their domain?) has my non-techy friends arrive at a big warning screen saying "This connection cannot be verified" and having them manually accept the certificate and add an exception yada yada. Most people don't get past that.

Secondly, from a more technical perspective, using SSL feels inefficient for a REST service. SSL/TLS is cheap for a durable HTTP session between two computers. Our scenario is repeated short stateless interactions. Adding the initial handshake/negotiation every time you change directory seems wrong to me, when it would be enough to calculate a hash sum of the request parameters plus a secret key and have the server validate it. Plus the more obscure caveats: URLs normally gets logged by web servers (and you don't want your password written there). If invoked from a browser, it gets cached there too.

Lastly, I don't understand how server security can be optional with default value false. If SSL is what keeps unwanted visitors out of your server, then at least make it mandatory?

[WIL]

Not sure what's going on here.. all has been well, including beta 1 and 2, until upgrading to beta 3. Now when I try to open subsonic in the browser, i get a 404 error for the index.view

I tried changing ports, but it doesn't seem to help...

I've tried starting/stopping the service multiple times... doesn't help.

It's like it's no longer pointing to the proper local drive location for subsonic's web files. Did something get changed back to use the "default" hard coded install location?

I'm having this same problem too... But I can't get any of the 4.7.beta's to work! I'm currently running Ubuntu 12.04LTS and Subsonic 4.6. This is the first update that I have had trouble installing... All of my permissions seem [linux noob] to be correct... I've checked my ports and made sure the configuration file was correct. Is there something else I'm missing?

I remember a couple of versions back something was changed and Subsonic was very picky about admin rights - is this the case again?

Thanks for any insight!

The 4.7beta3 package works fine... I suggest you do a complete uninstallation of Subsonic and then reinstall it.

I installed 4.7.beta3 and got Error 503... So reading from passed posts, I renamed my db folder and reinstalled the update. I copied in the files from the old db hoping that I would save my users, but no luck. I was able to point to my music and start a scan. It is pretty quick for the amount I have [867GB]. Now I'm building my users - no biggie at least I have my Subsonic back!

[dr3van]

I don't think that an SSL connection is so appealing. Partly, I have to admit, this is a user interface issue. Using the self signed certificates (and how many Subsonic users know how to issue a valid certificate for their domain?) has my non-techy friends arrive at a big warning screen saying "This connection cannot be verified" and having them manually accept the certificate and add an exception yada yada. Most people don't get past that.

Secondly, from a more technical perspective, using SSL feels inefficient for a REST service. SSL/TLS is cheap for a durable HTTP session between two computers. Our scenario is repeated short stateless interactions. Adding the initial handshake/negotiation every time you change directory seems wrong to me, when it would be enough to calculate a hash sum of the request parameters plus a secret key and have the server validate it. Plus the more obscure caveats: URLs normally gets logged by web servers (and you don't want your password written there). If invoked from a browser, it gets cached there too.

Lastly, I don't understand how server security can be optional with default value false. If SSL is what keeps unwanted visitors out of your server, then at least make it mandatory?

You can create a valid certificate from a CA for subsonic for free. I have even created a guide. It is simple, the whole process can be done in about 30 min give or take.
http://forum.subsonic.org/forum/viewtopic.php?f=6&t=9977

[hakko]

Thanks! But don't you agree that it would be better if the default, out-of-the-box installation of Subsonic didn't require people to go through those extra steps? Some people ask on the forum on how to install Java. With them in mind, I think that the default mode should be made more secure than what it currently is. Isn't it better that twenty developers make minor adjustments in the code they've written on how to pass user credentials around, than to force every single user to go through creating a certificate (if they happen to have a domain)? To me, the SSL suggestions feels more like work-arounds (as if Subsonic is a monolith) than actual attempts to create secure, efficient programming solutions to an existing problem.

[jol]

I don't think that an SSL connection is so appealing. Partly, I have to admit, this is a user interface issue. Using the self signed certificates (and how many Subsonic users know how to issue a valid certificate for their domain?) has my non-techy friends arrive at a big warning screen saying "This connection cannot be verified" and having them manually accept the certificate and add an exception yada yada. Most people don't get past that.

Agree, unfortunately.

Secondly, from a more technical perspective, using SSL feels inefficient for a REST service. SSL/TLS is cheap for a durable HTTP session between two computers. Our scenario is repeated short stateless interactions. Adding the initial handshake/negotiation every time you change directory seems wrong to me, when it would be enough to calculate a hash sum of the request parameters plus a secret key and have the server validate it.

Disagree. You will have difficulties measuring the overhead, as most SSL/TLS implementations will avoid renogotiation, knowing that HTTP connections are opened/closed anytime. And then a proven concept is always better than inventing your own mechanism and figuring out later that you did a design error in your crypto protocol...

URLs normally gets logged by web servers (and you don't want your password written there). If invoked from a browser, it gets cached there too.

Agree, but has nothing to do with the choice of SSL/TLS over plain HTTP - afaik the REST API support basic authentication, which using https is totally fine, only some apps (including Android) do not (yet) leverage it.

Lastly, I don't understand how server security can be optional with default value false. If SSL is what keeps unwanted visitors out of your server, then at least make it mandatory?

Although the process is complex, the answer to your question is easy: TLS/SSL requires trust in certificates, and this requires configuration, at least unless DNS-SEC with DANE takes off sometime in the future. Otherwise you can communicate perfectly private with someone you cannot authenticate - is that what you want?

Thanks! But don't you agree that it would be better if the default, out-of-the-box installation of Subsonic didn't require people to go through those extra steps? Some people ask on the forum on how to install Java. With them in mind, I think that the default mode should be made more secure than what it currently is.

You cannot eliminate the trust issue easily. And then many of the players in the certificates market try to make a fortune. A good example of what can be done to ease the process is what Microsoft does with the Windows Home Server - you get a trusted certificate for free and installed automatically. Now if someone would come up with an easy integration with that or hide Subsonic behind IIS using AAR (I do, but not using the MS cert) this would probably boost SSL usage...

Isn't it better that twenty developers make minor adjustments in the code they've written on how to pass user credentials around, than to force every single user to go through creating a certificate (if they happen to have a domain)? To me, the SSL suggestions feels more like work-arounds (as if Subsonic is a monolith) than actual attempts to create secure, efficient programming solutions to an existing problem.

I am sure you will not come up with something more secure than SSL, and even if you can I suspect you will have a similar configuration problem than SSL. After all you want to authenticate the server before you present your credentials to it.
Best regards, jol

[hakko]

Just to make my point more clear:

I think that having a default mechanism for authenticating apps by sending username+password in clear text or using a weak hash is bad practice and bond to have user accounts compromised sooner or later. I think that this default mechanism should be deprecated (to force app developers to upgrade to a more secure authentication).

I don't want to replace the existing SSL option. I'm not trying to re-invent SSL. But I think that it would be a good idea to add a new default mechanism, where the level of security is in-between the current default level (which I think is too weak) and the existing SSL level (which is strong and fine but has other drawbacks).

[andvaranaut]

I don't think that an SSL connection is so appealing. Partly, I have to admit, this is a user interface issue. Using the self signed certificates (and how many Subsonic users know how to issue a valid certificate for their domain?) has my non-techy friends arrive at a big warning screen saying "This connection cannot be verified" and having them manually accept the certificate and add an exception yada yada. Most people don't get past that.

You do have a point, but until now I have had no problems telling people to just bypass the authentication screen when giving them their username/password. It does feel clumsy, specially if you use the Subsonic redirection service (which redirects to an IP, which will never match a self-signed certificate with a domain name, which means that there is no way to avoid the scare screen in browsers such as Chrome), but it has not been a showstopper in my experience. My gut feeling is that most people would tend to click through without thinking twice, although perhaps your friends are smarter in the security department than mine

Secondly, from a more technical perspective, using SSL feels inefficient for a REST service. SSL/TLS is cheap for a durable HTTP session between two computers. Our scenario is repeated short stateless interactions. Adding the initial handshake/negotiation every time you change directory seems wrong to me, when it would be enough to calculate a hash sum of the request parameters plus a secret key and have the server validate it.

I'm nowhere near being a REST guru, but isn't being stateless one of the key points of REST? There is no way (at least, no obvious way) to avoid the negotiation and remain stateless. Any challenge authentication mechanism needs a different secret key per session (or request) to be valid, so you would need the secret key to be somehow stored on the server and either be generated on the fly for each request/session (implies handshaking, defeats statelessness) or be fixed per user (which would mean that, unless some periodic key-changing mechanism is used, a "captured" REST URL will still be valid even if the user/password is not known). The de facto standard for this would probably be OAuth, which is quite a can of worms to open.

REST over SSL solves most of these problems. I'm not arguing that it's the absolute best solution, but it does get close without needing a major reworking of the authentication system, by delegating the handshaking/encryption to a separate (and well tried) protocol. And, fortunately, it is now perfectly possible to encrypt everything with SSL at negligible computational cost. REST+SSL is (roughly) equivalent to having a separate challenge per request, whereas REST+SSL+persistent connections is roughly equivalent to a separate challenge per session (the ideal behaviour).

Plus the more obscure caveats: URLs normally gets logged by web servers (and you don't want your password written there). If invoked from a browser, it gets cached there too.

Well, Subsonic runs its own server, which means that it can control what it logs (besides, if you can access the Subsonic logs, you already have access to the system). And the Subsonic API allows HTTP Basic authentication to be used, which probably is a more sensible mechanism than passing user/password in the REST URLs. Besides, I don't see any obvious scenario (other than testing) in which manually performing REST calls from a browser is needed.

Lastly, I don't understand how server security can be optional with default value false. If SSL is what keeps unwanted visitors out of your server, then at least make it mandatory?

I agree that this should be enabled by default; the only reason I can think of is the 'scare screen' problem.

By the way, I do agree with your point that a more secure authentication mechanism for apps would be desirable, although I don't think that 4.7 should be the place for it (it needs time to plan, implement, test and provide a sensible deprecation path for the current mechanism). I think that discussing this in a new thread in the 'feature requests' section is probably more sensible.

In order to be constructive, perhaps a good course of action would be providing a handshake within Subsonic - begin all communications by requesting a secret key (unique per session and per user), then provide a hash of the URL + password + secret in the REST request. A 'serial number' (an unique, arbitrary id, with the restriction that every request should have an id greater than all previously passed ids in a given session) could be included to make all URLs unique. This would be reasonably secure, even with two technical weaknesses: it would need the Subsonic server to keep track of the secrets, ids and sessions (which goes against the REST philosophy and might be complicated) and it essentially means replacing the SSL connection with an alternative (and inferior) mechanism. But hey, you need to compromise somewhere.

Best regards

[rastapi]

Hi Sindre and others.

Thanks for a wonderful 4.7.beta3. It has been working near perfectly- on my raspberry pi with raspbian installed. However I have come across what seem like bugs- namely that java can go haywire for a bit. It has ran perfectly most of the time for the last two weeks, but the other day, and then this morning, java makes cpu usage go right up, to 99% at times. This makes the raspberry pi completely unusable and means I cannot connect to subsonic locally or externally. When this happened the other day I reinstalled 4.7 and this remedied the issue. Then, after putting up loads more music last night (and going to bed thinking I should not hack anything today), I woke this morning with all music working but then felt motivated to have a mess with icon files. I didn't actually change anything, but after a sudo service subsonic restart java made cpu usage go to extreme heights again.

Quite frustrating because it renders everything usless, but I suppose I can expect such things to happen on such a machine.

This seems like a java bug with subsonic because it is intermittent and raspberry pi has handled it fine previously, though I do understand java on the pi is always going to leave some issues.

Could anyone give me advice as how to make cpu usage better?? or anything at all?

Thanks.

[hakko]

I'm nowhere near being a REST guru, but isn't being stateless one of the key points of REST? There is no way (at least, no obvious way) to avoid the negotiation and remain stateless

I don't think I agree on this. (But I might be wrong, I'm not a crypto expert!)

Imagine this: the Subsonic server knows the names of all users, and the hash of each user's password.

The app knows one username, and one hashed password.

When the app sends a request to the Subsonic server, the request itself is not secret. It's something like /rest/getArtists.view?u=hakko&id=42 for getting artist number 42.

Now, imagine that before creating this request, the app create two new variables, $timestamp=1345134090 (unix timestamp) and $token=hash(/rest/getArtists.view?id=42timestamp=1345134090&u=hakko&hash(password)). Hash is a strong hash function like SHA256 or whatever and the parameters are UTF-8 and sorted with hash(password) appended last. Then, what the app actually sends to the server is /rest/getArtists.view?u=hakko&id=42&timestamp=1345134090&token=$token.

The Subsonic server can read this request, look up it's hashed password for user hakko, and by using the same hash function as the app used, it can calculate what it would expect the token to be. If it matches with what the app sent, all is well. If it doesn't match, or if the timestamp is too old, an error message is returned (to avoid replay).

What we rely on here is the strength of the hash() function. If it feels weak, we could do hash(hash(hash())) etc.

I believe that this is a very small addition to the current protocol. It doesn't require handshakes/negotiation. It's not as safe as SSL. But it stops passwords from being passed around as they are today without the average user knowing how insecure the handling is. I think that's a pretty good addition.

[jol]

I think that's a pretty good addition.

Sorry, disagree. Essentially your algorithm replaces the password by its hash, i.e. if someone gets access to the hash he can impersonate the user. In fact the change would result in making password hashing in the user store useless. And btw, password hashes are not really secure, best practice is salted hashed passwords, with sufficient length salt and hash length and a good hash algorithm (I have seen bad implementations of all). How can your proposal support this? And what if Subsonic authenticates against an external user store like LDAP?

And then a proven concept is always better than inventing your own mechanism and figuring out later that you did a design error in your crypto protocol...

I did not expect this to happen that fast...
Best regards, jol

[hakko]

Today, Subsonic stores password in clear-text (hex) and sends them in clear-text (hex), unless you explicitly turn on SSL. As I've said, what I'm suggesting here is a way of making this default mode more safe. I'm not suggesting replacing SSL for those who want to use that. I don't claim that this is the new industry standard of securing things.

The idea is to replace the password with a hash, that's correct. If someone gets access to the hash, he/she can impersonate the user. That's also correct.
Now, the current authentication forces clients to store user passwords in clear-text, and, in default mode, to send them in clear-text. That's about how easy it gets for somebody to steal the password, impersonate the user, and immediately start trying the same password at other services. I fail to see why it's not an improvement to send hashed passwords instead. (note: improvement)

What do you reckon is the biggest risk that somebody would get access to the hashed password this way?

How do you think that Subsonic apps should be made safer in default mode?

[GJ51]

Isn't the easiest way to just isolate the host server from the rest of the network? Either in a VM or a seperate machine that has limited access to other network resources?

Can't the user account for the SS service logon be restricted to only access the defined folders through security settings?

[jol]

Today, Subsonic stores password in clear-text (hex) and sends them in clear-text (hex), unless you explicitly turn on SSL.

The storage is unsafe, but irrespective of whether SSL used for transmission. Paranoid guys like me can use LDAP authentication to avoid using Subsonic´s user store. In fact it was not paranoia but the fact that I need to double maintain users, and in order to avoid double maintenance, I implemented a mini-LDAP server that reuses the local Windows user store which is pretty safe, and users will not have to remember distinct passwords (or have the same password compromized by the week store of Subsonic).

I'm not suggesting replacing SSL for those who want to use that. I don't claim that this is the new industry standard of securing things.

But your suggestion reduces the security level of those using SSL or external user stores as well, and is incompatible with the request to implement a more secure user store in Subsonic.

Now, the current authentication forces clients to store user passwords in clear-text, and, in default mode, to send them in clear-text. That's about how easy it gets for somebody to steal the password, impersonate the user, and immediately start trying the same password at other services. I fail to see why it's not an improvement to send hashed passwords instead. (note: improvement)

A client is not forced to store the password at all, and it is a bad practice to store them unencrypted. But I know it is done, thus my Android mobile has its own user&password with close to no authorizations. During transmission the password is protected as I am using SSL, and I definitely recommend SSL to anyone exposing Subsonic to the internet.

What do you reckon is the biggest risk that somebody would get access to the hashed password this way?

As you recognized users tend to reuse passwords for multiple services. Thus one can easily take the password hash, use a rainbow table and lookup the original password and then try to authenticate somewhere else. Remember Linked-in bashed last month for not using salted passwords? And then of course your proposal is incompatible with external user stores or Subsonic using salted hashed passwords.

How do you think that Subsonic apps should be made safer in default mode?

It is not just about Subsonic, this is local optimization if at all. Think big and invest into tools and infrastructure that make adoption of SSL/TLS, DNS-SEC, and DANE easy for the mainstream of users. Actually I think this is all slow, because thought to the end, it endangers the business of certification authorities like Verisign.
Best regards, jol

[SenSurroun]

Also, while I'm pretty happy with 4.7, I really miss the old playlist directory functionality. I had a script that exported a bunch of smart playlists from iTunes to the playlist directory on a nightly basis. Maybe I could get by if I could trigger playlist import from the API.

I agree. I was doing something similar, but just with a manual execution of the script when I wanted the playlists exported from iTunes. Subsonic does not manage the location of the files, so how will it know to correct the playlist if an outside program moves the file? I understand why you'd want to store the playlists in a database, but there needs to be a way to import multiple instead of just one at a time. And if the import won't overwrite a playlist with the same name, there also needs to be a multiple delete option.

Other than that, I've enjoyed 4.7.beta3.