MusicCabinet - artist radio, genre radio & related artists

[hakko]

500.000 files should be a pretty good stress test in itself.

Yes, additional rescans (they take part automatically when you update the Subsonic search index) do two things:
- they look for newly added artists and albums and fetches info for them
- they pick the oldest 1/30 of your current data and fetches updates for that (to keep it updated over time)

Since you have so much data, you might want to check the log for how long the import takes. Both this first time (that's an interesting stat figure), and on the next subsequent scan. The MusicCabinet update is asynchronous, meaning on subsequent scans, you can still use your existing index for generating playlists while it fetches new last.fm data. But the Subsonic Lucene index is not (I think) so maybe this will stop you from searching for a little while every night.

I could make the figure 1/30 configurable so that each update would become smaller for you (and the refresh cycle for your library slower).

It's also interesting that there's such a difference in progress for different imports. I wonder if I need to make the parallel execution of them more fair.

Thanks for the update!

[hakko]

Minor update: On request, I've added the possibility to select all / none genres, and to configure how many songs to include in generated playlists.




It's up as a new .war file at http://dilerium.se/musiccabinet/subsonic.war (0.5.35).

[hakko]

Minor update: Aktarus asked if it was possible to have the newest albums displayed first on the artist page. And also if it was possible to choose which tab to open by default when pressing the Home button (Random, Recently added, Top rated etc).

Now it is! Each user can choose to have it their way.



Replace your current subsonic.war with http://dilerium.se/musiccabinet/subsonic.war (0.5.35b) and restart your Subsonic service.

[Aktarus]

Hy,

You're the Best !!! I will try it this afternoon when I'll be back at home

MERCI

[TheWarden]

I tried this on linux (Fedora) and I'm pretty sure the postgres service is running and setup correctly but MusicCabinet doesn't seem to see it. There are no attempted login attempts in the postgres logs either.

Daire

[hakko]

What's happening when you go to Settings -> MusicCabinet? Something like this? http://i.imgur.com/ap2YB.png

Are the Subsonic process and the Postgres process running on the same machine? From the machine where Subsonic is running, can you issue "telnet localhost 5432" and get a connection? Or do you get something like "Connection refused"?

Do you have any internal firewall that could stop it (unlikely)? In your /etc/hosts file, do you map "localhost" to 127.0.0.1?

Which version of Postgres are you running?

[TheWarden]

What's happening when you go to Settings -> MusicCabinet? Something like this? http://i.imgur.com/ap2YB.png

Yep that's the one.

Are the Subsonic process and the Postgres process running on the same machine? From the machine where Subsonic is running, can you issue "telnet localhost 5432" and get a connection? Or do you get something like "Connection refused"?

I seems to seems to be connecting... well it doesn't say connection refused but telnet eventually times out (Connection closed by foreign host).
# netstat -l | grep 5432
unix 2 [ ACC ] STREAM LISTENING 12610058 /tmp/.s.PGSQL.5432

Do you have any internal firewall that could stop it (unlikely)? In your /etc/hosts file, do you map "localhost" to 127.0.0.1?

Nope. iptables is disabled. "127.0.0.1 localhost.localdomain localhost" in /etc/hosts.

Which version of Postgres are you running?

Running 9.1.3. Does it really have to be 9.1.13 exactly?

[hakko]

I seems to seems to be connecting... well it doesn't say connection refused but telnet eventually times out (Connection closed by foreign host).
# netstat -l | grep 5432
unix 2 [ ACC ] STREAM LISTENING 12610058 /tmp/.s.PGSQL.5432

That looks good, it's expected behavior. It's waiting for you to send some HELO characters and when you don't, it closes the connection.

Running 9.1.3. Does it really have to be 9.1.13 exactly?

Nah. One guy reported errors for 8.4.8 (his installation hadn't plpgsql by default and function names where truncated to 32 characters). I just wrote the latest version but I'd expect it to work with the latest major versions (7.* or so).

Since you've given such a detailed error report and everything looks good in theory, the only solution I can think of is adding more logging to nail what's going wrong. Do you build from source or use the subsonic.war?

What's happening behind the scenes is that it assumes that Postgres isn't there. Then, it tries to open a socket to localhost:5432. If that works, it writes a certain sequence of characters to the stream (what psql would write) and then it checks that the answer returned is what Postgres would write (to make sure it's not another service running at port 5432). Only if this succeeds, it decides that it's OK to continue.

I'd be interested to see which part's going wrong, but to do that, I'd need to either send you code or a new subsonic.war with added logging. Would you be up for that? I'd appreciate it but I don't expect you to go out of your way to help me.

[TheWarden]

Yea no problem testing things for you. Just put the war somewhere and I'll install it.

[hakko]

Thanks! I have a new subsonic.war with added logging (no new features) up here: http://dilerium.se/musiccabinet/subsonic-thewarden.war

Rename to subsonic.war and install as usual. There should then be a file called musiccabinet.log in your temp directory (wherever that is.. probably /var). I guess you have to search for it.

Just paste the content from it here and I'll see if I can make something out of it!

[jfmus]

Really loving this mod, but I am noticing one quirk . I have two folders that I am pulling my library from. Studio and Bootleg. So when you are in the Left frame, you see artists that can be filtered by Studio Folders, Bootleg Folders, or All Folders. Once the Music Cabinet has created the database, it creates two sets of related artists in the main frame. So in other words, if you choose the Related Artists link you get a duplicate listing for an Artist with the studio folders in one link, and the bootleg folders in the other. Any suggestions?

Thanks,

jfmus

[hakko]

jfmus: I actually fixed that yesterday night, I have a friend who organizes his music kind of the same way who made me aware of it. The new version removes the duplicate from the "Genre" view and the "Related artists" view, leading to the same artist view as you would end up on from the left frame.

The bad news is that it's still not gonna work perfectly from other links, like "Home -> Recently added" or from links in the playlist view. I'll post here when I have an update for that (it might take a while, I consider waiting for the 4.7 release that will help take care of it).

Minor update:

I also noticed that some artists get an ugly biography, if they share their name with other artists.
Like this: http://www.last.fm/music/First+Aid+Kit

Therefore, there's now a manual update possibility for artist biographies:



The new subsonic.war is up at http://dilerium.se/musiccabinet/subsonic.war (0.5.37b)

[jfmus]

Thanks for the response....I will give the new version a try and report back. As far as perfect goes, this is such a leap forward from what subsonic could do before that the warts are really not a big deal. Keep up the good work. It is appreciated!

Also, I just noticed another thing in relation to the duplicate artist. The same thing is happening when you do a search. The results return an artist line for each root folder.

jfmus

[hakko]

Thank you! It makes me happy to hear. But keep reporting the oddities and I might fix them!

Edit: yes, regarding the search, the playlist and so on, that's Subsonic issues. I guess Sindre is aware and I think it's (rather coincidentally) gonna get fixed in 4.7.

[Ultraviolet]

Well, the current limit of what this can handle is somewhere below 500,000 tracks. It has slowed to the point where completion time will easily be in the months, if not years. I actually stopped it after a few days to see if I could restart the indexing again and have it pick up where it left off, but nothing was saved to postgres so I started from scratch again. Is it possible to do some chunking so that info is saved to the db incrementally?