last.fm data incorrectly fetching

[theeraser]

Moving this to it's own thread.

... it seems my top artists aren't fetching properly from last.fm. I have over 50k plays on my last.fm, so it should be enough. It seems to be omitting certain artists regardless of the time frame selected. Any suggestions?

Perhaps on a related note, there is just one artist showing up on most played.

... you can try this:

Code: Select all
elect lastfm_user, page, session_key is null, invocation_time from library.webservice_history h inner join music.lastfmuser u on h.lastfmuser_id = u.id where h.calltype_id = 7 order by lastfm_user, page;



Page is number of days (tre months, six months, twelve months, overall (-1)). session_key tells if you've identified correctly to last.fm. invocation_time is last time we received information from last.fm.

So I ran that and the output was:
-1;f;"2012-12-13 21:37:07.246"
90;f;"2012-12-13 21:37:03.563"
180;f;"2012-12-13 21:37:05.23"
360;f;"2012-12-13 21:37:06.261"

Not really sure what that is telling me other than it received information at the time. It's very odd that only certain artists are omitted. The artists listed are in order, but there are definitely ones missing from the list.

It's odd, because awhile ago when I tested MC out, it fetched all my last.fm info perfectly. It wouldn't have to do with any other applications I attached to my last.fm profile, could it?

[hakko]

I sometimes wish I had a back door to log in and verify database status instead of posting SQL here asking people to run it... but for your own security, you have to help out and run this to figure what's going wrong.

These are the artist that last.fm has reported as top artists per user/time period (in days, -1 = forever):

Code: Select all

select lastfm_user, days, artist_name_capitalization, rank from music.usertopartist uta inner join music.artist a on uta.artist_id = a.id inner join music.lastfmuser u on uta.lastfmuser_id = u.id order by lastfmuser_id, days, rank;

You could verify that it seems to make sense.

These are the artists that are displayed as top artists in "Home -> Top artists".

Code: Select all

select u.lastfm_user, days, rank, a.artist_name_capitalization from music.artistinfo ai inner join music.artist a on ai.artist_id = a.id inner join music.usertopartist uta on uta.artist_id = a.id inner join music.lastfmuser u on uta.lastfmuser_id = u.id where exists (select 1 from library.track lt inner join music.track mt on lt.track_id = mt.id where mt.artist_id = a.id) order by u.lastfm_user, days, rank;

If artists are missing in this latter view from the first one, they either don't have "artist info" which is biography, artist image etc. We might have failed fetching that, last.fm is not always accessible. Or you don't have a single track in your library by said artist. Only you would know...

Can you see any pattern based on that?

[theeraser]

After running both those scripts, they come back with the correct artist lists from my last.fm

Definitely not the incorrect ones displayed within subsonic.

I also tried a fresh install, no luck there either.

Thanks for your dedication on this hakko

[hakko]

Could it be a caching issue in your browser then? Those are the queries that are run when the page is displayed. (with the addition that only data for the currently logged in user's last.fm account is shown, for chosen period of time)

So I can't really figure how only half of it could be returned in your browser when it works in the database. Weird.

You could try and see if it behaves the same way in another browser that you normally don't use.

[theeraser]

Hm, just tried it from my office using Chrome. Same results as firefox back at home. In addition I tried internet explorer, no luck there either.

[hakko]

You could always always do this:

Code: Select all

copy (select lastfm_user, days, artist_name_capitalization, rank from music.usertopartist uta inner join music.artist a on uta.artist_id = a.id inner join music.lastfmuser u on uta.lastfmuser_id = u.id order by lastfmuser_id, days, rank) to 'C:\top.csv' with csv;

.. and put up the resulting file (C:\top.csv) at DropBox etc or send it to musiccabinet@dilerium.se. I can't really figure what but maybe it contains an artist name with an odd character breaking a JavaScript or whatever.. I could at least have a look.

[theeraser]

Alright, I'll give that a try next time I'm home.

But honestly some of the artists that are missing are Radiohead, Crystal Castles, Flying Lotus, The XX... nothing really abstract or anything with odd characters.

[hakko]

I checked the code again and only the top 25 artists for each time period are displayed. I compared what's shown in the web interface with the database and it matches for me (the database holds top 50 for each period, and the top 25 present artists are shown in the web interface). The artists that you cannot see, are they pretty low on your most listened list or are they random, like #1, #4, #5, #8 etc?

[LJeneral]

Hi Hakko. I think I'm having a similar issue. I've only just noticed but some big artists like Radiohead, R.E.M, Queens of the Stone Age are not having their details shown when I select them from the left side. Smaller or more obscure artists have their details tho. I've tried a fresh install. And used Chrome, Firefox and Internet Explorer but the problem is still there. Any help would be appreciated.

[hakko]

I wrote the "artist details" page over a day so it's not really polished. The "Details" link only appears if you have at least one of the artist's top tracks in your library. So it could either be that you don't have enough music by that artist, or that the last.fm service was down last time you tried asking for top tracks for said artist.

The artist link in the left frame goes to artist.view?id=XXX. You could check by going to artistDetails.view?id=XXX instead and see what it displays.

Normally, an install/uninstall of the MusicCabinet software won't make a difference, as it mostly depends on what information we've been able to retrieve from last.fm. Sometimes they just give erroneous answers and then further lookups from them are paused for a little while.

[LJeneral]

Hi Hakko, just to clarify I'm also missing the band info/synopsis information for these artists.

[hakko]

It's the same story for artist biographies as for top tracks. Sometimes last.fm returns errors instead of biographies, and then we don't have any biography to display. The biographies are updated periodically so hopefully last.fm will deliver one during the next update cycle.

[theeraser]

Sorry for the late response, I've been traveling a bit.

So if I'm looking at:

Top Artists (3 Months) It's missing # 1,2,6
Top Artists (6 Months) It's missing # 1,2,5,11,14,16 (and a bunch of others going forward) *3 & 6 are swapped
Top Artists (12 Months) It's missing # 1,4,5,11,15,16,18 (and more as it continues)
Overall has a bunch missing too

[hakko]

No problem, I haven't made any progress anyway. If you pass your last.fm username (post here or pm), I can add you add as a user in my install and see if I can reproduce it.