Indexing, searching, and browsing

[mookieproof]

For some reason, Subsonic reads everything in the ID3 v2.3 tags for mp3s except the year (no problem with oggs or other formats). Any ideas?

From the subsonic9.index file, for example:

F ixYxi 1153377736000 ixYxi e:\albums\augustus pablo - 1976 - king tubbys meets rockers uptown\Augustus Pablo - 12 - Satta Dub.mp3 ixYxi 6902678 ixYxi AUGUSTUS PABLO ixYxi KING TUBBYS MEETS ROCKERS UPTOWN ixYxi SATTA DUB ixYxi ixYxi Reggae
A ixYxi 1196560784234 ixYxi e:\albums\autoclave - 1991 - complete ixYxi 0 ixYxi ixYxi ixYxi ixYxi ixYxi
F ixYxi 1110172080000 ixYxi e:\albums\autoclave - 1991 - complete\Autoclave - 01 - Go Far.ogg ixYxi 4313889 ixYxi AUTOCLAVE ixYxi AUTOCLAVE ixYxi GO FAR ixYxi 1991 ixYxi Rock

[Dide]

Hi,

I have a strange problem when I search a file.
Is there maybe limit since I have almost 370Gb of Mp3's ?

15.510 artiest
4.607 albums
68.761 nummers
369,17 GB (~ 5.734 hours)

I can't find some files with the search option in Subsonic but they are in the subsonic9.index file.

For example, this file/line is in the index file:
F ixYxi 1188013867121 ixYxi D:\MyPublic\MyMusic\NL TOP 40 (alle singles) - 1976\76_033 - Andre? Van Duin - Willempie.mp3 ixYxi 4489216 ixYxi ANDRÉ VAN DUIN ixYxi TOP 40 JAARLIJSTEN - 1976 ixYxi WILLEMPIE ixYxi 1976 ixYxi Top 40

But I can't find it with "search".

I use the Windows installation/package.

Log:
[2008-10-21 11:40:06,306] INFO SearchService - Starting to create search index.
[2008-10-21 11:40:06,884] INFO PodcastService - Starting scheduled Podcast refresh.
[2008-10-21 11:40:06,884] INFO PodcastService - Completed scheduled Podcast refresh.
[2008-10-21 11:40:07,040] INFO SearchService - Created search index with 1000 entries.
[2008-10-21 11:40:07,587] INFO SearchService - Created search index with 2000 entries.
[2008-10-21 11:40:07,602] INFO SearchService - Created search index with 3000 entries.
[2008-10-21 11:40:07,806] INFO SearchService - Created search index with 4000 entries.
[2008-10-21 11:40:08,087] INFO SearchService - Created search index with 5000 entries.
[2008-10-21 11:40:08,540] INFO SearchService - Created search index with 6000 entries.
[2008-10-21 11:40:08,884] INFO SearchService - Created search index with 7000 entries.
[2008-10-21 11:40:09,227] INFO SearchService - Created search index with 8000 entries.
[2008-10-21 11:40:09,493] INFO SearchService - Created search index with 9000 entries.
[2008-10-21 11:40:09,806] INFO SearchService - Created search index with 10000 entries.
[2008-10-21 11:40:10,056] INFO SearchService - Created search index with 11000 entries.
[2008-10-21 11:40:10,290] INFO SearchService - Created search index with 12000 entries.
[2008-10-21 11:40:10,540] INFO SearchService - Created search index with 13000 entries.
[2008-10-21 11:40:10,790] INFO SearchService - Created search index with 14000 entries.
[2008-10-21 11:40:11,071] INFO SearchService - Created search index with 15000 entries.
[2008-10-21 11:40:11,337] INFO SearchService - Created search index with 16000 entries.
[2008-10-21 11:40:11,602] INFO SearchService - Created search index with 17000 entries.
[2008-10-21 11:40:11,852] INFO SearchService - Created search index with 18000 entries.
[2008-10-21 11:40:12,118] INFO SearchService - Created search index with 19000 entries.
[2008-10-21 11:40:12,399] INFO SearchService - Created search index with 20000 entries.
[2008-10-21 11:40:12,681] INFO SearchService - Created search index with 21000 entries.
[2008-10-21 11:40:13,040] INFO SearchService - Created search index with 22000 entries.
[2008-10-21 11:40:13,306] INFO SearchService - Created search index with 23000 entries.
[2008-10-21 11:40:13,556] INFO SearchService - Created search index with 24000 entries.
[2008-10-21 11:40:13,837] INFO SearchService - Created search index with 25000 entries.
[2008-10-21 11:40:14,102] INFO SearchService - Created search index with 26000 entries.
[2008-10-21 11:40:14,384] INFO SearchService - Created search index with 27000 entries.
[2008-10-21 11:40:14,696] INFO SearchService - Created search index with 28000 entries.
[2008-10-21 11:40:14,915] INFO SearchService - Created search index with 29000 entries.
[2008-10-21 11:40:15,181] INFO SearchService - Created search index with 30000 entries.
[2008-10-21 11:40:15,431] INFO SearchService - Created search index with 31000 entries.
[2008-10-21 11:40:15,681] INFO SearchService - Created search index with 32000 entries.
[2008-10-21 11:40:15,946] INFO SearchService - Created search index with 33000 entries.
[2008-10-21 11:40:16,212] INFO SearchService - Created search index with 34000 entries.
[2008-10-21 11:40:16,509] INFO SearchService - Created search index with 35000 entries.
[2008-10-21 11:40:16,852] INFO SearchService - Created search index with 36000 entries.
[2008-10-21 11:40:17,102] INFO SearchService - Created search index with 37000 entries.
[2008-10-21 11:40:17,352] INFO SearchService - Created search index with 38000 entries.
[2008-10-21 11:40:17,618] INFO SearchService - Created search index with 39000 entries.
[2008-10-21 11:40:17,899] INFO SearchService - Created search index with 40000 entries.
[2008-10-21 11:40:18,102] INFO SearchService - Created search index with 41000 entries.
[2008-10-21 11:40:18,368] INFO SearchService - Created search index with 42000 entries.
[2008-10-21 11:40:18,649] INFO SearchService - Created search index with 43000 entries.
[2008-10-21 11:40:18,915] INFO SearchService - Created search index with 44000 entries.
[2008-10-21 11:40:19,181] INFO SearchService - Created search index with 45000 entries.
[2008-10-21 11:40:19,446] INFO SearchService - Created search index with 46000 entries.
[2008-10-21 11:40:19,696] INFO SearchService - Created search index with 47000 entries.
[2008-10-21 11:40:19,962] INFO SearchService - Created search index with 48000 entries.
[2008-10-21 11:40:20,290] INFO SearchService - Created search index with 49000 entries.
[2008-10-21 11:40:20,571] INFO SearchService - Created search index with 50000 entries.
[2008-10-21 11:40:20,837] INFO SearchService - Created search index with 51000 entries.
[2008-10-21 11:40:21,134] INFO SearchService - Created search index with 52000 entries.
[2008-10-21 11:40:21,399] INFO SearchService - Created search index with 53000 entries.
[2008-10-21 11:40:21,665] INFO SearchService - Created search index with 54000 entries.
[2008-10-21 11:40:21,915] INFO SearchService - Created search index with 55000 entries.
[2008-10-21 11:40:22,227] INFO SearchService - Created search index with 56000 entries.
[2008-10-21 11:40:22,509] INFO SearchService - Created search index with 57000 entries.
[2008-10-21 11:40:22,790] INFO SearchService - Created search index with 58000 entries.
[2008-10-21 11:40:23,071] INFO SearchService - Created search index with 59000 entries.
[2008-10-21 11:40:23,337] INFO SearchService - Created search index with 60000 entries.
[2008-10-21 11:40:23,634] INFO SearchService - Created search index with 61000 entries.
[2008-10-21 11:40:23,931] INFO SearchService - Created search index with 62000 entries.
[2008-10-21 11:40:24,212] INFO SearchService - Created search index with 63000 entries.
[2008-10-21 11:40:24,493] INFO SearchService - Created search index with 64000 entries.
[2008-10-21 11:40:24,774] INFO SearchService - Created search index with 65000 entries.
[2008-10-21 11:40:25,087] INFO SearchService - Created search index with 66000 entries.
[2008-10-21 11:40:25,352] INFO SearchService - Created search index with 67000 entries.
[2008-10-21 11:40:25,571] INFO SearchService - Created search index with 68000 entries.
[2008-10-21 11:40:25,915] INFO SearchService - Created search index with 69000 entries.
[2008-10-21 11:40:26,056] INFO SearchService - Created search index with 70000 entries.
[2008-10-21 11:40:26,306] INFO SearchService - Created search index with 71000 entries.
[2008-10-21 11:40:26,556] INFO SearchService - Created search index with 72000 entries.
[2008-10-21 11:40:29,024] INFO SearchService - Created search index with 72002 entries.

[sindre_mehus]

Hi,

If you email me the offending mp3 file I can try to reproduce it.

Thanks,
Sindre

[wintermute]

Hi Sindre,

I like Subsonic very much but I'd also appreciate the use of an SQL database (6250 artists, 5920 albums, 70607 tracks = 466 GiB).

At the moment my collection is organized like this:

/audio/music/{first_letter}/{artist}-{album}-{year}/

e.g. /audio/music/n/northern_lite-temper-2005/

Cause I really like Subsonic, I tried to work around the issue by creating a (kinda) virtual collection maintained by a python script I wrote. It basically creates a directory for each artist and symlinks to the real album directories in them, for example:

/audio/virtual/Northern Lite/Temper (2005)

This works well: my script only takes about half a minute to finish and thus can be run automatically by a cron job.

The albums and songs are listed in Subsonic too, but I'm unable to play a song or add it to the playlist. Download works though. Any idea what might be the problem with symlinks?

(I could do hardlinks but I'd like to avoid that.)



Thanks,
Wintermute (running Arch Linux, Subsonic 3.6 Beta 2 Standalone)

[wintermute]

Found the problem. Sorry, my bad.

[donpearson]

Found the problem. Sorry, my bad.

What was the problem then?

[wintermute]

It did work after I restarted jetty. The times before I just readded the directory which didn't change anything.

I'm going to add a little parsing to my script so one can specify his own directory structure easily (e.g. "/music/%ARTIST-%YEAR-%ALBUM") and let the script create a custom virtual structure (e.g. "/virtual/%ARTIST/%YEAR-%ALBUM").

I'll post it here, maybe it is useful for others here too.

[koboabe]

Why is it, then, that albums are identified by directory structure?

o I think people, on average, have better file structure than tag quality.

Damn youre right!
You're right, this argument counts a lot, as at least 10% of the files have no id3-tag at all...anyway - would be great to be able to browse through 'Genres' or st. like that

you should check out Music Brainz Picard http://musicbrainz.org/doc/PicardTagger

it has done wonders to my collection...
Media Monkey is ok, but Picard is awesome

[jeremyh]

The initial 10-30 seconds doesn't seem out of the ordinary. I think you will be hard pressed to get it much faster than that with the number of files they are talking about (and since they are all links). But maybe you are just proposing a different layout to prevent loading of it all initially.

We have 105171 songs totaling around 571GB, and we get similar load times on the A-Z list pane on first load (10-30seconds). But it caches fine for us once loaded and the index speed is very fast at about 100 seconds (about 1000/second).

The box we are using is a dual Pentium 4 2.80GHz machine with 2.5GB of ram.

[zeekay]

I removed the artist frame entirely. I then edited main.jsp so that it shows a list of my root folders on initial load (which are organized by genre anyways). Subsonic loads initially in .391 seconds according to fasterfox (after being cached). I prefer this browsing/layout anyways (I have far too many artists to try and browse by artist name), and it does have the added benefit of loading subsonic faster.

[pampelmusenmann]

Why is it, then, that albums are identified by directory structure?

o I think people, on average, have better file structure than tag quality.

Damn youre right!
You're right, this argument counts a lot, as at least 10% of the files have no id3-tag at all...anyway - would be great to be able to browse through 'Genres' or st. like that

you should check out Music Brainz Picard http://musicbrainz.org/doc/PicardTagger

it has done wonders to my collection...
Media Monkey is ok, but Picard is awesome

cool, thx
read it only now...