Automatic library updates fail to update indexes on OSX

[mshick1]

I just ran into this issue. I had updates happening right away before (I think?), so I know it can work, and how it works. I do not have "Fast Access Mode" checked. Yet since a fresh OS X 10.10.3 install, and moving my library to an internal drive on my Mac mini, I haven't been able to get music to show up until a full scan once a day. Nothing shows up in the log either.

Any thoughts on inner workings that might affect this? FSevents or something? Would love some help, full scans are a drag and I add music almost daily.

<-------------------------Subsonic Help Request---------------------->
Problem Description: Automatic library updates do not register in search indexes until a full scan has been performed.
Troubleshooting Steps: Add a folder of music files to an existing artist folder (files are nested 2 deep) to your root, indexed, music folder. The files will never appear in any search or general viewing, but WILL appear navigating via folders in the sidebar.
Playback Application and version: Subsonic default web admin
Subsonic Version: 5.2.1 (also tried 5.1, 5.0)
Server Version: jetty-6.1.x, java 1.8.0_25, Mac OS X (88.9 MB / 137.0 MB)
Hardware Platform: Max OSX 10.10.3 (tested with Mac mini 2010, Macbook Pro 2014)
Java Memory Limit: Unknown (109MB used reported by menu bar)
Problem Filename: n/a
Output from ffmpeg -i:

Code: Select all

ffmpeg version 2.6.2 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-libfaac --enable-nonfree --enable-vda

Last ten log file lines:

Code: Select all

5/9/15 12:30:04 PM EDT]   INFO   DaoHelper   Checking database schema.
[5/9/15 12:30:06 PM EDT]   INFO   DaoHelper   Done checking database schema.
[5/9/15 12:30:06 PM EDT]   INFO   VersionService   Resolved local Subsonic version to: 5.2.1
[5/9/15 12:30:07 PM EDT]   INFO   MediaScannerService   Automatic media library scanning scheduled to run every 1 day(s), starting at Sun May 10 03:00:00 EDT 2015
[5/9/15 12:30:07 PM EDT]   INFO   UPnPService   Starting UPnP service...
[5/9/15 12:30:07 PM EDT]   INFO   PodcastService   Automatic Podcast update scheduled to run every 24 hour(s), starting at Sat May 09 12:35:07 EDT 2015
[5/9/15 12:30:07 PM EDT]   INFO   UPnPService   Enabling UPnP/DLNA media server
[5/9/15 12:30:07 PM EDT]   INFO   UPnPService   Starting UPnP service - Done!
[5/9/15 12:30:17 PM EDT]   INFO   NetworkService   Successfully forwarding port 4040.
[5/9/15 12:32:31 PM EDT]   WARN   CoverArtController   Failed to process cover art /Volumes/Storage/Music/iTunes/iTunes Media/Music/Foxy/TK Disco TKD-412/cover.jpg: java.lang.NullPointerException
[5/9/15 12:35:07 PM EDT]   INFO   PodcastService   Starting scheduled Podcast refresh.
[5/9/15 12:35:07 PM EDT]   INFO   PodcastService   Completed scheduled Podcast refresh.

<-------------------------Subsonic Help Request---------------------->

[mshick1]

A little more info here. I _can_ get new music to show up instantly if I add the music folder directly to the root music folder as defined in Subsonic. However nested folders are _not_ showing up. This is tested on two separate drives with the same behavior.

Music folder = /Volumes/Storage/Music

For example, this will show up, adding a folder of songs, such that the path is:

/Volumes/Storage/Music/New Music/New Song.m4a

This will not:

/Volumes/Storage/Music/New Artist/New Music/New Song.m4a

It seems like fsevents are not bubbling up somehow?

I suspected the recent JRE update as the cause, so I downgraded to 5.1, but have the same problem there.

[mshick1]

As I've debugged further I've seen that updates register instantly in the folder nav / sidebar ONLY. They apparently only hit the database (or maybe lucene?) after the full media scan. They will register everywhere, instantly, when placed directly in the music folder, ie, music files with one-level of nesting (as described above).

I've now tested on two separate machines (2010 Mac mini, 2014 Macbook Pro), with three different music folders. I've tested with a very full music folder (35000 tracks, 6000 albums) and a completely fresh folder. All cases give identical results.

This must be a bug. My experience with solr has taught me the trickiness of commits, maybe it's something in there?

Is there a proper way to report a bug other than these forums?

[mshick1]

Even more debugging: There doesn't seem to be any way to refresh the search indexes, even when albums / songs show up in other places, other than with a full scan. Even when I am able to get music to show up in the folder navigation, even when it is appearing in Recently Added in the web admin, it still doesn't show up in search results, nor in REST endpoint requests made by DSub. Only after a full scan does anything get refreshed.

Maybe this is a caching issue, or possibly a lucene commit issue and the REST endpoint hits the search index before returning results? It's hard to understand the DSub / web admin discrepancy otherwise. It seems like indexes are really the core problem here.

At a minimum some finer grained full scan settings would be great. Having to wait a day (without manual intervention) for music to show up is a drag. The scans only take about 1 minute on my box, so I'd just have them run hourly as a workaround...