MOD-File Transcoding/Streaming with XMP - [possible BUG]

[NDscham]

tracker-music.zip

Hi Guys,
I was trying to add mod-tracker playback (IT/MOD/S3M and others) via XMP (http://xmp.sourceforge.net/) and lame, but even though it works fine on CLI, the builtin-flashplayer doesn't play anything.

This is what I've done so far:

added the following transcoding line for s3m>mp3:

Code: Select all

Step 1: xmp -c %s // this outputs the tracker-file to stdout as a raw pcm stream
Step 2: lame -r - - // this takes the raw stream from stdin (-r for raw input data) and outputs to stdout


While this works fine on command line with

Code: Select all

xmp -c "modfile.s3m" | lame -r - "modfile.mp3"

, it does not give me any playback in SubSonic.
Note, I *DID* enable the new transcoding option for the player in the subsonic settings. And *YES*, the xmp binary is in subsonic's transcoding directory.
I am running subsonic 4.5 on a gentoo linux x86-amd64 machine.

BTW, the tracker file output takes a few seconds, not as fast as ffmpeg for other audio file types, could this be the reason?

If someone wants to try it, I've attached a zip archive with some tracker-files (it, s3m) to test with:

tracker-music.zip

Who can help here?


best regards,
ndscham

[NDscham]

*Bump*

[marsmayhem]

Have you seen this tutorial? He's on Linux too..

http://forum.subsonic.org/forum/viewtopic.php?f=6&t=5973

[marsmayhem]

Offhand, I think you might want to add a few more parameters to Step 2's lame, eg. (borrowed from aforementioned tut):

Step 1: xmp -c %s
Step 2: lame -r -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - -

I'll play around with this. I also have an interest in getting MOD playback sorted out. I was looking at a java player, but it was giving me problems.. XMP looks like it might be more promising, and there's a Windows port, which mikmod didn't really have (that worked for me).

EDIT:

Okay, just tried this on my Windows box.. last Win32 build I could find was xmp 3.3.0 (latest is 3.4.1).. I used the steps listed above to render some mod and s3m's.

Good news is I'm getting audio playback and no error messages in the log..
Bad news is there's a ton of static on top of the audio, too much to be able to listen to, but I can make out some of the song underneath it..

Let me know if you have the same experience on linux with the latest 3.4.1 build..

EDIT 2:
I tried your step 2 and I get the same static playback.. so you must have some other issue going on if you're not getting any audio output. Check the About section's log and see if it's complaining about anything else there..

[marsmayhem]

I see VLC will also do MOD/S3M/etc playback.. so I think I might try that.. still figuring out the commandline.. any VLC gurus, feel free to jump in at this point.

[NDscham]

Offhand, I think you might want to add a few more parameters to Step 2's lame, eg. (borrowed from aforementioned tut):

Step 1: xmp -c %s
Step 2: lame -r -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - -

Actually, I tried that and decided to simplify parameters. On the command line, I was successful piping those commands. The resulting mp3 was perfect, no stuttering, no static. You could try lame's -x parameter to swap input bytes, if you get static noise only. How is the wav quality if you use xmp to convert to wav only?

Let me know if you have the same experience on linux with the latest 3.4.1 build..

Well, I was actually using a self-compiled XMP 3.4.1 for my machine. It works just fine except for taking quite a few seconds to finish MOD to WAV conversion on a Core2Duo 1,86GHz. That's what made me think this could be a timeout issue.

I tried your step 2 and I get the same static playback.. so you must have some other issue going on if you're not getting any audio output.

Err, as mentioned, it works fine on command line, the flash player just doesn't start playback at all.

Check the About section's log and see if it's complaining about anything else there..

Didn't give me any other clues than on command line, so I turned it to silent mode by adding XMP's -q parameter.

I see VLC will also do MOD/S3M/etc playback.. so I think I might try that.. still figuring out the commandline.. any VLC gurus, feel free to jump in at this point.

Hmm, I am using a headless machine with no X-server, didn't think of trying vlc. I'll emerge that and try around.

Thank you for your suggestions. Let's get this working together!
Best regards,
NDscham

[NDscham]

No luck in getting vlc compiled because it depends on X11-libraries, which i don't have and don't need since this server has no GUI. So vlc just died here. I'll focus on the xmp solution again.

[NDscham]

Ok, another progress here. Obviously, not even the preset .wav > mp3 transcoding profile in Subsonic works for me, though it works if entered on the command line.
I'll upgrade to 4.5 final (was using 4.5 beta1) and see if that fixes it for me. Video transcoding works fine, though.

//EDIT: Upgraded to 4.5final with manual upgrade to JWPlayer 5.7 and wav to mp3 transcoding working properly. No luck on the xmp - lame combination, though. Also, mikmod is not working for me using yoshi's setup. I will keep experimenting, but I wish Sindre could give this a look. I am still wondering about this timeout thing...

[marsmayhem]

I'll try the lame -x parameter..

vlc can run commandline only.. not sure about linux commandline, but win32 uses "-I dummy --quiet-dummy".. you might see debug references to "use cvlc" for commandline only mode..

EDIT:

This is what I tried in VLC.. it's supposed to write to stdout, but I didn't get any audio playback from this.. at one point I got a command window open and all the data was piping into it and getting dumped like a text dump or something.. I assume that was the wave data..

vlc -I dummy --dummy-quiet --aout=file --audiofile-format=s16l --audiofile-channels=2 --audiofile-file=- --audiofile-wav %s vlc://quit

you could also try simplifying the params to:

vlc -I dummy --dummy-quiet --aout=file --audiofile-file=- %s vlc://quit

I couldn't get it to work properly though. I had more luck with xmp than VLC..

VLC's mod support looks quite extensive - from latest source on vlc wiki:

static const char *ppsz_mod_ext[] =
{
"mod", "s3m", "xm", "it", "669", "amf", "ams", "dbm", "dmf", "dsm",
"far", "mdl", "med", "mtm", "okt", "ptm", "stm", "ult", "umx", "mt2",
"psm", "abc", NULL
};

I think xmp's support is larger than the above.. but if it doesn't work for you, it's all for naught I suppose :/

[marsmayhem]

Trying XMP again.. this is what the debug output looks like:

[9/12/11 3:00:09 PM PDT] INFO PlaylistInputStream admin listening to "#MUSIC\ARYX.S3M"
[9/12/11 3:00:09 PM PDT] DEBUG TranscodeInputStream Starting transcoder: [c:\subsonic\transcode\xmp.exe] [-c] [C:\MEDIA\BT\#COMPLETED\#MUSIC\ARYX.S3M]
[9/12/11 3:00:09 PM PDT] DEBUG TranscodeInputStream Starting transcoder: [c:\subsonic\transcode\lame] [-x] [-r] [-] [-]
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Extended Module Player 3.3.0
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Copyright (C) 1996-2010 Claudio Matsuoka and Hipolito Carraro Jr
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Using Output to stdout
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Mixer set to 16bit, 44100 Hz, interpolated stereo
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Loading C:\MEDIA\BT\#COMPLETED\#MUSIC\ARYX.S3M... (1 of 1)
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Module title : aryx
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Module type : SCRM (Scream Tracker 3.20)
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Module length : 42 patterns
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Stored patterns: 12 ............
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Instruments : 9 ........
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Channels : 12 [ 4 c 4 c 4 c 4 c 4 c 4 c ]
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Estimated time : 2min21s
[9/12/11 3:00:11 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe)

and there's a lot of output with tempo and channel in it..

and when I stop playback, it shows this..

[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Elapsed time : 0min37s
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\xmp.exe) Channels used : 12/12, avg 10.86 (90.5%)
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) Assuming raw pcm input file : Forcing byte-swapping
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) LAME 3.98.4 32bits (http://www.mp3dev.org/)
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) CPU features: MMX (ASM used), SSE (ASM used), SSE2
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) Using polyphase lowpass filter, transition band: 16538 Hz - 17071 Hz
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) Encoding to
[9/12/11 3:00:47 PM PDT] DEBUG InputStreamReaderThread (c:\subsonic\transcode\lame) Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3

Is it the "byte-swapping" bit that's causing the static perhaps? I can hear bits of the tune, but there's a lot of static on top of it.. I can't find a win32 build more recent than 3.3.0

Thanks for your help..

EDIT:
I got rid of the "-r" parameter and it made no difference.. same static junk on top of song playback.. with/without -x, no difference..

[NDscham]

Thanks for your efforts, marsmayhem. No luck with VLC so far. Interestingly enough, the transcoding jobs obviously never finish,
so I'm left with a lot of instances of lame and xmp in the running tasks.
Here is the output of ps -A aw | grep subsonic after clicking an S3M-file in a temp folder:

Code: Select all

19593 pts/0    S      0:00 /opt/subsonic/transcode/xmp -c /storage/pub/incoming/s3m.s3m
19596 pts/0    S      0:00 /opt/subsonic/transcode/lame -r -b 128 --tt s3m --ta --tl  -S --resample 44.1 - -
19861 pts/1    S+     0:00 grep --colour=auto subsonic
20528 pts/0    Sl     5:54 /etc/java-config-2/current-system-vm/bin/java -Xmx250m -Dsubsonic.home=/opt/subsonic -Dsubsonic.host=0.0.0.0 Dsubsonic.port=6005 -Dsubsonic.httpsPort=6006 -Dsubsonic.contextPath=/ -Dsubsonic.defaultMusicFolder=/storage/pub/Music -Dsubsonic.defaultPodcastFolder= -Dsubsonic.defaultPlaylistFolder=/storage/pub/Music/zzzPlaylists -Djava.awt.headless=true -verbose:gc -jar subsonic-booter-jar-with-dependencies.jar

I wonder what causes this behaviour. subsonic.log gives me:

Code: Select all

[9/14/11 8:58:41 PM CEST]    DEBUG   TranscodeInputStream   Starting transcoder: [/opt/subsonic/transcode/xmp] [-c] [/storage/pub/incoming/s3m.s3m]
[9/14/11 8:58:41 PM CEST]    DEBUG   TranscodeInputStream   Starting transcoder: [/opt/subsonic/transcode/lame] [-r] [-b] [128] [--tt] [s3m] [--ta] [] [--tl] [] [-S] [--resample] [44.1] [-] [-]
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Extended Module Player 3.4.1
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Copyright (C) 1996-2011 Claudio Matsuoka and Hipolito Carraro Jr
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Using Output to stdout
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Mixer set to 16bit, 44100 Hz, interpolated stereo
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Loading /storage/pub/incoming/s3m.s3m... (1 of 1)
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Module title : Carrotus
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Module type : SCRM (Scream Tracker 3.20)
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Module length : 32 patterns
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Stored patterns: 33 .................................
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Instruments : 28 ...........................
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Channels : 12 [ 9 7 9 7 9 7 9 7 9 7 9 7 ]
[9/14/11 8:58:41 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) Estimated time : 1min52s

Note there is no lame output, even though i removed the -S silent switch. Killing xmp manually on the command line leaves me with

Code: Select all

[9/14/11 9:17:40 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp)
[9/14/11 9:17:40 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) *** Interrupted: signal 15 caught
[9/14/11 9:17:40 PM CEST]    DEBUG   InputStreamReaderThread   (/opt/subsonic/transcode/xmp) can't reset terminal!

which makes me think xmp hangs somehow. I will try your older version and report back, even though I don't get why it's working fine on command line.
You seem to get a step further, how is your quality when outputting via xmp to wav (xmp [infile.tracker] -o [outfile.wav])?

[NDscham]

I may have a solution for you:
Try using -d wav for xmp and omitting -r for lame:

Code: Select all

xmp -d wav %s -c | lame -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - -

This will make xmp output wave data avoiding lame having to handle raw input (and should improve quality).

I'm pretty convinced there has to be a bug with the file handling on linux. No xmp process started by subsonic ever finishes.
I tried making it write into a fixed file by entering xmp -d wav %s -o /path/to/file.wav into the transcoding settings.
The resulting wav only contained a header and had a size of 44 bytes. -> BUG?

[marsmayhem]

Hi
WAV output to physical file = working (eg. xmp -d wav -o output.wav input.s3m)
XMP playback from command line (eg xmp test.s3m) = working

xmp -d wav %s -c | lame -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - - = not working

no luck with that command line suggestion, I don't get any audio output at all (I think it's lame's fault)

I also tried

Code: Select all

xmp -d wav -o - %s | lame -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - -
xmp -c - %s | lame -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - -


Here's subsonic log from first commandline, it looks very similar to yours:

Code: Select all

[9/14/11 2:13:15 PM PDT]   INFO   PlaylistInputStream   admin listening to "testmod\2ND_PM.S3M"
[9/14/11 2:13:15 PM PDT]   DEBUG   TranscodeInputStream   Starting transcoder: [c:\subsonic\transcode\xmp.exe] [-d] [wav] [-c] [C:\MEDIA\BT\#COMPLETED\#MUSIC\testmod\2ND_PM.S3M]
[9/14/11 2:13:15 PM PDT]   DEBUG   TranscodeInputStream   Starting transcoder: [c:\subsonic\transcode\lame] [-b] [128] [--tt] [2ND_PM] [--ta] [] [--tl] [testmod] [-S] [--resample] [44.1] [-] [-]


But there's no audio output with this..

I also noticed VLC doesn't terminate its process when you're trying to get it to work. It just stays out there. Very annoying.

I have a 44 byte WAV file from previous commandline fu I was doing.. I don't remember what caused it.. it might be caused by XMP writing out the output file without validating all the input parameters first (bad coding?)

Maybe the stdout processing on xmp is hosed somehow.. wav file and win32 drivers are working fine (for me)..


I get audio output with this, but again, with static on top of it - I can make out the song, but too much static on top:

Code: Select all

xmp -d wav -c %s | lame -b %b - -


Here's the log output:

Code: Select all

[9/14/11 2:16:39 PM PDT]   INFO   PlaylistInputStream   admin listening to "testmod\2ND_PM.S3M"
[9/14/11 2:16:39 PM PDT]   DEBUG   TranscodeInputStream   Starting transcoder: [c:\subsonic\transcode\xmp.exe] [-d] [wav] [-c] [C:\MEDIA\BT\#COMPLETED\#MUSIC\testmod\2ND_PM.S3M]
[9/14/11 2:16:39 PM PDT]   DEBUG   TranscodeInputStream   Starting transcoder: [c:\subsonic\transcode\lame] [-b] [128] [-] [-]
[9/14/11 2:16:39 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Extended Module Player 3.3.0
[9/14/11 2:16:39 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Copyright (C) 1996-2010 Claudio Matsuoka and Hipolito Carraro Jr
[9/14/11 2:16:39 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Using WAV writer: stdout
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Mixer set to 16bit, 44100 Hz, interpolated stereo
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Loading C:\MEDIA\BT\#COMPLETED\#MUSIC\testmod\2ND_PM.S3M... (1 of 1)
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Module title : UnreaL ][ / PM
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Module type : SCRM (Scream Tracker 3.01)
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Module length : 84 patterns
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Stored patterns: 77 .............................................................................
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Instruments : 54 .........................................
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Channels : 8 [ 0 f 0 f 0 f 0 f ]
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Estimated time : 6min11s
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe)
[9/14/11 2:16:40 PM PDT]   DEBUG   InputStreamReaderThread   (c:\subsonic\transcode\xmp.exe) Tempo[03] BPM[82] Pos[00/53] Pat[49/4C] Row[ / ] Chn[ / ]
...
etc
...

[NDscham]

Hi
WAV output to physical file = working (eg. xmp -d wav -o output.wav input.s3m)
XMP playback from command line (eg xmp test.s3m) = working
xmp -d wav %s -c | lame -b %b --tt %t --ta %a --tl %l -S --resample 44.1 - - = not working
But there's no audio output with this..

I also noticed VLC doesn't terminate its process when you're trying to get it to work. It just stays out there. Very annoying.

Same here with failed transcoding jobs by subsonic.

I have a 44 byte WAV file from previous commandline fu I was doing.. I don't remember what caused it.. it might be caused by XMP writing out the output file without validating all the input parameters first (bad coding?)

I got that when making subsonic start xmp with a file target instead of stdout.

Maybe the stdout processing on xmp is hosed somehow.. wav file and win32 drivers are working fine (for me)..

Works fine, as tested on commandline.

I get audio output with this, but again, with static on top of it - I can make out the song, but too much static on top:

What about this on command line?

Code: Select all

xmp -d wav -c [path\to\input.s3m] | lame -b %b - [path\to\output.mp3]


This doesn't work for me at all. I'm pretty sure there has to be a bug with subsonic's way of handling the transcoders.
Maybe you could also try adding -q for xmp to prevent logfile spamming.

Anyway, I sent Sindre Mehus, the developer of subsonic, a PM. Maybe he finds some time to look into it, I believe there could be a bug with subsonics transcoder handling.

[marsmayhem]

I tried the -q and it didn't make a difference to the output.

Let me know if you hear anything back from Sindre.

I wonder what it takes to build a new XMP from source on windows..