[lipid]

So I've got myself a gig lined up, playing uhh..

videogame-influenced-electronica+ambience (whatever that means...)

And I'd like to have some sort of visual presentation.

Question 1:
So I can just have a totally separate machine generating/displaying visuals with no influence from the music...

But how could I get MZX to interact with the music?

- use the file input functionality to read from the sound device (in linux) or something similar?

- manually punch in a beat using the computer keyboard (DJ mixer beatcounter style) to sync up MZX to the music and preprogramme some visual sequences?

- hack in MIDI functionality to MZX? (http://www.sreal.com...ties-for-unix/)

Question 2:
Do I get you digimzx goons to make the presentation for me (NO TIME FOR LIPID) and donate proceeds (in addition to that $30 I owe Exo already) to the MZX dev team?

[Koji]

MZX dev team (read: Exo)

[Exophase]

Okay, so, what is it that you need exactly? Sample output data from MZX? Or would simply knowing the sample position mean anything to you, because 2.81 will offer that (but it's on a once per cycle basis, so you're not going to get super granularity, besides that that's limited by the audio update timing as well). I don't add to MZX proper for donations, but I MIGHT be able to make a custom build, for your usage only (where you wouldn't distribute it, I don't know what the GPL allows though) to do something out of the ordinary. However it seems like it'd be a lot more sensible to do this in something else where you'd have direct access to the output of the audio engine. Could you describe what kind of thing you're after here?

- Exo

[lipid]

First of all Exo, I agree it would be a lot more sensible to do this in something with direct access to audio output!

The sample position may be helpful, but understand that I'm not intending to generate any sound from MZX.

Let's not go crazy just yet. I'm looking for some discussion!

---

Visualise the "sound data" in three forms.

1. Pure PCM

2. Amplitude Frequency Spectrum ( Fast Fourier Transforms in MZX? )

3. Note data.

Usefulness:

The Fourier transform could trigger some visualisation sequence based on rate of change of bass volume, absolute volume of middle section, etc.

The pure PCM data (oscilloscope) coupled with Fourier transform seems to be what drives most winamp vis. plugins, WMP visualisations, etc.

Note data (from a MIDI port, or via whatever method) could be used as triggers for some sort of sequence. Highly customisable, as it ties directly to what's being played live.

---

GPL link:
http://www.gnu.org/c...t/gpl.html#SEC3

It seems that the definition of _distribution_ is the key here.

Can MZX read from a joystick? (game port device?)

[Spectere]

I think what he wants is to make MegaZeux react to audio that Winamp is playing (i.e. making MegaZeux a Winamp plugin, moreorless).

[djtiesto]

Sounds rather interesting, especially if this could be used for gameplay in a way much like Tetsuya Mizuguchi does in games like "Rez" and "Lumines"... I'm a big fan of trippy visuals and electronic music so naturally I'd be into it.

[Nedemai]

lipid, on Aug 17 2005, 09:21 PM, said:

Can MZX read from a joystick? (game port device?)

Yes, it can.

http://www.digitalmz...es/joysetup.zip

This is in the utilities section as joystick mapper.

[lipid]

Nedemai, on Aug 18 2005, 03:17 AM, said:

lipid, on Aug 17 2005, 09:21 PM, said:

Can MZX read from a joystick? (game port device?)

Yes, it can.

http://www.digitalmz...es/joysetup.zip

This is in the utilities section as joystick mapper.

So the next question is,

where does it read and parse the input from the joystick?

at the SDL layer? (Simple DirectMedia Layer Layer?)
directly from the game port?
Windows something something?

A typical MIDI cable plugs into the game port... (to transmit note data)

Perhaps the interface to the hardware is already there... it just needs to be parsed and passed to MZX?

djtiesto, on Aug 18 2005, 03:03 AM, said:

Sounds rather interesting, especially if this could be used for gameplay in a way much like Tetsuya Mizuguchi does in games like "Rez" and "Lumines"... I'm a big fan of trippy visuals and electronic music so naturally I'd be into it.

Is there another one (PS1 game?) where the music is the white line and you're a little guy trying to stay on the music, or something like that?

[Exophase]

It uses SDL which uses DirectInput on Windows. Even though MIDI devices use the same port as old joysticks I really don't think that you can have it generate joystick events. If you could you'd have to map all of them to different keyboard keys (because that's how the joystick support works in MZX, you can't get games to read directly from them)

- Exo

[lipid]

This seems to be an interesting semi-related tool... built on SDL?
http://www.gephex.org/

I did some extra research, really only yielding:
http://www.libsdl.or...une/046711.html

Indicating maybe there IS some support, just not officially?

[lipid]

Another one, looks relevant, and is LGPLed. I guess I'll have a closer look when I get home from work!

http://audiality.org/download.html

[ajs]

The easiest way on Linux (and maybe Windows too) would be to record from PCM in if it's wav data, or read /dev/sequencer for MIDI. You'd need to hack MZX a little bit, but if all you wanted was some sort of averaged samples or an FFT with a 512byte window (this is nice and easy and what most visualisations do) you can write one in a few lines of code. You could then provide the peak as a counter in MZX (I'm sure Exo could show you how to do this).

An alternative, slow, moderately evil way would be to output the results of your FFT from a standalone C program (I could write one pretty quickly to do ALSA->audio in->FFT->write out) and just poll a file on disc. This wouldn't require modifications to MZX.

On Windows, obviously the same things are possible. SDL itself doesn't support MIDI. SDL_mixer can play MIDI using timidity, but that just renders MIDIs, it can't read in MIDI data. If whatever you use involves MIDI, it's unlikely you could write it in a clean, moderately acceptable way (if the changes were ever to be integrated into MZX), but PCM/FFT stuff would be easy to do and cheap to add (but will require knowledge of MZX's source code regarding updating counters and making them available to robotic.. but this can't be too hard).

Cheers,
ajs.

[djtiesto]

lipid, on Aug 18 2005, 03:51 AM, said:

Is there another one (PS1 game?) where the music is the white line and you're a little guy trying to stay on the music, or something like that?

Yeah, it's called Vib Ribbon and never came out in the States... you put in a CD and it generates levels (with monochrome, vector graphics) based on the rhythm of the song... sounds unique and it used to get a lot of hype back in the day, but I've never played it myself.

[Koji]

An external midi playing aparatus, plugs into the "game port" and they send data down the "game port" That mzx would then interpret as joypad/stick button presses...

So if you have an external midi player(the majority of midi instruments have a playback feature that feeds notes to the computer to be recorded into the file format), it'd be perfect for this project... cause all mzx would then be reacting to was joypad/stick input.

However, this is a quick fix... I presonally would love to make games react to music, I'd love mzx to have more music interactivity than just the current mod order playing...

Things I'd really like to see

mod order position.
mod position's note given the track.
General volume
Exact volume

And why not also allow robots to have an affect on the music, the speed, tempo, maybe a note shift or something...

And anything else that might be possible...

Does modplug have structures to read this data from? Can we get these functionalities in mzx?

I'm sure it'd open a few doors to game genera's not only limited to ddr clones.

This post has been edited by Koji: 18 August 2005 - 11:47 PM

[apage43]

it doesn't work that way. Sorry, even though MIDI plugs into the game port, you can't make it look like a joystick.

[lipid]

Koji, on Aug 18 2005, 11:40 PM, said:

An external midi playing aparatus, plugs into the "game port" and they send data down the "game port" That mzx would then interpret as joypad/stick button presses...

Yeah, it would be cool. But I think we established

(Exo, correct me if I'm wrong...)

that MZX implements a joystick through the SDL and DirectMedia layers.

It sounds like these two layers have transformed any "raw data" entering the game port into joystick events, ignoring any "non-standard joystick data" (i.e. MIDI data) through the game port.

[Koji]

Oh blast..

Well then add thoes functionalities!!!

[Exophase]

lipid, on Aug 18 2005, 08:28 PM, said:

Koji, on Aug 18 2005, 11:40 PM, said:

An external midi playing aparatus, plugs into the "game port" and they send data down the "game port" That mzx would then interpret as joypad/stick button presses...

Yeah, it would be cool. But I think we established

(Exo, correct me if I'm wrong...)

that MZX implements a joystick through the SDL and DirectMedia layers.

It sounds like these two layers have transformed any "raw data" entering the game port into joystick events, ignoring any "non-standard joystick data" (i.e. MIDI data) through the game port.

Most joysticks don't even use gameport, and modern OS's don't really have direct gameport support built in. A device driver can of course facilitate this. In theory a driver could make MIDI look like a gamepad to Windows (or Linux) but that's just too much work...

- Exo

[lipid]

PROTOTYPE IS WORKING!


MWUAHAHA

It's a dirty Windows hack at the moment...

I'll tidy it up and look a bit at how to make linux work (through ALSA).

Essentially so far,
MZX app updates MIDITYPE, MIDICH, MIDIVAL1, MIDIVAL2 counters according to whatever's being read off the midi device.

It kinda sucks (interface feature-wise) at the moment.
I'll expand the functionality a bit to map the counters:

"MIDITYPE<device><channel>", "MIDIVAL1<device><channel>","MIDIVAL2<device><channel>"

This seems to be the most flexible way to expose the information to the MZX environment.
---

Alternatively, a robot could request the MZX app to write to it's local counters for a specific device and channel...

[Sai'ke]

hey, now you could make a "learn how to play the keyboard"-game ;D

[weasel]

It could pave the way to a DDR, Beatmania, or even Lumines clone.

[Spectere]

Actually, a DDR/Beatmania clone would be possible just by checking the mod playback position (and, IIRC, that's going to be a feature in the next version). You could kind of do it now by just syncing it to a certain time, but with the ability to check the mod position you'd be able to keep it properly synched on all speed settings (not to mention that you wouldn't have to do anything special for BPM changes or stops). StepMania uses a similar method to keep in sync, actually.

Now if you're able to get a glimpse of exactly where you are when playing OGGs, it's possible that you could rig something up that actually took advantage of DWI/SM files. The BPM calculations might be a bit of a pain, though.