Custom ICE Node: Wave To Spectrum

It’s been something that I wanted to achieve for a quite a while, and finally got the right mix and enough to get it to work. Picking a WAVe file, get the spectrum decomposition of it (using a Fast Fourier Transform library named KISS FFT) and finally wrap the whole thing into a Custom ICE Node so that I could use it to drive particles.

Here is a few trial, one using a typical spectrum bars, and the other one using spectrum information to drive strands and deformation.

I’ve also made a tutorial on how to use the plug-in:

And finally here is a screenshot of the ICE Tree (there is a smaller ICE Tree before that that sets for each point its frequency, a normalized value between 0 and 1 covering the full frequency range from the audio file):

wavespectrum_commented

Scene is available for download: spectrumBars scene.

I am not an expert in C++ nor in Softimage Development, so there is sure some room for improvement! The code is available just for that 🙂 You can browse and fork it on Github: https://github.com/claudevervoort-perso/xsi-audio-spectrum. The DLL is also available on GitHub.

6 thoughts on “Custom ICE Node: Wave To Spectrum

  1. Michael Karnerfors

    Hello Claude!

    We – my employer – have been asked to help with a fun little art project where they are trying to visualize music by letting it deform a model. Your node here seems to be just the thing that can be used for it and we have managed to get it to run here.

    We want to try to add another dimension to it though by making a “waterfall” (scrolling spectrum) effect and for that we either want time to be a per particle scope variable instead of per object scope, or add another port to the node that lets us set a time offset in the same scope.

    I’m a software developer myself and while I have managed to get the project to open and compile in Visual Studio 2012, I am a wee bit lost on how to make the modification since I’m not all that familiar with Softimage. And my colleagues that use SI are not programmers so the code is mostly greek to them. If you have the time, we would be most happy if you would drop us a line and a hint on how to add another port that lets us “offset” the time based on each particle.

    We will of course – when the project is done – credit you as much as you like for your help. There is not money involved though… this is a little pro bono thing. I hope that is allright with you.

    Oh, and a tip: to make the DLL work, one does not have to have all of Visual Studio installed. What you need is the corresponding Visual Studio C++ redistributable. So if you build it with VS 2010, you need the VS C++ 2010 redistributable. For VS 2012, you need that redistributable and so on. These are small downloads that are available from Microsoft. Just thought I’d let you know. 🙂

    Would love to hear from you. Cheers!

    /Michael

    Reply
  2. claude Post author

    Hi Michael,

    I’ve pushed some comments on the code itself, hopefully that will help a bit. I’d say the plugin could be modified to run the FFT per point rather than by object by moving where the analyzer is instantiated and the getFFT(timeBlock) is called to the per-point evaluation. The entry point in the plugin should also accept any context (rather than the current singleton). However that would mean each particle runs its own FFT, not sure how bad that would be.

    Now does each particle really needs a distinct time offset? If not, and if you can have a pre determined set of delay, then maybe you can create X instances of the Node itself on the tree, each node taking the global time – n*timeOffset. Now a particle can pick, based on some switch pick, its value. If that was possible then you do not need any code change (although the file is going to be loaded once per node, so being a Wave, it can end up being costly if you create many instances).

    Another minor point: to get Wave that actually worked, I’ve used Itunes Wave Encoder.

    Hope this helps a bit,

    Claude

    (thx for the DLL tip!)

    Reply
  3. john

    Hey Claude,
    I love your plugin but just as I started plying with it I needed a reformat and now I have a ‘missing library’ error when I try to load the plugin to softimage. Im not experienced in c++ and I dont want to take too much of your time but if you know what my problem is I would love some help.

    I used Dependency Walker to analyse the wavetospectrum.dll and it says that the following files could not be found: MSVCP100D.DLL MSVCR100D.DLL SICPPSDK.DLL

    I found all 3 of those files in either the softimage SDK/library folder or in my Win/ststem32 folder so I dont know why they cant be found. I tried loading the wintospectrum.cpp file into visual studio but Im pretty overwhelmed by it and couldnt figure out how to reference those files i need to load.

    Thanks for any help you can give me.

    -john

    Reply
  4. Bullit

    Hello Claude.
    One question what i the way to offset the animation?
    I offset the file in mixer 200 frames but the animation doesn’t respect that.

    Reply
  5. Pingback: Custom ICE Node: Wave To Spectrum | claudeonthe...

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.