ChristopheCVB ChristopheCVB - 1 month ago 8
Android Question

Using FMod in Android for pitch/tempo/mix/trim

I want to be able to do some Pitch and Tempo Shifting, mix and trim sounds and apply effects.
I tried ffmpeg but unfortunately, it has some huge delay on processing the audio file (like 40secs for just pitch+tempo on a 36secs file).

So I searched the web for a library that could do all these features and I found FMod could be the answer.

I have never played with NDK though, and I'm bad at reading or even writing C code.

Could you help me on how to start this adventure ?

Answer

I finally decided to make an SDK of my own using FMod to apply all the effects I want.

Here are the signatures of the Java class calling the NDK :

public static native String mix(String[] inputFiles, float secondaryVolume, String outFile);

public static native String trim(String inFile, String outFile, long startMs, long endMs);

public static native String fadeOut(String inFile, String outFile, long startMs, long endMs);

public static native String processDSPs(String inFile, String outFile, FMODDSP[] dsps);

where the abstract FMODDSP looks like :

public abstract class FMODDSP
{
    public static final int FMOD_DSP_TYPE_COMPRESSION = 1;
    public static final int FMOD_DSP_TYPE_ECHO = 2;
    public static final int FMOD_DSP_TYPE_FLANGE = 3;
    public static final int FMOD_DSP_TYPE_LOWPASS = 4;
    public static final int FMOD_DSP_TYPE_HIGHPASS = 5;
    public static final int FMOD_DSP_TYPE_PITCH = 6;
    public static final int FMOD_DSP_TYPE_REVERBERATION = 7;
    public static final int FMOD_DSP_TYPE_DISTORTION = 8;
    public static final int FMOD_DSP_TYPE_TEMPO = 9;
    public static final int FMOD_DSP_TYPE_CHORUS = 10;

    protected int type;

    public FMODDSP(int type)
    {
        this.type = type;
    }

    public int getType()
    {
        return this.type;
    }
}

and an example implementation of a FMODDSP of the pitch is :

public class FMODDSPPitch extends FMODDSP
{
    /**
     * Pitch value.  0.5 to 2.0.  Default = 1.0. 0.5 = one octave down, 2.0 = one octave up.  1.0 does not change the pitch.
     */
    public float pitch = 1f;
    /**
     * FFT window size.  256, 512, 1024, 2048, 4096.  Default = 1024.  Increase this to reduce 'smearing'.  This effect is a warbling sound similar to when an mp3 is encoded at very low bitrates.
     */
    public float fftSize = 1024f;

    public FMODDSPPitch()
    {
        super(FMODDSP.FMOD_DSP_TYPE_PITCH);
    }

    public FMODDSPPitch(float pitch, float fftSize)
    {
        super(FMODDSP.FMOD_DSP_TYPE_PITCH);

        this.pitch = pitch;
        this.fftSize = fftSize;
    }

    public float getPitch()
    {
        return this.pitch;
    }

    public float getFFTSize()
    {
        return this.fftSize;
    }
}

I have not planned to make the whole thing open source but if you guys are interested, feel free to ask me, I'll do my best ;)