SeismicSquall SeismicSquall - 5 months ago 45
iOS Question

Using MPSImageConvolution kernel with Metal compute shaders

I am using the MetalVideoCapture example located here The only thing I altered in my version was using MPSImageConvolution (instead of MPSImageGaussianBlur) with kernel values:

[-2.0, -1.0, 0.0,
-1.0, 1.0, 1.0,
0.0, 1.0, 2.0]

Using the values above failed to alter the output in any visible way. But an edge enhance kernel e.g

[0.0, -1.0, 0.0,
0.0, 1.0, 0.0,
0.0, 0.0, 0.0]

works, mind you in column-major order only; it does not work in row-major order even though thats what MPSImageConvolution expects. I'm really stumped by this. I do not know if there is an obvious reason that a convolution kernel cannot work in a compute pipeline (only in a render pipeline) but i couldn't find any info on this online.

I also modified the codebase to apply the kernel to a static image instead of a live video feed; This yielded the same results, however.

I also wanted to point out that I posted the same question on the example project's message board ( The author of the example was equally as stumped as I was, which lead me to believe it was some sort of bug or a gap in my conceptual knowledge of why this is not even suppose to work.


I do have a workaround and that's to avoid using an in-place texture. Try this: create a separate destination texture:

    let descriptor = MTLTextureDescriptor.texture2DDescriptorWithPixelFormat(
        width: drawable.texture.width,
        height: drawable.texture.height,
        mipmapped: false)

    let destination: MTLTexture = device!.newTextureWithDescriptor(descriptor)

Have the YCbCrColorConversion shader target the destination:

commandEncoder.setTexture(destination, atIndex: 2) // out texture

...and then use the alternative encodeToCommandBuffer that uses a destination:

encodeToCommandBuffer(commandBuffer, sourceTexture: destination, destinationTexture: drawable.texture)

This stuff can be removed:

//        let inPlaceTexture = UnsafeMutablePointer<MTLTexture?>.alloc(1)
//        inPlaceTexture.initialize(drawable.texture)


With thanks to Warren!