Excludos Excludos - 8 days ago 4x
C++ Question

Creating transparent overlay for camera in OpenCV c++

I'm trying to create an overlay for a camera feed, and I want the overlay to be blurred, and about 50% transparent. One way of solving this is to copy each frame from the camera, draw onto it, and merge them together using addWeighted. This doesn't work for me because the blur effect takes up so much resources the output fps drops to 10.

Another solution I thought up is to create the overlay once (It's static after all, why recreate it every frame?) and merge it with the camera feed. However the resulting video gets noticeably darker when doing this, seemingly because the overlay mat refuses to be transparent.

(*cap) >> frameOriginal;

orientationBackground = cv::Mat(frameOriginal.rows, frameOriginal.cols,
frameOriginal.type(), cv::Scalar(0,0,0,0));
cv::Mat headingBackground;

cv::Point layerpt1(1800, 675);
cv::Point layerpt2(1850, 395);
cv::rectangle(orientationBackground, layerpt1, layerpt2,
cv::Scalar(255,80,80), CV_FILLED, CV_AA);

cv::blur(orientationBackground, orientationBackground, cv::Size(7,30));

double alpha = 0.5;
addWeighted(orientationBackground, alpha, frameOriginal, 1-alpha, 0, frameOriginal);

The before(left) and after(right) adding the overlay:overlay problem

I'm using OpenCV 3.10 on windows x64 btw


Try this:

    cv::Mat input = cv::imread("C:/StackOverflow/Input/Lenna.png");

    // define your overlay position
    cv::Rect overlay = cv::Rect(400, 100, 50, 300);

    float maxFadeRange = 20;

    // precompute fading mask:
    cv::Size size = input.size();
    cv::Mat maskTmp = cv::Mat(size, CV_8UC1, cv::Scalar(255));

    // draw black area where overlay is placed, because distance transform will assume 0 = distance 0
    cv::rectangle(maskTmp, overlay, 0, -1);

    cv::Mat distances;
    cv::distanceTransform(maskTmp, distances, CV_DIST_L1, CV_DIST_MASK_PRECISE);

    cv::Mat blendingMask = cv::Mat(size, CV_8UC1);

    // create blending mask from 
    for (int j = 0; j < blendingMask.rows; ++j)
    for (int i = 0; i < blendingMask.cols; ++i)
        float dist = distances.at<float>(j, i);

        float maskVal = (maxFadeRange - dist)/maxFadeRange * 255; // this will scale from 0 (maxFadeRange distance) to 255 (0 distance)
        if (maskVal < 0) maskVal = 0;

        blendingMask.at<unsigned char>(j, i) = maskVal;

    cv::Scalar overlayColor = cv::Scalar(255, 0, 0);

    // color a whole image in overlay colors so that rect and blurred area are coverered by that color
    cv::Mat overlayImage = cv::Mat(size, CV_8UC3, overlayColor);

    // this has created all the stuff that is expensive and can be precomputed for a fixes roi overlay

    float transparency = 0.5f; // 50% transparency

    // now for each image: just do this:

    cv::Mat result = input.clone();

    for (int j = 0; j < blendingMask.rows; ++j)
    for (int i = 0; i < blendingMask.cols; ++i)
        const unsigned char & blendingMaskVal = blendingMask.at<unsigned char>(j, i);

        if (blendingMaskVal) // only blend in areas where blending is necessary
            float alpha = transparency * blendingMaskVal / 255.0f;

            result.at<cv::Vec3b>(j, i) = (alpha)*overlayImage.at<cv::Vec3b>(j, i) + (1.0f - alpha)*result.at<cv::Vec3b>(j, i);


Giving this result with 50% transparency and a fading range of 20 pixels:

enter image description here

enter image description here

and this is 20% transparency (variable value = 0.2f) and 100 pixels fading:

enter image description here

enter image description here