pythonimus pythonimus - 2 months ago 104
C# Question

Detect display corners with Emgu

I want to detect a display on an image (more precisely its corners).
I segment the image in display color and not display color:

Image<Gray, byte> segmentedImage = greyImage.InRange(new Gray(180), new Gray(255));


enter image description here

Then I use corner Harris to find the corners:

Emgu.CV.Image<Emgu.CV.Structure.Gray, Byte> harrisImage = new Image<Emgu.CV.Structure.Gray, Byte>(greyImage.Size);
CvInvoke.CornerHarris(segmentedImage, harrisImage, 2);
CvInvoke.Normalize(harrisImage, harrisImage, 0, 255, NormType.MinMax, DepthType.Cv32F);


enter image description here

There are now white pixels in the corners, but I cannot access them:

for (int j = 0; j < harrisImage.Rows; j++)
{
for (int i = 0; i < harrisImage.Cols; i++)
{
Console.WriteLine(harrisImage[j, i].Intensity);
}
}


It writes only 0s. How can I access them? And if I can access them, how can I find the 4 corners of the screen in the harris image? Is there a function to find a perspectively transformed rectangle from points?

EDIT:

On the OpenCV IRC they said FindContours is not that precise. And when I try to run it on the segmentedImage, I get this:
enter image description here
(ran FindContours on the segmentedImage, then ApproxPolyDP and drew the found contour on the original greyscale image)

I cannot get it to find the contours more precise...

EDIT2:
I cannot get this to work for me. Even with your code, I get the exact same result...
Here is my full Emgu code:

Emgu.CV.Image<Emgu.CV.Structure.Gray, Byte> imageFrameGrey = new Image<Emgu.CV.Structure.Gray, Byte>(bitmap);
Image<Gray, byte> segmentedImage = imageFrameGrey.InRange(new Gray(180), new Gray(255));
// get rid of small objects
int morph_size = 2;
Mat element = CvInvoke.GetStructuringElement(Emgu.CV.CvEnum.ElementShape.Rectangle, new System.Drawing.Size(2 * morph_size + 1, 2 * morph_size + 1), new System.Drawing.Point(morph_size, morph_size));
CvInvoke.MorphologyEx(segmentedImage, segmentedImage, Emgu.CV.CvEnum.MorphOp.Open, element, new System.Drawing.Point(-1, -1), 1, Emgu.CV.CvEnum.BorderType.Default, new MCvScalar());

// Find edges that form rectangles
List<RotatedRect> boxList = new List<RotatedRect>();
using (VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint())
{
CvInvoke.FindContours(segmentedImage, contours, null, Emgu.CV.CvEnum.RetrType.External, ChainApproxMethod.ChainApproxSimple);
int count = contours.Size;
for (int i = 0; i < count; i++)
{
using (VectorOfPoint contour = contours[i])
using (VectorOfPoint approxContour = new VectorOfPoint())
{
CvInvoke.ApproxPolyDP(contour, approxContour, CvInvoke.ArcLength(contour, true) * 0.01, true);
if (CvInvoke.ContourArea(approxContour, false) > 10000)
{
if (approxContour.Size == 4)
{
bool isRectangle = true;
System.Drawing.Point[] pts = approxContour.ToArray();
LineSegment2D[] edges = Emgu.CV.PointCollection.PolyLine(pts, true);
for (int j = 0; j < edges.Length; j++)
{
double angle = Math.Abs(edges[(j + 1) % edges.Length].GetExteriorAngleDegree(edges[j]));
if (angle < 80 || angle > 100)
{
isRectangle = false;
break;
}
}

if (isRectangle)
boxList.Add(CvInvoke.MinAreaRect(approxContour));
}
}
}
}
}

Answer

So as promised i tried it myself. In C++ but you should adopt it easy to Emgu. First i get rid of small object in your segmented image with an opening:

int morph_elem = CV_SHAPE_RECT;
int morph_size = 2;
Mat element = getStructuringElement(morph_elem, Size(2 * morph_size + 1, 2 * morph_size + 1), Point(morph_size, morph_size));
// Apply the opening
morphologyEx(segmentedImage, segmentedImage_open, CV_MOP_OPEN, element);

Then detect all the contours and take the large ones and check for rectangular shape:

vector< vector<Point>> contours;
findContours(segmentedImage_open, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

for each (vector<Point> var in contours)
{
    double area = contourArea(var);
    if (area> 30000) 
    {
        vector<Point> approx;
        approxPolyDP(var, approx, 0.01*arcLength(var, true), true);

        if (4 == approx.size()) //rectangular shape 
        {
            // do something
        }
    }
}

Here is the result with the contour in red and the approximated curve in green:

enter image description here

Edit:

You can improve your code by increasing the approximation factor until you get a contour with 4 points or you pass a threshold. Just wrap a for loop around approxPolyDP. You can define a range for your approximation value and prevent your code to fail if your object differs too much from a rectangle.

Comments