This was one of the later assignments for my Image Processing course in London - we were tasked with implementing a Canny Edge Detector in Matlab. Matlab already has a built-in Canny function, and this is it's output when supplied this picture of a building and a few other parameters.
The built-in function does a very good job. It captures all the dominant edges, and manages to ignore all the grass and most of the weird texture on the lighthouse. The edges do break down as the fence recedes into the distance, and some of the slats on the shed are missing. The first thing I did was experiment with some basic methods - convolution with Sobel and Prewitt kernels.
The process of convolving the image with these kernels essentially reduces to, at each pixel, measuring the strength of the edge as the difference in intensity between the pixel in focus and its neighbors. The horizontal and vertical directions are considered separately, then the edge information is combined at the end. This produces a grayscale map of the "edginess" of each pixel. If that image is then thresholded, it produces results as above. This method does indeed detect edges. However, the lines are messy and there are a lot of undesirable "edges" in the grass and on the tower.
To improve these results, I implemented non-maximal suppression. This takes the messy edges above and reduces them to lines that are only one pixel wide. The program does this by going over the edges and keeping only the strongest "cores" of the lines.
The above image shows the results of performing non-maximal suppression after convolution with the Sobel kernel. The convolution step produces a grayscale map of the edge strengths, then non-maximal suppression reduces the edges to thin lines. Here, the strongest edges are the brightest lines. The picture on the right shows the results of non-maximal suppression after being thresholded. The performance here is okay, there are a lot of broken edges and unnecessary mess.
The next step in writing a Canny edge detector is to implement hysteresis thresholding. This takes the output after non-maximal suppression and thresholds it at two values - high and low. It then uses the data from the high threshold (these are the absolute most resilient edges) to select the best long, continuous edges from the low threshold image. This vastly improves the results.
The combined image directly above is the output of my program after the hysteresis thresholding step. The edges seen in the high threshold image are used to pick out lines from the low threshold image and generate the image here. It has the effect of "healing" broken lines while not allowing the gross edges from the grass. The performance here is quite good. For comparison, here is my output alongside that of the built-in function.
Not bad! My function retained some more of the slats on the shed and fixed the broken roof line, but has some more lines on the light house. Overall the performance is very solid.