Convolution, Filtering, and Edges

Convolution is the basic operation of image processing. A small grid of weights, the kernel, slides over the image. At each spot, the output pixel is the weighted sum of the pixels under the kernel. Different weights blur, sharpen, or pick out edges.

Kernel Animate the sliding window

Convolution

For a kernel $K$ of size $(2k+1) \times (2k+1)$, each output pixel is a weighted sum of its neighborhood:

$$O(x, y) = \sum_{i=-k}^{k}\sum_{j=-k}^{k} K(i, j)\; I(x + i,\ y + j)$$

The Sobel kernels estimate the brightness gradient. Combining them gives the edge strength $\sqrt{G_x^2 + G_y^2}$.

What to notice

Blur kernels have all positive weights that sum to 1, so they average neighbors and smooth out noise and detail. The sharpen kernel boosts the center pixel against its neighbors. The Sobel kernels have positive and negative weights that sum to 0, so they output nothing on flat areas and respond strongly where brightness changes, at the edges of objects. Edge detection is often the first step before finding lines, corners, or object outlines, and the first layer of a trained neural network often learns kernels that look a lot like these.