I am writing this short post on a small interesting old fact that I found being unknown to some (probably few): derivatives of Gaussian kernels are steerable… wait stee-what?
In computer vision and image processing is not uncommon to find ourselves in the need of oriented filters to analyze responses under different orientations.
We can describe as “Steerable filters” [Freeman and Adelson 1991] a class of filters in which we can synthesize a filter for an arbitrary orientation through a linear combination of basis filters. To put this into equations, if a class of filters is steerable we can have:
where: is our filter, the various are the filters in the basis and the factors are what we can call “steering functions”.
Not only we can easily create filters with arbitrary orientation, we can actually skip that and obtain directly their response if we already have a response for the basis filters. This is because when it comes to convolution:
This is great! It means that if we need outputs for lots of orientations, we don’t have to compute lots of expensive convolutions, but only as many as the number of filters in our basis.
Derivative of Gaussians steerability
One very useful class of filters that is steerable is the Derivative of Gaussians (DoG).
Derivative of Gaussians are used a lot in vision/image processing as a way to detect edges (or generally features). I would write more about this, but it would transform this in a lengthier post than I want, so I’ll throw there only two things:
- They are great to compute gradients and therefore to detect discontinuities. With a large standard deviation we have poor localization (bad at finidng the exact location containing the edge) but good detection (good at detecting proper edges). With low standard deviation we have the opposite, good localization, but poor detection.
- First order derivatives at different scales can be used to approximate a Laplacian of Gaussian (LoG), the second order can be used to actually compute the Laplacian. LoG are extremely useful to detect blobs.
So that’s what the first order of Derivative of Guassians look like in x and y directions:
These two nice guys here are one possible choice of basis filters to compute a DoG oriented however we like. But what are the steering functions? It’s real simple, if we want to find the filter oriented along the direction , then the steering functions are simply:
Therefore, using the first equation in the post, our oriented DoG is obtained with:
I’ve put a short proof of this in Appendix A at the bottom of the post.
With these filters we can now compute, for example, the gradients at a certain direction:
which could be useful, for example, in feature detection scenarios and I am sure in other cases that don’t come to my mind now 🙂
Note: All of the above extends to higher derivative orders.
Additional use cases and further readings
I wanted to keep this post short as this is just a little nice property. Even so, I have treated it probably too shortly, there are other interesting facts and use cases. Here is a quick list of possible use cases for steerable filters (not limited to DoG) and links to papers you could read to understand more about them.
- Feature detection: I have briefly illustrated how to use steerable filters to find edges in a specific direction. Steerable filters are also useful for example in junctions detection [Perona 92] or detecting the orientation of an image [Freeman and Adelson 1991]. Plenty of other papers on this field can be found (e.g. [Jacob and Unser 2004] and many many more).
- Scene relighting: An interesting usage of steerable functions can be seen for surface re-lighting. One of the first example of steerable light sources is from [Teo et al. 1997]; they steered light sources to relight a static scene, however using a very large (probably impractical) basis.
Another very interesting (at least historically) example are the Steerable Illumination textures of [Ashikhmin and Shirley 2002]; They calculate a basis of “baked” illumination, steered to obtain the final result on bumpy surfaces from an arbitrary light position without any cross-fading. They use smaller basis than [Teo et al. 97].
- Image stitching: Steerable filters have been used to aid in image stitching where finding orientation of corners could help. This has been done for “classical” image mosaics [Subramanyam 2013] or with a stronger focus on satellite images [Li et al. 2007].
- Image filtering: [Freeman and Adelson 1991] shows how to remove noise from an image, while enhancing oriented structures. [Gotsman 1994] used steerable gaussian for texture filtering.
- Motion detection: Steerable filters are also used to aid in motion detection algorithms. I don’t have a specific paper to link, but steerable filters are often used, implicitly or not.
That is all! It’s probably not the most interesting or incredibly obscure thing, but I figured to post it in the wild nonetheless 🙂 I hope you enjoyed it at least a bit.
’til the next one!
Appendix A: Derivation of steerability functions for Derivative of Gaussians
Let’s first express the first order derivative of a gaussian in the x direction in polar coordinates as:
If we then rotate by we will obviously obtain the derivative in the y direction:
Now, let’s rotate the filter by an arbitrary angle , using trigonometric identities we obtain:
which is what I mentioned before 🙂