When doing computer vision the first thing being done is usually camera calibration. The most common calibration technique is to make images of a physical object with know spatial properties (grid patterns are often used) and to extract 2D point positions from the resulting images (e.g. using a corner detector). Given the match between the extracted 2D points and the corresponding locations on the calibration grid it is possible to determine camera and lens parameters such focal length and distortion.

An in-house camera calibration application at my company solved the correspondence problem by using an algorithm that looked at properties that are not invariant under projection (i.e. angles and distances). This made the calibration process extremely fragile. The algorithm often failed to detect the calibration grid even when the image was crystal clear, which made calibrating a camera a very frustrating endeavour.

Since it was an in-house application there was never much priority to fix it. Eventually I got so annoyed though that I wrote a whole new real-time algorithm for the detection of arbitrarily-sized grids from a set of 2D points. The algorithm is capable of dealing with a significant amount of outliers (even when between the valid grid points), it can handle missing grid points and it is not affected by perspective or non-linear lens distortion. In the end it took me longer that I had hoped, but the resulting algorithm is one that still makes me proud.