The importance of assumptions

When we are using methods from image analysis, or most other computational techniques, we must make assumptions about the kinds of images the methods will work on.

No image analysis approach will work on all image data, in all situations. Even cutting-edge machine learning approaches, which are famous for working in many imaging situations, still need certain properties in the image data to work correctly.

Assumptions about images can come in many forms. It may be as simple as making the assumption that the foreground object is brighter or darker than the background. A leaf, for example, could be imaged against a lighter background, meaning thresholding could be used to segment the darker leaf. But if a light leaf is imaged against a darker background (ie. the inverse scenario), that assumption is broken, and, as you can imagine, any code written to threshold the image will fail. Even relatively small changes in the brightness of an image can cause thresholding to fail dramatically, as seen in the image below.

Figure 1. Two wheat root images are thresholded with the same global value. The lower image is only slightly darker, yet thresholding with the same threshold produces a much worse segmentation. The assumptions about the appearance of the root have been broken.

Of course, many image analysis methods are more resilient than global thresholding, but will have assumptions of their own which need to be met in order for developed methods to work. Some common assumptions that image analysis approaches make may include:

• Background appearance in the image
• Lighting conditions
• Where in the image the object of interest is located
• The size/zoom of the object in the image
• The pose/orientation of the object
• Image properties: Resolution of the image, image format, is the image colour or greyscale
• Shape properties (e.g. a method which uses a shape model may work on one class of leaf shape, but not another)
• Statistical properties of the data (e.g. that the object occupies 50% of the image; or that the histogram of the image is bi-modal).
• The image is sufficiently free of noise

There are potential many explicit (or implied) assumptions present when you are using a software tool. What is key is to understand how the approach will work on your data. That requires either understanding the techniques which make up the system, or performing thorough testing with representative images; or, even better, both!