As a preprocessing step, we perform various kinds of augmentations to our images to see which of them can logically enhance our dataset, thereby increasing the ability to generalize on unseen images. You can learn more about the effects of augmentation in Improving Model Performance. This is done during the workflow after your dataset and annotations are completed, as these augmentations don't have to be performed on the images all at once.

While augmentations are good for dataset robustness, keep in mind that augmentations make it harder for the model to learn as well, so randomly selecting augmentations will not necessarily improve model inference and can hinder model training.

Here is an example image of how the augmentation menu will look like in Advanced Mode, where you can choose your own custom options.

Augmentations Module Advanced Mode Options (Click image to enlarge)

Only one Augmentations module is required for each workflow, regardless of the number of augmentation types enabled.

Augmentation Types

There are three types of augmentations: composite image augmentations, random position augmentations, and color space augmentations.

Composite Augmentations

These augmentations consist of putting multiple images together to increase diversity within a single image by combining multiple images in various ways.

Composite Augmentations	Options	Description
CutMix	Probability	CutMix takes a random crop from another image in the dataset and pastes it on top of the base image.
Mosaic	Probability	Mosaic takes multiple images, typically 4 images, and places it in a tiled image. Subsequently, a random crop is taken of on the canvas of 4 tiled images.

Positional Augmentations

These are what we call the geometric transforms. Common positional augmentations include HorizontalFlip, VerticalFlip and ShiftScaleRotate and our library has them all. Positional augmentations are ways to keep the size of the data input more similar but also augmenting the dataset by shifting or removing certain details in the visual data such that the model isn't reliant on specific features. The full list is below. The image here displays an example of positional augmentations like HorizontalFlip, VerticalFlip, ShiftScaleRotate, RandomRotate, and RandomSizedCrop using our Preview Augmentations feature at the bottom right corner of the Workflow canvas next to the green Run Training button.

📘
To see how to preview these augmentations, look at Previewing Augmentations!

2022 — Positional Augmentation Example (Click image to enlarge)

Spatial Augmentations	Options	Description
Center Crop	Probability, Height, Width	Center crop takes a crop of set height and width from the middle of the image with set probability. Probability should be a value from 0 to 1, height and width should be less than or equal to the image height and width.
Flip Horizontal	Probability	Flips the image horizontally with set probability. Probability should be a value from 0 to 1.
Flip Vertical	Probability	Flips the image vertically with set probability. Probability should be a value from 0 to 1.
Random Crop	Probability, Height, Width	Random crop takes a crop of set height and width from anywhere in the image with set probability. Probability should be a value from 0 to 1, height and width should be less than or equal to the image height and width.
Random Rotate	Probability, Minimum Rotation, Maximum Rotation	Randomly rotates the image by some angle between the minimum and maximum rotation with set probability. Minimum rotation is -360 degrees, maximum rotation is 360 degrees.
Random Resized Crop	Probability, Height, Width, Minimum Scale, Maximum Scale, Minimum Aspect Ratio, Maximum Aspect Ratio	Random crop with set height and width taken from anywhere in the image and then resized with random scale between minimum and maximum scale, and transformed to a random aspect ratio between the minimum and maximum aspect ratio. Randomly apply affine transforms: translate, scale and rotate the input with set probability.
Random Sized Crop	Probability, Minimum Crop Height, Maximum Crop Height, Height, Width, Width to Height Ratio	Random crop with set width to height ratio and random crop height with set probability. The set crop height bounds should be within the image dimensions.
Shift Scale Rotate	Probability, Minimum Shift, Maximum Shift, Minimum Scale, Maximum Scale, Minimum Rotation, Maximum Rotation	Randomly apply affine transforms: translate, scale and rotate the input by randomly selected shift, scale, and rotate with set probability.

Color Augmentations

The other popular type of augmentation is color space augmentaiton. Basically, this type of augmentation does not shift the location of the pixels, but instead, will alter pixel values such that the color representation changes. For example, the RandomPixelValueScale randomly scales a pixel value between the min_scale and the max_scale. The random selection is based on a uniform distribution, so all values are equally likely to be picked.

The reason to use color augmentations is so that the model is not dependent on the color space for making predictions. Often times, data collecting instruments will not be identical and the lighting in which the visual data is collected may not be the same, so colors can be drastically different in test or inference environments even if the main features in the data remain the same. Thus, to artificially create color diversity in our train set, we use color augmentations to bolster the model's ability to detect features in different color spaces.

Below is an example of only color space augmentations such as color jitter, random brightness, random contrast, and solarize.

Other color augmentations are very extensive with the following being included:

Color Augmentations	Options	Description
Channel Dropout	Probability, Min Channel Dropout Limit, Max Channel Dropout Limit, Fill Value	Drop a random number of RGB channels between the minimum and maximum values, and replaced with a set fill value with set probability.
Channel Shuffle	Probability	Randomly rearrange RGB channel value order with set probability.
CLAHE	Probability, Min Clip Limit, Max Clip Limit, Min Grid Limit, Max Grid Limit	Apply Contrast Limited Adaptive Histogram Equalization to the input image used for improving the visibility level of foggy image or video. Minimum and maximum clip limits act as upper threshold values for contrast limiting, and the grid limits define where CLAHE is applied.
Color Jitter	Probability, Min Brightness, Max Brightness, Min Contrast, Max Contrast, Min Saturation, Max Saturation, Min Hue, Max Hue	ColorJitter randomly changes the brightness, contrast, saturation, and hue of an image. For brightness, contrast, and saturation, you supply the maximum and minimum bound values, which change with set probability.
Cutout	Probability, Number of Holes, Max Hole Height, Max Hole Width, Fill Value	Randomly select a number of areas with random sized height and width with defined maximums to fill with some set value.
Equalize	Probability	Improve contrast in images by effectively spreading out the most frequent intensity values, i.e. stretching out the intensity range of the image.
Fancy PCA	Probability, Alpha	Augments the color of the image by adding the most significant eigenvectors (defined through Principle Component Analysis or PCA) multiplied by some set alpha to the RGB values of each pixel. This creates the effect of sharpening the most vivid colors but in a more natural way than random augmentation. According to the original paper, alpha should be selected from a normal distribution with mean 0 and standard deviation 0.1.
Gaussian Blur	Probability, Min Sigma Minimum, Max Sigma Minimum	Blur the input image using a Gaussian filter with a random kernel size, using Gaussian function with sigma between the minimum and maximum value.
Gaussian Noise	Probability, Min Variance Range, Max Variance Range, Mean	Gaussian noise adds noise values selected from a Gaussian distribution with variance between the minimum and maximum variance range and with the set mean.
Glass Blur	Probability, Sigma, Max Delta, Iterations	Applies glass noise based on the inputted sigma for the Gaussian function, and the maximum delta for the maximum distance between swapped pixels and iterations being the number of times this occurs.
Grayscale	Probability	Grayscale converts all color pixels into a black and white scale, making the image entirely grayscale.
Hue Saturation Value	Probability, Min Hue Shift, Max Hue Shift, Min Saturation Shift, Max Saturation Shift, Min Value Shift, Max Value Shift	Randomly change hue, saturation and value of the input image bounded by the set minimum and maximum values.
Image Compression	Probability, Lower Quality Limit, Upper Quality Limit	Compresses the image on a scale of 0 to 100 within the lower and upper quality limit set.
ISO Noise	Probability, Min Color Shift, Max Color Shift, Min Intensity, Max Intensity	ISO noise applies camera sensor noise, which can be useful if your image data is collected from camera instruments in the real world.
Median Blur	Probability, Max Blur	The Median blur operation is similar to the other averaging methods. Here, the central element of the image is replaced by the median of all the pixels in the kernel area. This operation processes the edges while removing the noise.
Motion Blur	Probability, Max Blur	Using convolution filters and a randomly generated line, motion blur is constructed with the blur depending on the maximum blur set.
Multiplicative Noise	Probability, Min Multiplication Limit, Max Multiplication Limit	Multiply image to random number selected between the minimum and maximum multiplication limit.
Posterize	Probability, R Channel Number of Bits, G Channel Number of Bits, B Channel Number of Bits	Reduces the number of bits for each color channel.
Random Blur	Probability, Min Blur Limit, Max Blur Limit	Random blur applied with blur limited in between minimum and maximum.
Random Brightness	Probability, Min Brightness Limit, Max Brightness Limit	Randomly change brightness of the image within the minimum and maximum brightness.
Random Brightness Contrast	Probability, Min Brightness Limit, Max Brightness Limit, Min Contrast Limit, Max Contrast Limit	Randomly change brightness and contrast of the image within the minimum and maximum brightness and contrast.
Random Contrast	Probability, Min Contrast Limit, Max Contrast Limit	Randomly change contrast of the image within the minimum and maximum contrast.
Random Fog	Probability, Lower Coefficient Limit, Upper Coefficient Limit, Alpha Coefficient Limit	Applies fog artificially to an image. You can control the fog intensity levels with the coefficient limit, and alpha is the transparency of the fog circle.
Random Gamma	Probability, Min Gamma Limit, Max Gamma Limit	Gamma changes the brightness of the image restricted to minimum and maximum gamma limits.
Random Rain	Probability, Lower Slant, Upper Slant, Drop Length, Drop Width, Drop Color Red, Drop Color Green, Drop Color Blue, Blur Value, Brightness Coefficient	Adds rain like effects where you can control the drop color with the red, green, and blue as well as the drop size with length and width, and the lower and upper slant for shape.
Random Shadow	Probability, X Min, Y Min, X Max, Y Max, Number of Shadows Lower Limit, Number of Shadows Upper Limit, Number of Edges	Random shadow applies a number of shadows between the lower and upper limit within a certain area space.
Random Snow	Probability, Lower Snow Limit, Upper Snow Limit, Brightness Coefficient	Random snow effects are applied, bounded by the limit values.
Random Sun Flare	Probability, X Min, Y Min, X Max, Y Max, Angle Lower Limit, Angle Upper Limit, Number of Flares Lower Limit, Number of Flares Upper Limit	Applies random number of sun flares within a certain area from a random angle.
RGB Shift	Probability, R Channel Shift Lower Limit, R Channel Shift Upper Limit, G Channel Shift Lower Limit, G Channel Shift Upper Limit, B Channel Shift Lower Limit, B Channel Shift Upper Limit	Random RGB shift for each individual channel by a random amount for each.
Sepia	Probability	Adds the common image effect of sepia to the image.
Solarize	Probability, Threshold	Inverts all pixels above the preset threshold.