Data

Classes and methods which can be used to create and modify datasets.

Datasets

class tensornet.data.BaseDataset(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

Loads a dataset.

Parameters

• train_batch_size (int, optional) – Number of images to consider in each batch in train set. (default: 0)

• val_batch_size (int, optional) – Number of images to consider in each batch in validation set. (default: 0)

• cuda (bool, optional) – True is GPU is available. (default: False)

• num_workers (int, optional) – How many subprocesses to use for data loading. (default: 0)

• path (str, optional) – Path where dataset will be downloaded. If no path provided, data will be downloaded in a pre-defined directory.

• train_split (float, optional) – Fraction of dataset to assign for training. This parameter will not work for MNIST and CIFAR-10 datasets. (default: 0.7)

• resize (tuple, optional) – Resize the input to the given height and width. (default: (0, 0))

• padding (tuple, optional) – Pad the image if the image size is less than the specified dimensions (height, width). (default: (0, 0))

• crop (tuple, optional) – Randomly crop the image with the specified dimensions (height, width). (default: (0, 0))

• horizontal_flip_prob (float, optional) – Probability of an image being horizontally flipped. (default: 0)

• vertical_flip_prob (float, optional) – Probability of an image being vertically flipped. (default: 0)

• rotate_degree (float, optional) – Angle of rotation for image augmentation. (default: 0)

• cutout_prob (float, optional) – Probability that cutout will be performed. (default: 0)

• cutout_dim (tuple, optional) – Dimensions of the cutout box (height, width). (default: (8, 8))

• hue_saturation_prob (float, optional) – Probability of randomly changing hue, saturation and value of the input image. (default: 0)

• contrast_prob (float, optional) – Randomly changing contrast of the input image. (default: 0)

data(train=True)

Return data based on train mode.

Parameters

train (bool, optional) – True for training data. (default: True)

Returns

Training or validation data and targets.

unnormalize(image, transpose=False, data_type=None)

Un-normalize a given image.

Parameters

• image (numpy.ndarray or torch.Tensor) – A ndarray or tensor. If tensor, it should be in CPU.

• transpose (bool, optional) – If True, transposed output will be returned. This param is effective only when image is a tensor. If tensor, the output will have channel number as the last dim. (default: False)

• data_type (str, optional) – Type of image. Required only when dataset has multiple types of images. (default: None)

Returns

Unnormalized image

Return type

(numpy.ndarray or torch.Tensor)

normalize(image, transpose=False, data_type=None)

Normalize a given image.

Parameters

• image (numpy.ndarray or torch.Tensor) – A ndarray or tensor. If tensor, it should be in CPU.

• transpose (bool, optional) – If True, transposed output will be returned. This param is effective only when image is a tensor. If tensor, the output will have channel number as the last dim. (default: False)

• data_type (str, optional) – Type of image. Required only when dataset has multiple types of images. (default: None)

Returns

Normalized image

Return type

(numpy.ndarray or torch.Tensor)

loader(train=True)

Create data loader.

Parameters

train (bool, optional) – True for training data. (default: True)

Returns

Dataloader instance.

class tensornet.data.MNIST(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

MNIST Dataset.

Note: This dataset inherits the BaseDataset class.

class tensornet.data.CIFAR10(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

CIFAR-10 Dataset.

Note: This dataset inherits the BaseDataset class.

class tensornet.data.CIFAR100(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

CIFAR-100 Dataset.

Note: This dataset inherits the BaseDataset class.

class tensornet.data.TinyImageNet(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

Tiny ImageNet Dataset.

Note: This dataset inherits the BaseDataset class.

class tensornet.data.MODESTMuseum(train_batch_size=1, val_batch_size=1, cuda=False, num_workers=1, path=None, train_split=0.7, resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0)

MODEST Museum Dataset.

Note: This dataset inherits the BaseDataset class.

Processing

class tensornet.data.processing.Transformations(resize=0, 0, padding=0, 0, crop=0, 0, horizontal_flip_prob=0.0, vertical_flip_prob=0.0, gaussian_blur_prob=0.0, rotate_degree=0.0, cutout_prob=0.0, cutout_dim=8, 8, hue_saturation_prob=0.0, contrast_prob=0.0, mean=0.5, 0.5, 0.5, std=0.5, 0.5, 0.5, normalize=True, train=True)

Wrapper class to pass on albumentaions transforms into PyTorch.

__call__(image)

Process and image through the data transformation pipeline.

Parameters

image – Image to process.

Returns

Transformed image.

Return type

(torch.Tensor)

tensornet.data.processing.data_loader(data, shuffle=True, batch_size=1, num_workers=1, cuda=False)

Create data loader

Parameters

• data (torchvision.datasets) – Downloaded dataset.

• shuffle (bool, optional) – If True, shuffle the dataset. (default: True)

• batch_size (int, optional) – Number of images to considered in each batch. (default: 1)

• num_workers (int, optional) – How many subprocesses to use for data loading. (default: 1)

• cuda (bool, optional) – True is GPU is available. (default: False)

Returns

DataLoader instance.

Return type

(torch.utils.data.DataLoader)

class tensornet.data.processing.InfiniteDataLoader(data_loader, auto_reset=True)

Create infinite loop in a data loader.

Parameters

• data_loader (torch.utils.data.DataLoader) – DataLoader object.

• auto_reset (bool, optional) – Create an infinite loop data loader. (default: True)

get_batch()

Load next batch from the dataset.