tonic.datasets

`tonic.datasets`#

Submodules#

Package Contents#

Classes#

`ASLDVS`	ASL-DVS
`CIFAR10DVS`	CIFAR10-DVS
`DAVISDATA`	DAVIS event camera dataset
`DSEC`	DSEC
`DVSLip`	DVS-Lip
`DVSGesture`	IBM DVS Gestures
`EBSSA`	EBSSA
`SHD`	Spiking Heidelberg Digits
`SSC`	Spiking Speech Commands
`MVSEC`	MVSEC
`NCALTECH101`	N-CALTECH101
`NMNIST`	N-MNIST
`POKERDVS`	POKER-DVS
`SMNIST`	Spiking sequential MNIST
`TUMVIE`	TUM-VIE
`VPR`	Visual Place Recognition

class tonic.datasets.ASLDVS(save_to: str, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

ASL-DVS

@inproceedings{bi2019graph,
    title={Graph-based Object Classification for Neuromorphic Vision Sensing},
    author={Bi, Y and Chadha, A and Abbas, A and and Bourtsoulatze, E and Andreopoulos, Y},
    booktitle={2019 IEEE International Conference on Computer Vision (ICCV)},
    year={2019},
    organization={IEEE}
}

Parameters:

save_to (string) – Location to save files to on disk.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

url = 'https://www.dropbox.com/sh/ibq0jsicatn7l6r/AACNrNELV56rs1YInMWUs9CAa?dl=1'#

filename = 'ASLDVS.zip'#

file_md5 = '33f8b87bf45edc0bfed0de41822279b9'#

folder_name = ''#

classes#

int_classes#

sensor_size = (240, 180, 2)#

dtype#

ordering#

__getitem__(index: int) → Tuple[Any, Any][source]#

Returns:: (events, target) where target is index of the target class.
Parameters:: index (int) –
Return type:: Tuple[Any, Any]

__len__()[source]#

class tonic.datasets.CIFAR10DVS(save_to: str, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

CIFAR10-DVS

@article{li2017cifar10,
title={Cifar10-dvs: an event-stream dataset for object classification},
author={Li, Hongmin and Liu, Hanchao and Ji, Xiangyang and Li, Guoqi and Shi, Luping},
journal={Frontiers in neuroscience},
volume={11},
pages={309},
year={2017},
publisher={Frontiers}
}

Parameters:

save_to (string) – Location to save files to on disk.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

url = 'https://figshare.com/ndownloader/files/38023437'#

filename = 'CIFAR10DVS.zip'#

file_md5 = 'ce3a4a0682dc0943703bd8f749a7701c'#

data_filename = ['airplane.zip', 'automobile.zip', 'bird.zip', 'cat.zip', 'deer.zip', 'dog.zip', 'frog.zip',...#

folder_name = ''#

dtype#

ordering#

sensor_size = (128, 128, 2)#

classes#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__()[source]#

class tonic.datasets.DAVISDATA(save_to: str, recording: Union[str, List[str]], transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

DAVIS event camera dataset

@article{mueggler2017event,
  title={The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM},
  author={Mueggler, Elias and Rebecq, Henri and Gallego, Guillermo and Delbruck, Tobi and Scaramuzza, Davide},
  journal={The International Journal of Robotics Research},
  volume={36},
  number={2},
  pages={142--149},
  year={2017},
  publisher={SAGE Publications Sage UK: London, England}
}

Parameters:

save_to (string) – Location to save files to on disk. Will save files in a sub folder ‘davis_dataset’.
recording (string) – Use the name of the recording or a list thereof to download it, for example ‘dynamic_6dof’ or [‘slider_far’, ‘urban’]. See project homepage for a list of available recordings. Can use ‘all’ to download all available recordings.
transform (callable, optional) – A callable of transforms to apply to events and/or images.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'http://rpg.ifi.uzh.ch/datasets/davis/'#

recordings#

sensor_size = (240, 180, 2)#

dtype#

ordering#

folder_name = ''#

__getitem__(index)[source]#

Returns:: tuple of (data, target), where data is another tuple of (events, imu, images) and target is the opti track ground truth

__len__()[source]#

download()[source]#: Downloads from a given url, places into target folder and verifies the file hash.

class tonic.datasets.DSEC(save_to: str, split: Union[str, List[str]], data_selection: Union[str, List[str]], target_selection: Optional[Union[str, List[str]]] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

DSEC

This is a fairly large dataset, so in order to save some disk space, event and image zips are deleted after extraction. If your download gets interrupted and you are left with a corrupted file on disk, Tonic will not be able to detect that and just proceed to download files that are not yet on disk. If you experience issues loading a particular recording, delete that folder manually and Tonic will re-download it the next time. Optical flow targets are not available for every recording, so if you select optical flow targets, only a subset of 18 training recordings will be selected.

Note

To be able to read this dataset, you will need hdf5plugin, PIL and imageio packages installed.

Parameters:

save_to (str) – Location to save files to on disk.
split (str) – Can be ‘train’, ‘test’ or a selection of individual recordings such as ‘interlaken_00_c’ or [‘thun_00_a’, ‘zurich_city_00_a’]. Cannot mix across train/test.
data_selection (str) – Select which data to load per sample. Can be ‘events_left’, ‘events_right’, ‘images_rectified_left’, ‘images_rectified_right’, ‘image_timestamps’ or any combination thereof in a list.
target_selection (str, optional) – Select which targets to load per sample. Can be ‘disparity_events’, ‘disparity_images’, ‘disparity_timestamps’, ‘optical_flow_forward_event’, ‘optical_flow_forward_timestamps’, ‘optical_flow_backward_event’, ‘optical_flow_backward_timestamps’ or a combination thereof in a list. Note that optical flow targets are not available for every recording.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://download.ifi.uzh.ch/rpg/DSEC/'#

recordings#

data_names#

target_names#

sensor_size = (640, 480, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (data, target) where data is another tuple of data_selction and target a tuple of target_selection if train=True.

__len__()[source]#

class tonic.datasets.DVSLip(save_to: str, train: bool = True, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

DVS-Lip

@inproceedings{tan2022multi,
    title={Multi-Grained Spatio-Temporal Features Perceived Network for Event-Based Lip-Reading},
    author={Tan, Ganchao and Wang, Yang and Han, Han and Cao, Yang and Wu, Feng and Zha, Zheng-Jun},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    pages={20094--20103},
    year={2022}
}

Implementation inspired from original script: https://github.com/tgc1997/event-based-lip-reading/blob/main/utils/dataset.py

Parameters:

save_to (string) – Location to save files to on disk.
train (bool) – If True, uses training subset, otherwise testing subset.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://drive.google.com/file/d/1dBEgtmctTTWJlWnuWxFtk8gfOdVVpkQ0/view'#

filename = 'DVS-Lip.zip'#

base_folder = 'DVS-Lip'#

file_md5 = '2dcb959255122d4cdeb6094ca282494b'#

sensor_size = (128, 128, 2)#

dtype#

ordering#

classes = ['accused', 'action', 'allow', 'allowed', 'america', 'american', 'another', 'around', 'attacks',...#

ambiguous_classes = ['action', 'allow', 'allowed', 'america', 'american', 'around', 'being', 'benefit', 'benefits',...#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__()[source]#

class tonic.datasets.DVSGesture(save_to: str, train: bool = True, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

IBM DVS Gestures

Note

This is (exceptionally) a preprocessed version of the original dataset, where recordings that originally contained multiple labels have already been cut into respective samples. Also temporal precision is reduced to ms.

@inproceedings{amir2017low,
  title={A low power, fully event-based gesture recognition system},
  author={Amir, Arnon and Taba, Brian and Berg, David and Melano, Timothy and McKinstry, Jeffrey and Di Nolfo, Carmelo and Nayak, Tapan and Andreopoulos, Alexander and Garreau, Guillaume and Mendoza, Marcela and others},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={7243--7252},
  year={2017}
}

Parameters:

save_to (string) – Location to save files to on disk.
train (bool) – If True, uses training subset, otherwise testing subset.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

test_url = 'https://figshare.com/ndownloader/files/38020584'#

train_url = 'https://figshare.com/ndownloader/files/38022171'#

test_md5 = '56070e45dadaa85fff82e0fbfbc06de5'#

train_md5 = '3a8f0d4120a166bac7591f77409cb105'#

test_filename = 'ibmGestureTest.tar.gz'#

train_filename = 'ibmGestureTrain.tar.gz'#

classes = ['Hand clapping', 'Right hand wave', 'Left hand wave', 'Right arm cw', 'Right arm ccw', 'Left...#

sensor_size = (128, 128, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__()[source]#

class tonic.datasets.EBSSA(save_to: str, split: str = 'labelled', transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

EBSSA

There are six different splits provided in this dataset. The labelled section of the dataset contains 84 recordings and 84 label files. The unlabelled section of the dataset contains 153 recordings in folders marked “Unlabelled”.

@article{afshar2020event,
    title={Event-based object detection and tracking for space situational awareness},
    author={Afshar, Saeed and Nicholson, Andrew Peter and Van Schaik, Andre and Cohen, Gregory},
    journal={IEEE Sensors Journal},
    volume={20},
    number={24},
    pages={15117--15132},
    year={2020},
    publisher={IEEE}
}

Parameters:

save_to (string) – Location to save files to on disk.
split (string) – Which split to load. One of “labelled”, “unlabelled”, “all”.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

file_id = '1lCh2HWvxEzzaBHT5TlPuyUn6XPM5OVWN'#

folder_name = ''#

file_name = 'labelled_ebssa.h5'#

sensor_size = (240, 180, 2)#

dtype#

ordering#

__getitem__(index: int) → Tuple[Any, Any][source]#

Returns:: (events, target) where target is dict of bounding box and recording id.
Parameters:: index (int) –
Return type:: Tuple[Any, Any]

__len__()[source]#

download()[source]#: Downloads from a given url, places into target folder and verifies the file hash.

class tonic.datasets.SHD(save_to: str, train: bool = True, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)[source]#

Bases: HSD

Spiking Heidelberg Digits

@article{cramer2020heidelberg,
  title={The heidelberg spiking data sets for the systematic evaluation of spiking neural networks},
  author={Cramer, Benjamin and Stradmann, Yannik and Schemmel, Johannes and Zenke, Friedemann},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2020},
  publisher={IEEE}
}

Parameters:

save_to (string) – Location to save files to on disk. Will put files in an ‘hsd’ subfolder.
train (bool) – If True, uses training subset, otherwise testing subset.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.

Returns:

A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets).

property speaker#

test_zip = 'shd_test.h5.zip'#

train_zip = 'shd_train.h5.zip'#

test_md5 = '1503a5064faa34311c398fb0a1ed0a6f'#

train_md5 = 'f3252aeb598ac776c1b526422d90eecb'#

folder_name = ''#

class tonic.datasets.SSC(save_to: str, split: str = 'train', transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)[source]#

Bases: HSD

Spiking Speech Commands

@article{cramer2020heidelberg,
  title={The heidelberg spiking data sets for the systematic evaluation of spiking neural networks},
  author={Cramer, Benjamin and Stradmann, Yannik and Schemmel, Johannes and Zenke, Friedemann},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2020},
  publisher={IEEE}
}

Parameters:

save_to (string) – Location to save files to on disk. Will put files in an ‘hsd’ subfolder.
split (string) – One of ‘train’, ‘test’ or ‘valid’.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.

Returns:

A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets).

test_zip = 'ssc_test.h5.zip'#

train_zip = 'ssc_train.h5.zip'#

valid_zip = 'ssc_valid.h5.zip'#

test_md5 = 'a35ff1e9cffdd02a20eb850c17c37748'#

train_md5 = 'd102be95e7144fcc0553d1f45ba94170'#

valid_md5 = 'b4eee3516a4a90dd0c71a6ac23a8ae43'#

folder_name = ''#

class tonic.datasets.MVSEC(save_to: str, scene: str, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

MVSEC

@article{zihao2018multi,
  title={The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception},
  author={Zihao Zhu, Alex and Thakur, Dinesh and Ozaslan, Tolga and Pfrommer, Bernd and Kumar, Vijay and Daniilidis, Kostas},
  journal={arXiv e-prints},
  pages={arXiv--1801},
  year={2018}
}

Parameters:

save_to (string) – Location to save files to on disk.
scene (string) – Choose one of 4 scenes: outdoor_night, outdoor_day, indoor_flying, motorcycle. If you already have the data on your system, make sure to place the .bag files in a subfolder ‘MVSEC/{scene}/bag_files.bag’.
transform (callable, optional) – A callable of transforms to apply to events and / or images for both left and right cameras.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

resources#

base_url = 'http://visiondata.cis.upenn.edu/mvsec/'#

sensor_size = (346, 260, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: tuple of (data, targets), where data is another tuple of (events_left, events_right, imu_left, imu_right, images_left, images_right) and targets is a tuple of (depth_rect_left, depth_rect_right, pose) for ground truths.

__len__()[source]#

download()[source]#: Downloads from a given url, places into target folder and verifies the file hash.

class tonic.datasets.NCALTECH101(save_to: str, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

N-CALTECH101

Events have (xytp) ordering.

@article{orchard2015converting,
  title={Converting static image datasets to spiking neuromorphic datasets using saccades},
  author={Orchard, Garrick and Jayawant, Ajinkya and Cohen, Gregory K and Thakor, Nitish},
  journal={Frontiers in neuroscience},
  volume={9},
  pages={437},
  year={2015},
  publisher={Frontiers}
}

Parameters:

save_to (string) – Location to save files to on disk.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

url = 'https://data.mendeley.com/public-files/datasets/cy6cvx3ryv/files/36b5c52a-b49d-4853-addb-a836a888...'#

filename = 'N-Caltech101-archive.zip'#

file_md5 = '66201824eabb0239c7ab992480b50ba3'#

data_filename = 'Caltech101.zip'#

folder_name = 'Caltech101'#

sensor_size#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__()[source]#

class tonic.datasets.NMNIST(save_to: str, train: bool = True, first_saccade_only: bool = False, stabilize: bool = False, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

N-MNIST

Events have (xytp) ordering.

@article{orchard2015converting,
  title={Converting static image datasets to spiking neuromorphic datasets using saccades},
  author={Orchard, Garrick and Jayawant, Ajinkya and Cohen, Gregory K and Thakor, Nitish},
  journal={Frontiers in neuroscience},
  volume={9},
  pages={437},
  year={2015},
  publisher={Frontiers}
}

Parameters:

save_to (string) – Location to save files to on disk.
train (bool) – If True, uses training subset, otherwise testing subset.
first_saccade_only (bool) – If True, only work with events of the first of three saccades. Results in about a third of the events overall.
stabilize (bool) – If True, it stabilizes egomotion of the saccades, centering the digit.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://data.mendeley.com/public-files/datasets/468j46mzdv/files/'#

train_url#

train_filename = 'train.zip'#

train_md5 = '20959b8e626244a1b502305a9e6e2031'#

train_folder = 'Train'#

test_url#

test_filename = 'test.zip'#

test_md5 = '69ca8762b2fe404d9b9bad1103e97832'#

test_folder = 'Test'#

classes = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four', '5 - five', '6 - six', '7 - seven',...#

sensor_size = (34, 34, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__() → int[source]#

Return type:: int

class tonic.datasets.POKERDVS(save_to: str, train: bool = True, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

POKER-DVS

Events have (txyp) ordering.

@article{serrano2015poker,
  title={Poker-DVS and MNIST-DVS. Their history, how they were made, and other details},
  author={Serrano-Gotarredona, Teresa and Linares-Barranco, Bernab{'e}},
  journal={Frontiers in neuroscience},
  volume={9},
  pages={481},
  year={2015},
  publisher={Frontiers}
}

Parameters:

save_to (string) – Location to save files to on disk.
train (bool) – If True, uses training subset, otherwise testing subset.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://nextcloud.lenzgregor.com/s/'#

train_filename = 'pips_train.tar.gz'#

test_filename = 'pips_test.tar.gz'#

train_url#

test_url#

train_md5 = '412bcfb96826e4fcb290558e8c150aae'#

test_md5 = 'eef2bf7d0d3defae89a6fa98b07c17af'#

classes = ['cl', 'he', 'di', 'sp']#

int_classes#

sensor_size = (35, 35, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (events, target) where target is the index of the target class.

__len__()[source]#

class tonic.datasets.SMNIST(save_to, train=True, duplicate=True, num_neurons=99, dt=1000.0, transform=None, target_transform=None)[source]#

Bases: tonic.dataset.Dataset

Spiking sequential MNIST

Sequential MNIST (sMNIST) is a standard benchmark task for time series classification where each input consists of sequences of 784 pixel values created by unrolling the MNIST digits, pixel by pixel. In this spiking version, each of the 99 input neurons is associated with a particular threshold for the grey value, and this input neuron fires whenever the grey value crosses its threshold in the transition from the previous to the current pixel.

Parameters:

save_to (string) – Location to save files to on disk.
train (bool) – If True, uses training subset, otherwise testing subset.
duplicate (bool) – If True, emits two spikes per threshold crossing
num_neurons (integer) – How many neurons to use to encode thresholds(must be odd)
dt (float) – Duration(in microseconds) of each timestep
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform – A callable of transforms to apply to the targets/labels.

base_url = 'https://storage.googleapis.com/cvdf-datasets/mnist/'#

train_images_file = 'train-images-idx3-ubyte'#

train_labels_file = 'train-labels-idx1-ubyte'#

test_images_file = 't10k-images-idx3-ubyte'#

test_labels_file = 't10k-labels-idx1-ubyte'#

train_images_md5 = 'f68b3c2dcbeaaa9fbdd348bbdeb94873'#

train_labels_md5 = 'd53e105ee54ea40749a09fcbcd1e9432'#

test_images_md5 = '9fb629c4189551a2d022fa330f9573f3'#

test_labels_md5 = 'ec29112dd5afa0611ce80d1b7f02629c'#

dtype#

ordering#

classes = ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four', '5 - five', '6 - six', '7 - seven',...#

__getitem__(index)[source]#

__len__()[source]#

download()[source]#: Downloads from a given url, places into target folder and verifies the file hash.

class tonic.datasets.TUMVIE(save_to: str, recording: Union[str, List[str]], transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

TUM-VIE

TUM-VIE is an event camera dataset for developing 3D perception and navigation algorithms. It contains handheld and head-mounted sequences in indoor and outdoor environments with rapid motion during sports and high dynamic range. TUM-VIE includes challenging sequences where state-of-the art VIO fails or results in large drift. Hence, it can help to push the boundary on event-based visual-inertial algorithms.

The dataset contains:

Stereo event data Prophesee Gen4 HD (1280x720 pixels)
Stereo grayscale frames at 20Hz (1024x1024 pixels)
IMU data at 200Hz
6dof motion capture data at 120Hz (beginning and end of each sequence)

Timestamps between all sensors are synchronized in hardware.

Note

To be able to read this dataset, you will need hdf5plugin and PIL packages installed.

Note

Use bike-night instead of bike-dark in the list of recordings if you want that specific one.

@string{iros="International Conference on Intelligent Robots and Systems (IROS)"}
@inproceedings{klenk2021tumvie,
 author = {S Klenk and J Chui and N Demmel and D Cremers},
 title = {TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset},
 eprint = {2108.07329},
 eprinttype = {arXiv},
 eprintclass = {cs.CV},
 booktitle = {International Conference on Intelligent Robots and Systems (IROS)},
 year = {2021},
 keywords = {tumvie, event camera, dynamic vision sensor, SLAM, vslam},
}

Parameters:

save_to (string) – Location to save files to on disk. Will save files in a sub folder ‘davis_dataset’.
recording (string) – Use the name of the recording or a list thereof to download it, for example ‘skate-hard’ or [‘skate-hard’, ‘bike-easy’]. See project homepage for a list of available recordings. Can use ‘all’ to download all available recordings.
transform (callable, optional) – A callable of transforms to apply to events and/or images.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://tumevent-vi.vision.in.tum.de/'#

recordings = ['mocap-1d-trans', 'mocap-3d-trans', 'mocap-6dof', 'mocap-desk', 'mocap-desk2', 'mocap-shake',...#

filenames = ['events_left.h5', 'events_right.h5', 'vi_gt_data.tar.gz']#

sensor_size = (1280, 720, 2)#

dtype#

ordering#

folder_name = ''#

__getitem__(index)[source]#

Returns:: tuple of (data, target), where data is a dictionary of (events_left, events_right, imu) and targets is a dictionary of (images_left, images_right, mocap).

__len__()[source]#

class tonic.datasets.VPR(save_to, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, transforms: Optional[Callable] = None)[source]#

Bases: tonic.dataset.Dataset

Visual Place Recognition

Event-Based Visual Place Recognition With Ensembles of Temporal Windows. Events have (txyp) ordering.

Note

To be able to read this dataset and its GPS files, you will need the pynmea2 package installed.

@article{fischer2020event,
  title={Event-based visual place recognition with ensembles of temporal windows},
  author={Fischer, Tobias and Milford, Michael},
  journal={IEEE Robotics and Automation Letters},
  volume={5},
  number={4},
  pages={6924--6931},
  year={2020},
  publisher={IEEE}
}

Parameters:

save_to (string) – Location to save files to on disk.
transform (callable, optional) – A callable of transforms to apply to the data.
target_transform (callable, optional) – A callable of transforms to apply to the targets/labels.
transforms (callable, optional) – A callable of transforms that is applied to both data and labels at the same time.

base_url = 'https://zenodo.org/record/4302805/files/'#

recordings = [[['dvs_vpr_2020-04-21-17-03-03.bag', '04473f623aec6bda3d7eadfecfc1b2ce'],...#

sensor_size = (346, 260, 2)#

dtype#

ordering#

__getitem__(index)[source]#

Returns:: a tuple of (data, target) where data is another tuple of (events, imu, images) and target is gps positional data.

__len__()[source]#

download()[source]#: Downloads from a given url, places into target folder and verifies the file hash.

read_gps_file(nmea_file_path)[source]#

tonic.datasets

Contents

tonic.datasets#

Submodules#

Package Contents#

Classes#

`tonic.datasets`#