:py:mod:`tonic.datasets` ======================== .. py:module:: tonic.datasets Submodules ---------- .. toctree:: :titlesonly: :maxdepth: 1 asl_dvs/index.rst cifar10dvs/index.rst davisdataset/index.rst dsec/index.rst dvs_lips/index.rst dvsgesture/index.rst ebssa/index.rst hsd/index.rst mvsec/index.rst ncaltech101/index.rst nerdd/index.rst nmnist/index.rst ntidigits18/index.rst pokerdvs/index.rst pokerdvs131/index.rst s_mnist/index.rst threeET_eyetracking/index.rst tum_vie/index.rst visual_place_recognition/index.rst Package Contents ---------------- Classes ~~~~~~~ .. autoapisummary:: tonic.datasets.ASLDVS tonic.datasets.CIFAR10DVS tonic.datasets.DAVISDATA tonic.datasets.DSEC tonic.datasets.DVSLip tonic.datasets.DVSGesture tonic.datasets.EBSSA tonic.datasets.SHD tonic.datasets.SSC tonic.datasets.MVSEC tonic.datasets.NCALTECH101 tonic.datasets.NERDD tonic.datasets.NMNIST tonic.datasets.NTIDIGITS18 tonic.datasets.POKERDVS tonic.datasets.SMNIST tonic.datasets.ThreeET_Eyetracking tonic.datasets.TUMVIE tonic.datasets.VPR .. py:class:: ASLDVS(save_to: str, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `ASL-DVS `_ :: @inproceedings{bi2019graph, title={Graph-based Object Classification for Neuromorphic Vision Sensing}, author={Bi, Y and Chadha, A and Abbas, A and and Bourtsoulatze, E and Andreopoulos, Y}, booktitle={2019 IEEE International Conference on Computer Vision (ICCV)}, year={2019}, organization={IEEE} } :param save_to: Location to save files to on disk. :type save_to: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: url :value: 'https://www.dropbox.com/sh/ibq0jsicatn7l6r/AACNrNELV56rs1YInMWUs9CAa?dl=1' .. py:attribute:: filename :value: 'ASLDVS.zip' .. py:attribute:: file_md5 :value: '33f8b87bf45edc0bfed0de41822279b9' .. py:attribute:: folder_name :value: '' .. py:attribute:: classes .. py:attribute:: int_classes .. py:attribute:: sensor_size :value: (240, 180, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index: int) -> tuple[Any, Any] :returns: (events, target) where target is index of the target class. .. py:method:: __len__() .. py:class:: CIFAR10DVS(save_to: str, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `CIFAR10-DVS `_ :: @article{li2017cifar10, title={Cifar10-dvs: an event-stream dataset for object classification}, author={Li, Hongmin and Liu, Hanchao and Ji, Xiangyang and Li, Guoqi and Shi, Luping}, journal={Frontiers in neuroscience}, volume={11}, pages={309}, year={2017}, publisher={Frontiers} } :param save_to: Location to save files to on disk. :type save_to: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: url :value: 'https://figshare.com/ndownloader/files/38023437' .. py:attribute:: filename :value: 'CIFAR10DVS.zip' .. py:attribute:: file_md5 :value: 'ce3a4a0682dc0943703bd8f749a7701c' .. py:attribute:: data_filename :value: ['airplane.zip', 'automobile.zip', 'bird.zip', 'cat.zip', 'deer.zip', 'dog.zip', 'frog.zip',... .. py:attribute:: folder_name :value: '' .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: sensor_size :value: (128, 128, 2) .. py:attribute:: classes .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() .. py:class:: DAVISDATA(save_to: str, recording: str | list[str], transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `DAVIS event camera dataset `_ :: @article{mueggler2017event, title={The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM}, author={Mueggler, Elias and Rebecq, Henri and Gallego, Guillermo and Delbruck, Tobi and Scaramuzza, Davide}, journal={The International Journal of Robotics Research}, volume={36}, number={2}, pages={142--149}, year={2017}, publisher={SAGE Publications Sage UK: London, England} } :param save_to: Location to save files to on disk. Will save files in a sub folder 'davis_dataset'. :type save_to: string :param recording: Use the name of the recording or a list thereof to download it, for example 'dynamic_6dof' or ['slider_far', 'urban']. See project homepage for a list of available recordings. Can use 'all' to download all available recordings. :type recording: string :param transform: A callable of transforms to apply to events and/or images. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'http://rpg.ifi.uzh.ch/datasets/davis/' .. py:attribute:: recordings .. py:attribute:: sensor_size :value: (240, 180, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: folder_name :value: '' .. py:method:: __getitem__(index) :returns: tuple of (data, target), where data is another tuple of (events, imu, images) and target is the opti track ground truth .. py:method:: __len__() .. py:method:: download() Downloads from a given url, places into target folder and verifies the file hash. .. py:class:: DSEC(save_to: str, split: str | list[str], data_selection: str | list[str], target_selection: str | list[str] | None = None, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `DSEC `_ This is a fairly large dataset, so in order to save some disk space, event and image zips are deleted after extraction. If your download gets interrupted and you are left with a corrupted file on disk, Tonic will not be able to detect that and just proceed to download files that are not yet on disk. If you experience issues loading a particular recording, delete that folder manually and Tonic will re-download it the next time. Optical flow targets are not available for every recording, so if you select optical flow targets, only a subset of 18 training recordings will be selected. .. note:: To be able to read this dataset, you will need `hdf5plugin`, `PIL` and `imageio` packages installed. :param save_to: Location to save files to on disk. :type save_to: str :param split: Can be 'train', 'test' or a selection of individual recordings such as 'interlaken_00_c' or ['thun_00_a', 'zurich_city_00_a']. Cannot mix across train/test. :type split: str :param data_selection: Select which data to load per sample. Can be 'events_left', 'events_right', 'images_rectified_left', 'images_rectified_right', 'image_timestamps' or any combination thereof in a list. :type data_selection: str :param target_selection: Select which targets to load per sample. Can be 'disparity_events', 'disparity_images', 'disparity_timestamps', 'optical_flow_forward_event', 'optical_flow_forward_timestamps', 'optical_flow_backward_event', 'optical_flow_backward_timestamps' or a combination thereof in a list. Note that optical flow targets are not available for every recording. :type target_selection: str, optional :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'https://download.ifi.uzh.ch/rpg/DSEC/' .. py:attribute:: recordings .. py:attribute:: data_names .. py:attribute:: target_names .. py:attribute:: sensor_size :value: (640, 480, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (data, target) where data is another tuple of data_selction and target a tuple of target_selection if train=True. .. py:method:: __len__() .. py:class:: DVSLip(save_to: str, train: bool = True, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `DVS-Lip `_ :: @inproceedings{tan2022multi, title={Multi-Grained Spatio-Temporal Features Perceived Network for Event-Based Lip-Reading}, author={Tan, Ganchao and Wang, Yang and Han, Han and Cao, Yang and Wu, Feng and Zha, Zheng-Jun}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={20094--20103}, year={2022} } Implementation inspired from original script: https://github.com/tgc1997/event-based-lip-reading/blob/main/utils/dataset.py :param save_to: Location to save files to on disk. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'https://drive.google.com/file/d/1dBEgtmctTTWJlWnuWxFtk8gfOdVVpkQ0/view' .. py:attribute:: filename :value: 'DVS-Lip.zip' .. py:attribute:: base_folder :value: 'DVS-Lip' .. py:attribute:: file_md5 :value: '2dcb959255122d4cdeb6094ca282494b' .. py:attribute:: sensor_size :value: (128, 128, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: classes :value: ['accused', 'action', 'allow', 'allowed', 'america', 'american', 'another', 'around', 'attacks',... .. py:attribute:: ambiguous_classes :value: ['action', 'allow', 'allowed', 'america', 'american', 'around', 'being', 'benefit', 'benefits',... .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() .. py:class:: DVSGesture(save_to: str, train: bool = True, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `IBM DVS Gestures `_ .. note:: This is (exceptionally) a preprocessed version of the original dataset, where recordings that originally contained multiple labels have already been cut into respective samples. Also temporal precision is reduced to ms. :: @inproceedings{amir2017low, title={A low power, fully event-based gesture recognition system}, author={Amir, Arnon and Taba, Brian and Berg, David and Melano, Timothy and McKinstry, Jeffrey and Di Nolfo, Carmelo and Nayak, Tapan and Andreopoulos, Alexander and Garreau, Guillaume and Mendoza, Marcela and others}, booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, pages={7243--7252}, year={2017} } :param save_to: Location to save files to on disk. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: test_url :value: 'https://figshare.com/ndownloader/files/38020584' .. py:attribute:: train_url :value: 'https://figshare.com/ndownloader/files/38022171' .. py:attribute:: test_md5 :value: '56070e45dadaa85fff82e0fbfbc06de5' .. py:attribute:: train_md5 :value: '3a8f0d4120a166bac7591f77409cb105' .. py:attribute:: test_filename :value: 'ibmGestureTest.tar.gz' .. py:attribute:: train_filename :value: 'ibmGestureTrain.tar.gz' .. py:attribute:: classes :value: ['Hand clapping', 'Right hand wave', 'Left hand wave', 'Right arm cw', 'Right arm ccw', 'Left... .. py:attribute:: sensor_size :value: (128, 128, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() .. py:class:: EBSSA(save_to: str, split: str = 'labelled', transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `EBSSA `_ There are six different splits provided in this dataset. The labelled section of the dataset contains 84 recordings and 84 label files. The unlabelled section of the dataset contains 153 recordings in folders marked "Unlabelled". If the automatic download from Google Drive fails, please download the file manually from https://drive.google.com/uc?id=1lCh2HWvxEzzaBHT5TlPuyUn6XPM5OVWN and put it in a folder called 'EBSSA'. Then point the path to its parent folder. For example, if you put the data file in 'datasets/EBSSA/labelled_ebssa.h5', create the dataset object with 'dataset = tonic.datasets.EBSSA('datasets', split='labelled')' :: @article{afshar2020event, title={Event-based object detection and tracking for space situational awareness}, author={Afshar, Saeed and Nicholson, Andrew Peter and Van Schaik, Andre and Cohen, Gregory}, journal={IEEE Sensors Journal}, volume={20}, number={24}, pages={15117--15132}, year={2020}, publisher={IEEE} } :param save_to: Location to save files to on disk. :type save_to: string :param split: Which split to load. One of "labelled", "unlabelled", "all". :type split: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: file_id :value: '1lCh2HWvxEzzaBHT5TlPuyUn6XPM5OVWN' .. py:attribute:: folder_name :value: '' .. py:attribute:: file_name :value: 'labelled_ebssa.h5' .. py:attribute:: sensor_size :value: (240, 180, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index: int) -> tuple[Any, Any] :returns: (events, target) where target is dict of bounding box and recording id. .. py:method:: __len__() .. py:method:: download() Downloads from a given url, places into target folder and verifies the file hash. .. py:class:: SHD(save_to: str, train: bool = True, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None) Bases: :py:obj:`HSD` `Spiking Heidelberg Digits `_ :: @article{cramer2020heidelberg, title={The heidelberg spiking data sets for the systematic evaluation of spiking neural networks}, author={Cramer, Benjamin and Stradmann, Yannik and Schemmel, Johannes and Zenke, Friedemann}, journal={IEEE Transactions on Neural Networks and Learning Systems}, year={2020}, publisher={IEEE} } :param save_to: Location to save files to on disk. Will put files in an 'hsd' subfolder. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :returns: A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets). .. py:property:: speaker .. py:attribute:: test_zip :value: 'shd_test.h5.zip' .. py:attribute:: train_zip :value: 'shd_train.h5.zip' .. py:attribute:: test_md5 :value: '1503a5064faa34311c398fb0a1ed0a6f' .. py:attribute:: train_md5 :value: 'f3252aeb598ac776c1b526422d90eecb' .. py:attribute:: folder_name :value: '' .. py:class:: SSC(save_to: str, split: str = 'train', transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None) Bases: :py:obj:`HSD` `Spiking Speech Commands `_ :: @article{cramer2020heidelberg, title={The heidelberg spiking data sets for the systematic evaluation of spiking neural networks}, author={Cramer, Benjamin and Stradmann, Yannik and Schemmel, Johannes and Zenke, Friedemann}, journal={IEEE Transactions on Neural Networks and Learning Systems}, year={2020}, publisher={IEEE} } :param save_to: Location to save files to on disk. Will put files in an 'hsd' subfolder. :type save_to: string :param split: One of 'train', 'test' or 'valid'. :type split: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :returns: A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets). .. py:attribute:: test_zip :value: 'ssc_test.h5.zip' .. py:attribute:: train_zip :value: 'ssc_train.h5.zip' .. py:attribute:: valid_zip :value: 'ssc_valid.h5.zip' .. py:attribute:: test_md5 :value: 'a35ff1e9cffdd02a20eb850c17c37748' .. py:attribute:: train_md5 :value: 'd102be95e7144fcc0553d1f45ba94170' .. py:attribute:: valid_md5 :value: 'b4eee3516a4a90dd0c71a6ac23a8ae43' .. py:attribute:: folder_name :value: '' .. py:class:: MVSEC(save_to: str, scene: str, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `MVSEC `_ :: @article{zihao2018multi, title={The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception}, author={Zihao Zhu, Alex and Thakur, Dinesh and Ozaslan, Tolga and Pfrommer, Bernd and Kumar, Vijay and Daniilidis, Kostas}, journal={arXiv e-prints}, pages={arXiv--1801}, year={2018} } :param save_to: Location to save files to on disk. :type save_to: string :param scene: Choose one of 4 scenes: outdoor_night, outdoor_day, indoor_flying, motorcycle. If you already have the data on your system, make sure to place the .bag files in a subfolder 'MVSEC/{scene}/bag_files.bag'. :type scene: string :param transform: A callable of transforms to apply to events and / or images for both left and right cameras. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: resources .. py:attribute:: base_url :value: 'http://visiondata.cis.upenn.edu/mvsec/' .. py:attribute:: sensor_size :value: (346, 260, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: tuple of (data, targets), where data is another tuple of (events_left, events_right, imu_left, imu_right, images_left, images_right) and targets is a tuple of (depth_rect_left, depth_rect_right, pose) for ground truths. .. py:method:: __len__() .. py:method:: download() Downloads from a given url, places into target folder and verifies the file hash. .. py:class:: NCALTECH101(save_to: str, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `N-CALTECH101 `_ Events have (xytp) ordering. :: @article{orchard2015converting, title={Converting static image datasets to spiking neuromorphic datasets using saccades}, author={Orchard, Garrick and Jayawant, Ajinkya and Cohen, Gregory K and Thakor, Nitish}, journal={Frontiers in neuroscience}, volume={9}, pages={437}, year={2015}, publisher={Frontiers} } :param save_to: Location to save files to on disk. :type save_to: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: url :value: 'https://data.mendeley.com/public-files/datasets/cy6cvx3ryv/files/36b5c52a-b49d-4853-addb-a836a888...' .. py:attribute:: filename :value: 'N-Caltech101-archive.zip' .. py:attribute:: file_md5 :value: '66201824eabb0239c7ab992480b50ba3' .. py:attribute:: data_filename :value: 'Caltech101.zip' .. py:attribute:: folder_name :value: 'Caltech101' .. py:attribute:: sensor_size .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() .. py:class:: NERDD(root: str, transforms: collections.abc.Callable | None = None) `NeRDD `_ Neuromorphic Event-based Reactive Driving Dataset. :param root: Location to save files to on disk. :type root: string :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: filename :value: 'NeRDD.zip' .. py:attribute:: folder_name :value: 'DATA' .. py:attribute:: sensor_size :value: (1280, 720, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index: int) -> tuple[Any, Any] :returns: (events, target) where target is index of the target class. .. py:method:: __len__() .. py:class:: NMNIST(save_to: str, train: bool = True, first_saccade_only: bool = False, stabilize: bool = False, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `N-MNIST `_ Events have (xytp) ordering. :: @article{orchard2015converting, title={Converting static image datasets to spiking neuromorphic datasets using saccades}, author={Orchard, Garrick and Jayawant, Ajinkya and Cohen, Gregory K and Thakor, Nitish}, journal={Frontiers in neuroscience}, volume={9}, pages={437}, year={2015}, publisher={Frontiers} } :param save_to: Location to save files to on disk. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param first_saccade_only: If True, only work with events of the first of three saccades. Results in about a third of the events overall. :type first_saccade_only: bool :param stabilize: If True, it stabilizes egomotion of the saccades, centering the digit. :type stabilize: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'https://data.mendeley.com/public-files/datasets/468j46mzdv/files/' .. py:attribute:: train_url .. py:attribute:: train_filename :value: 'train.zip' .. py:attribute:: train_md5 :value: '20959b8e626244a1b502305a9e6e2031' .. py:attribute:: train_folder :value: 'Train' .. py:attribute:: test_url .. py:attribute:: test_filename :value: 'test.zip' .. py:attribute:: test_md5 :value: '69ca8762b2fe404d9b9bad1103e97832' .. py:attribute:: test_folder :value: 'Test' .. py:attribute:: classes :value: ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four', '5 - five', '6 - six', '7 - seven',... .. py:attribute:: sensor_size :value: (34, 34, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() -> int .. py:class:: NTIDIGITS18(save_to: str, train: bool = True, single_digits=False, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `N-TIDIGITS18 Dataset `_ Cochlea Spike Dataset. :: @article{anumula2018feature, title={Feature representations for neuromorphic audio spike streams}, author={Anumula, Jithendar and Neil, Daniel and Delbruck, Tobi and Liu, Shih-Chii}, journal={Frontiers in neuroscience}, volume={12}, pages={23}, year={2018}, publisher={Frontiers Media SA} } :param save_to: Location to save files to on disk. Will put files in an 'hsd' subfolder. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param single_digits: If True, only returns samples with single digits (o, 1, 2, 3, 4, 5, 6, 7, 8, 9, z), with class 0 for 'o' and 11 for 'z'. :type single_digits: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :returns: A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets). .. py:attribute:: base_url :value: 'https://www.dropbox.com/scl/fi/1x4lxt9yyw25sc3tez8oi/n-tidigits.hdf5?e=2&rlkey=w8gi5udvib2zqzosus...' .. py:attribute:: filename :value: 'n-tidigits.hdf5' .. py:attribute:: file_md5 :value: '360a2d11e5429555c9197381cf6b58e0' .. py:attribute:: folder_name :value: '' .. py:attribute:: sensor_size :value: (64, 1, 1) .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: class_map .. py:method:: download() -> None Downloads from a given url, places into target folder and verifies the file hash. .. py:method:: __getitem__(index) .. py:method:: __len__() .. py:class:: POKERDVS(save_to: str, train: bool = True, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `POKER-DVS `_ Events have (txyp) ordering. :: @article{serrano2015poker, title={Poker-DVS and MNIST-DVS. Their history, how they were made, and other details}, author={Serrano-Gotarredona, Teresa and Linares-Barranco, Bernab{'e}}, journal={Frontiers in neuroscience}, volume={9}, pages={481}, year={2015}, publisher={Frontiers} } :param save_to: Location to save files to on disk. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: train_filename :value: 'pips_train.tar.gz' .. py:attribute:: test_filename :value: 'pips_test.tar.gz' .. py:attribute:: train_md5 :value: '412bcfb96826e4fcb290558e8c150aae' .. py:attribute:: test_md5 :value: 'eef2bf7d0d3defae89a6fa98b07c17af' .. py:attribute:: classes :value: ['cl', 'he', 'di', 'sp'] .. py:attribute:: int_classes .. py:attribute:: sensor_size :value: (35, 35, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (events, target) where target is the index of the target class. .. py:method:: __len__() .. py:class:: SMNIST(save_to, train=True, duplicate=True, num_neurons=99, dt=1000.0, transform=None, target_transform=None) Bases: :py:obj:`tonic.dataset.Dataset` Spiking sequential `MNIST `_ Sequential MNIST (sMNIST) is a standard benchmark task for time series classification where each input consists of sequences of 784 pixel values created by unrolling the MNIST digits, pixel by pixel. In this spiking version, each of the 99 input neurons is associated with a particular threshold for the grey value, and this input neuron fires whenever the grey value crosses its threshold in the transition from the previous to the current pixel. :param save_to: Location to save files to on disk. :type save_to: string :param train: If True, uses training subset, otherwise testing subset. :type train: bool :param duplicate: If True, emits two spikes per threshold crossing :type duplicate: bool :param num_neurons: How many neurons to use to encode thresholds(must be odd) :type num_neurons: integer :param dt: Duration(in microseconds) of each timestep :type dt: float :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional Returns: A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets). .. py:attribute:: base_url :value: 'https://storage.googleapis.com/cvdf-datasets/mnist/' .. py:attribute:: train_images_file :value: 'train-images-idx3-ubyte' .. py:attribute:: train_labels_file :value: 'train-labels-idx1-ubyte' .. py:attribute:: test_images_file :value: 't10k-images-idx3-ubyte' .. py:attribute:: test_labels_file :value: 't10k-labels-idx1-ubyte' .. py:attribute:: train_images_md5 :value: 'f68b3c2dcbeaaa9fbdd348bbdeb94873' .. py:attribute:: train_labels_md5 :value: 'd53e105ee54ea40749a09fcbcd1e9432' .. py:attribute:: test_images_md5 :value: '9fb629c4189551a2d022fa330f9573f3' .. py:attribute:: test_labels_md5 :value: 'ec29112dd5afa0611ce80d1b7f02629c' .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: classes :value: ['0 - zero', '1 - one', '2 - two', '3 - three', '4 - four', '5 - five', '6 - six', '7 - seven',... .. py:method:: __getitem__(index) .. py:method:: __len__() .. py:method:: download() Downloads from a given url, places into target folder and verifies the file hash. .. py:class:: ThreeET_Eyetracking(save_to: str, split: str = 'train', transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` 3ET DVS eye tracking `3ET `_ :: @article{chen20233et, title={3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network}, author={Chen, Qinyu and Wang, Zuowen and Liu, Shih-Chii and Gao, Chang}, journal={arXiv preprint arXiv:2308.11771}, year={2023} } :param save_to: Location to save files to on disk. :type save_to: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param split: The dataset split to use, ``train`` or ``val``. :type split: string, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional :returns: A dataset object that can be indexed or iterated over. One sample returns a tuple of (events, targets). .. py:attribute:: url :value: 'https://dl.dropboxusercontent.com/s/1hyer8egd8843t9/ThreeET_Eyetracking.zip?dl=0' .. py:attribute:: filename :value: 'ThreeET_Eyetracking.zip' .. py:attribute:: file_md5 :value: 'b6c652b06fdfd85721f39e2dbe12f4e8' .. py:attribute:: sensor_size :value: (240, 180, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index: int) -> tuple[Any, Any] :returns: (events, target) where target is index of the target class. .. py:method:: __len__() .. py:method:: load_filenames(path) .. py:class:: TUMVIE(save_to: str, recording: str | list[str], transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `TUM-VIE `_ TUM-VIE is an event camera dataset for developing 3D perception and navigation algorithms. It contains handheld and head-mounted sequences in indoor and outdoor environments with rapid motion during sports and high dynamic range. TUM-VIE includes challenging sequences where state-of-the art VIO fails or results in large drift. Hence, it can help to push the boundary on event-based visual-inertial algorithms. The dataset contains: * Stereo event data Prophesee Gen4 HD (1280x720 pixels) * Stereo grayscale frames at 20Hz (1024x1024 pixels) * IMU data at 200Hz * 6dof motion capture data at 120Hz (beginning and end of each sequence) Timestamps between all sensors are synchronized in hardware. .. note:: To be able to read this dataset, you will need `hdf5plugin` and `PIL` packages installed. .. note:: Use `bike-night` instead of `bike-dark` in the list of recordings if you want that specific one. :: @string{iros="International Conference on Intelligent Robots and Systems (IROS)"} @inproceedings{klenk2021tumvie, author = {S Klenk and J Chui and N Demmel and D Cremers}, title = {TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset}, eprint = {2108.07329}, eprinttype = {arXiv}, eprintclass = {cs.CV}, booktitle = {International Conference on Intelligent Robots and Systems (IROS)}, year = {2021}, keywords = {tumvie, event camera, dynamic vision sensor, SLAM, vslam}, } :param save_to: Location to save files to on disk. Will save files in a sub folder 'davis_dataset'. :type save_to: string :param recording: Use the name of the recording or a list thereof to download it, for example 'skate-hard' or ['skate-hard', 'bike-easy']. See project homepage for a list of available recordings. Can use 'all' to download all available recordings. :type recording: string :param transform: A callable of transforms to apply to events and/or images. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'https://tumevent-vi.vision.in.tum.de/' .. py:attribute:: recordings :value: ['mocap-1d-trans', 'mocap-3d-trans', 'mocap-6dof', 'mocap-desk', 'mocap-desk2', 'mocap-shake',... .. py:attribute:: filenames :value: ['events_left.h5', 'events_right.h5', 'vi_gt_data.tar.gz'] .. py:attribute:: sensor_size :value: (1280, 720, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:attribute:: folder_name :value: '' .. py:method:: __getitem__(index) :returns: tuple of (data, target), where data is a dictionary of (events_left, events_right, imu) and targets is a dictionary of (images_left, images_right, mocap). .. py:method:: __len__() .. py:class:: VPR(save_to, transform: collections.abc.Callable | None = None, target_transform: collections.abc.Callable | None = None, transforms: collections.abc.Callable | None = None) Bases: :py:obj:`tonic.dataset.Dataset` `Visual Place Recognition `_ Event-Based Visual Place Recognition With Ensembles of Temporal Windows. Events have (txyp) ordering. .. note:: To be able to read this dataset and its GPS files, you will need the `pynmea2` package installed. :: @article{fischer2020event, title={Event-based visual place recognition with ensembles of temporal windows}, author={Fischer, Tobias and Milford, Michael}, journal={IEEE Robotics and Automation Letters}, volume={5}, number={4}, pages={6924--6931}, year={2020}, publisher={IEEE} } :param save_to: Location to save files to on disk. :type save_to: string :param transform: A callable of transforms to apply to the data. :type transform: callable, optional :param target_transform: A callable of transforms to apply to the targets/labels. :type target_transform: callable, optional :param transforms: A callable of transforms that is applied to both data and labels at the same time. :type transforms: callable, optional .. py:attribute:: base_url :value: 'https://zenodo.org/record/4302805/files/' .. py:attribute:: recordings :value: [[['dvs_vpr_2020-04-21-17-03-03.bag', '04473f623aec6bda3d7eadfecfc1b2ce'],... .. py:attribute:: sensor_size :value: (346, 260, 2) .. py:attribute:: dtype .. py:attribute:: ordering .. py:method:: __getitem__(index) :returns: a tuple of (data, target) where data is another tuple of (events, imu, images) and target is gps positional data. .. py:method:: __len__() .. py:method:: download() Downloads from a given url, places into target folder and verifies the file hash. .. py:method:: read_gps_file(nmea_file_path)