:py:mod:`tonic.functional` ========================== .. py:module:: tonic.functional Submodules ---------- .. toctree:: :titlesonly: :maxdepth: 1 crop/index.rst decimate/index.rst denoise/index.rst drop_event/index.rst drop_pixel/index.rst event_downsampling/index.rst refractory_period/index.rst spatial_jitter/index.rst time_jitter/index.rst time_skew/index.rst to_averaged_timesurface/index.rst to_bina_rep/index.rst to_frame/index.rst to_timesurface/index.rst to_voxel_grid/index.rst uniform_noise/index.rst Package Contents ---------------- Functions ~~~~~~~~~ .. autoapisummary:: tonic.functional.crop_numpy tonic.functional.decimate_numpy tonic.functional.denoise_numpy tonic.functional.drop_by_area_numpy tonic.functional.drop_by_time_numpy tonic.functional.drop_event_numpy tonic.functional.drop_pixel_numpy tonic.functional.drop_pixel_raster tonic.functional.identify_hot_pixel tonic.functional.identify_hot_pixel_raster tonic.functional.differentiator_downsample tonic.functional.integrator_downsample tonic.functional.refractory_period_numpy tonic.functional.spatial_jitter_numpy tonic.functional.time_jitter_numpy tonic.functional.time_skew_numpy tonic.functional.to_averaged_timesurface_numpy tonic.functional.to_bina_rep_numpy tonic.functional.to_frame_numpy tonic.functional.to_timesurface_numpy tonic.functional.to_voxel_grid_numpy tonic.functional.uniform_noise_numpy .. py:function:: crop_numpy(events, sensor_size, target_size) Crops the sensor size to a smaller sensor. x' = x - new_sensor_start_x y' = y - new_sensor_start_y :param events: ndarray of shape [num_events, num_event_channels] :param sensor_size: size of the sensor that was used [W,H] :param target_size: size of the sensor that was used [W',H'] :returns: events - events within the crop box sensor_size - cropped to target_size .. py:function:: decimate_numpy(events: numpy.ndarray, n: int) Returns 1/n events for each pixel location. :param events: structured numpy array of events :param n: filter rate .. py:function:: denoise_numpy(events, filter_time=10000) Drops events that are 'not sufficiently connected to other events in the recording.' In practise that means that an event is dropped if no other event occured within a spatial neighbourhood of 1 pixel and a temporal neighbourhood of filter_time time units. Useful to filter noisy recordings where events occur isolated in time. :param events: ndarray of shape [num_events, num_event_channels] :param filter_time: maximum temporal distance to next event, otherwise dropped. Lower values will mean higher constraints, therefore less events. :returns: filtered set of events. .. py:function:: drop_by_area_numpy(events: numpy.ndarray, sensor_size: tuple, area_ratio: float | tuple[float] = 0.2) Drops events located in a randomly chosen box area. The size of the box area is defined by a specified ratio of the sensor size. :param events: ndarray of shape [num_events, num_event_channels] :type events: np.ndarray :param sensor_size: size of the sensor that was used [W,H,P] :type sensor_size: Tuple :param area_ratio: Ratio of the sensor resolution that determines the size of the box area where events are dropped. - if a float, the value is used to calculate the size of the box area - if a tuple of 2 floats, the ratio is randomly chosen in [min, max) Defaults to 0.2. :type area_ratio: Union[float, Tuple[float]], optional :returns: augmented events that were not dropped (i.e., the events that are not located in the box area). :rtype: np.ndarray .. py:function:: drop_by_time_numpy(events: numpy.ndarray, duration_ratio: float | tuple[float] = 0.2) Drops events in a certain time interval with a length proportional to a specified ratio of the original length. :param events: ndarray of shape [num_events, num_event_channels] :type events: np.ndarray :param duration_ratio: the length of the dropped time interval, expressed in a ratio of the original sequence duration. - If a float, the value is used to calculate the interval length - If a tuple of 2 floats, the ratio is randomly chosen in [min, max). Defaults to 0.2. :type duration_ratio: Union[float, Tuple[float]], optional :returns: augmented events that were not dropped (i.e., the events that are not in the time interval). :rtype: np.ndarray .. py:function:: drop_event_numpy(events: numpy.ndarray, drop_probability: float) Randomly drops events with drop_probability. :param events: ndarray of shape [num_events, num_event_channels]. :param drop_probability: probability of dropping out event. :returns: augmented events that were not dropped. .. py:function:: drop_pixel_numpy(events: numpy.ndarray, coordinates) Drops events for pixel locations that fire. :param events: ndarray of shape [num_events, num_event_channels] :param coordinates: list of (x,y) coordinates for which all events will be deleted. :returns: subset of original events. .. py:function:: drop_pixel_raster(raster: numpy.ndarray, coordinates) Drops events for pixel locations. :param raster: ndarray of shape [p, h, w] or [t, p, h, w] :param coordinates: list of (x,y) coordinates for which all events will be deleted. :returns: The filtered raster or frame .. py:function:: identify_hot_pixel(events: numpy.ndarray, hot_pixel_frequency: float) Identifies pixels that fire above above a certain frequency, averaged across whole event recording. Such _hot_ pixels are sometimes caused by faulty hardware. :param events: ndarray of shape [num_events, num_event_channels] :param hot_pixel_frequency: number of spikes per pixel allowed for the recording, any pixel firing above that number will be deactivated. :returns: list of (x/y) coordinates for excessively firing pixels. .. py:function:: identify_hot_pixel_raster(events: numpy.ndarray, hot_pixel_frequency: float) Identifies pixels that fire above a certain predefined spike amount, supports both. :param events: ndarray of shape [P, H, W] or [T, P, H, W] :param hot_pixel_frequency: number of spikes per pixel allowed for the recording, any pixel firing above that number will be deactivated. :returns: list of (x/y) coordinates for excessively firing pixels. .. py:function:: differentiator_downsample(events: numpy.ndarray, sensor_size: tuple, target_size: tuple, dt: float, differentiator_time_bins: int = 2, noise_threshold: int = 0) Spatio-temporally downsample using the integrator method coupled with a differentiator to effectively downsample large object sizes relative to downsampled pixel resolution in the DVS camera's visual field. Incorporates the paper Ghosh et al. 2023, Insect-inspired Spatio-temporal Downsampling of Event-based Input, https://doi.org/10.1145/3589737.3605994 :param events: ndarray of shape [num_events, num_event_channels]. :type events: ndarray :param sensor_size: a 3-tuple of x,y,p for sensor_size. :type sensor_size: tuple :param target_size: a 2-tuple of x,y denoting new down-sampled size for events to be re-scaled to (new_width, new_height). :type target_size: tuple :param dt: step size for simulation, in ms. :type dt: float :param differentiator_time_bins: number of equally spaced time bins with respect to the dt to be used for the differentiator. :type differentiator_time_bins: int :param noise_threshold: number of events before a spike representing a new event is emitted. :type noise_threshold: int :returns: the spatio-temporally downsampled input events using the differentiator method. .. py:function:: integrator_downsample(events: numpy.ndarray, sensor_size: tuple, target_size: tuple, dt: float, noise_threshold: int = 0, differentiator_call: bool = False) Spatio-temporally downsample using with the following steps: 1. Differencing of ON and OFF events to counter camera shake or jerk. 2. Use an integrate-and-fire (I-F) neuron model with a noise threshold similar to the membrane potential threshold in the I-F model to eliminate high-frequency noise. Multiply x/y values by a spatial_factor obtained by dividing sensor size by the target size. :param events: ndarray of shape [num_events, num_event_channels]. :type events: ndarray :param sensor_size: a 3-tuple of x,y,p for sensor_size. :type sensor_size: tuple :param target_size: a 2-tuple of x,y denoting new down-sampled size for events to be re-scaled to (new_width, new_height). :type target_size: tuple :param dt: temporal resolution of events in milliseconds. :type dt: float :param noise_threshold: number of events before a spike representing a new event is emitted. :type noise_threshold: int :param differentiator_call: Preserve frame spikes for differentiator method in order to optimise differentiator method. :type differentiator_call: bool :returns: the spatio-temporally downsampled input events using the integrator method. .. py:function:: refractory_period_numpy(events: numpy.ndarray, refractory_period: float) Sets a refractory period for each pixel, during which events will be ignored/discarded. We keep events if: .. math:: t_n - t_{n-1} > t_{refrac} :param events: ndarray of shape [num_events, num_event_channels] :param refractory_period: refractory period for each pixel in microseconds :returns: filtered set of events. .. py:function:: spatial_jitter_numpy(events: numpy.ndarray, sensor_size: list[int], var_x: float = 1, var_y: float = 1, sigma_xy: float = 0, clip_outliers: bool = False) Changes x/y coordinate for each event by adding samples from a multivariate Gaussian distribution. It with the following properties: .. math:: mean = [x,y] \Sigma = [[var_x, sigma_{xy}],[sigma_{xy}, var_y]] Jittered events that lie outside the focal plane will be dropped if clip_outliers is True. :param events: ndarray of shape [num_events, num_event_channels] :param var_x: squared sigma value for the distribution in the x direction :param var_y: squared sigma value for the distribution in the y direction :param sigma_xy: changes skewness of distribution, only change if you want shifts along diagonal axis. :param clip_outliers: when True, events that have been jittered outside the sensor size will be dropped. :returns: array of spatially jittered events. .. py:function:: time_jitter_numpy(events: numpy.ndarray, std: float = 1, clip_negative: bool = False, sort_timestamps: bool = False) Changes timestamp for each event by drawing samples from a Gaussian distribution and adding them to each timestamp. :param events: ndarray of shape [num_events, num_event_channels] :param std: the standard deviation of the time jitter :param clip_negative: drops events that have negative timestamps :param sort_timestamps: sort the events by timestamps after jittering :returns: temporally jittered set of events. .. py:function:: time_skew_numpy(events: numpy.ndarray, coefficient: float, offset: int = 0) Skew all event timestamps according to a linear transform, potentially sampled from a distribution of acceptable functions. :param events: ndarray of shape [num_events, num_event_channels]. :param coefficient: a real-valued multiplier applied to the timestamps of the events. E.g. a coefficient of 2.0 will double the effective delay between any pair of events. :param offset: value by which the timestamps will be shifted after multiplication by the coefficient. Negative offsets are permissible but may result in in an exception if timestamps are shifted below 0. :returns: the input events with rewritten timestamps. .. py:function:: to_averaged_timesurface_numpy(events, sensor_size, cell_size, surface_size, time_window, tau, decay) Representation that creates averaged timesurfaces for each event for one recording. Taken from the paper Sironi et al. 2018, HATS: Histograms of averaged time surfaces for robust event-based object classification https://openaccess.thecvf.com/content_cvpr_2018/papers/Sironi_HATS_Histograms_of_CVPR_2018_paper.pdf :param cell_size: size of each square in the grid :type cell_size: int :param surface_size: has to be odd :type surface_size: int :param time_window: how far back to look for past events for the time averaging. Expressed in microseconds. :type time_window: int :param tau: time constant to decay events around occuring event with. Expressed in microseconds. :type tau: int :param decay: can be either 'lin' or 'exp', corresponding to linear or exponential decay. :type decay: str :returns: array of histograms (numpy.Array with shape (n_cells, n_pols, surface_size, surface_size)) .. py:function:: to_bina_rep_numpy(event_frames: numpy.ndarray, n_frames: int = 1, n_bits: int = 8) Representation that takes T*B binary event frames to produce a sequence of T frames of N-bit numbers. To do so, N binary frames are interpreted as a single frame of N-bit representation. Taken from the paper Barchid et al. 2022, Bina-Rep Event Frames: a Simple and Effective Representation for Event-based cameras https://arxiv.org/pdf/2202.13662.pdf. :param event_frames: numpy.ndarray of shape (T*BxPxHxW). The sequence of event frames. :param n_frames: the number T of bina-rep frames. :type n_frames: int :param n_bits: the number N of bits used in the N-bit representation. :type n_bits: int :returns: (numpy.ndarray) the sequence of bina-rep event frames with dimensions (TxPxHxW). .. py:function:: to_frame_numpy(events, sensor_size, time_window=None, event_count=None, n_time_bins=None, n_event_bins=None, overlap=0.0, include_incomplete=False, start_time=None, end_time=None) Accumulate events to frames by slicing along constant time (time_window), constant number of events (event_count) or constant number of frames (n_time_bins / n_event_bins). :param events: ndarray of shape [num_events, num_event_channels] :param sensor_size: size of the sensor that was used [W,H,P] :param time_window: window length in us. :type time_window: None :param event_count: number of events per frame. :type event_count: None :param n_time_bins: fixed number of frames, sliced along time axis. :type n_time_bins: None :param n_event_bins: fixed number of frames, sliced along number of events in the recording. :type n_event_bins: None :param overlap: overlap between frames defined either in time in us, number of events or number of bins. :type overlap: 0. :param include_incomplete: if True, includes overhang slice when time_window or event_count is specified. Not valid for bin_count methods. :type include_incomplete: False :returns: numpy array with dimensions (TxPxHxW) .. py:function:: to_timesurface_numpy(events, sensor_size: tuple[int, int, int], dt: float, tau: float, overlap: int = 0, include_incomplete: bool = False) Representation that creates timesurfaces for each event in the recording. Modeled after the paper Lagorce et al. 2016, Hots: a hierarchy of event-based time-surfaces for pattern recognition https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7508476. Unlike the paper, surfaces are always generated across the whole sensor, not just around the event. :param sensor_size: x/y/p dimensions of the sensor :param dt: time interval at which the time-surfaces are accumulated :param tau: time constant to decay events around occuring event with. :type tau: float :returns: array of timesurfaces with dimensions (n_events//dt, p, h , w) .. py:function:: to_voxel_grid_numpy(events, sensor_size, n_time_bins=10) Build a voxel grid with bilinear interpolation in the time domain from a set of events. Implements the event volume from Zhu et al. 2019, Unsupervised event-based learning of optical flow, depth, and egomotion. :param events: ndarray of shape [num_events, num_event_channels] :param sensor_size: size of the sensor that was used [W,H]. :param n_time_bins: number of bins in the temporal axis of the voxel grid. :returns: numpy array of n event volumes (n,w,h,t) .. py:function:: uniform_noise_numpy(events: numpy.ndarray, sensor_size: tuple[int, int, int], n: int) Adds a fixed number of noise events that are uniformly distributed across sensor size dimensions. :param events: ndarray of shape (n_events, n_event_channels) :param sensor_size: 3-tuple of integers for x, y, p :param n: the number of noise events added.