ToFrame#

class tonic.transforms.ToFrame(sensor_size: Optional[Tuple[int, int, int]], time_window: Optional[float] = None, event_count: Optional[int] = None, n_time_bins: Optional[int] = None, n_event_bins: Optional[int] = None, overlap: float = 0, include_incomplete: bool = False)[source]#

Accumulate events to frames by slicing along constant time (time_window), constant number of events (event_count) or constant number of frames (n_time_bins / n_event_bins). All the events in one slice are added up in a frame for each polarity. If you want binary frames, you can manually clamp them to 1 afterwards. You can set one of the first 4 parameters to choose the slicing method. Depending on which method you choose, overlap will be defined differently. As a rule of thumb, here are some considerations if you are unsure which slicing method to choose:

  • If your recordings are of roughly the same length, a safe option is to set time_window. Bare in mind that the number of events can vary greatly from slice to slice, but will give you some consistency when training RNNs or other algorithms that have time steps.

  • If your recordings have roughly the same amount of activity / number of events and you are more interested in the spatial composition, then setting event_count will give you frames that are visually more consistent.

  • The previous time_window and event_count methods will likely result in a different amount of frames for each recording. If your training method benefits from consistent number of frames across a dataset (for easier batching for example), or you want a parameter that is easier to set than the exact window length or number of events per slice, consider fixing the number of frames by setting n_time_bins or n_event_bins. The two methods slightly differ with respect to how the slices are distributed across the recording. You can define an overlap between 0 and 1 to provide some robustness.

Parameters:

  • sensor_size (Optional[Tuple[int, int, int]]) – A 3-tuple of x,y,p for sensor_size. If omitted, the sensor size is calculated for that sample. However, do use this feature sparingly as when not all pixels fire in a sample, this might cause issues with batching/ stacking tensors further down the line.

  • time_window (float) – Time window length for one frame. Use the same time unit as timestamps in the event recordings. Good if you want temporal consistency in your training, bad if you need some visual consistency for every frame if the recording’s activity is not consistent.

  • event_count (int) – Number of events per frame. Good for training CNNs which do not care about temporal consistency.

  • n_time_bins (int) – Fixed number of frames, sliced along time axis. Good for generating a pre-determined number of frames which might help with batching.

  • n_event_bins (int) – Fixed number of frames, sliced along number of events in the recording. Good for generating a pre-determined number of frames which might help with batching.

  • overlap (float) – Overlap between frames. The definition of overlap depends on the slicing method. For slicing by time_window, the overlap is defined in microseconds. For slicing by event_count, the overlap is defined by number of events. For slicing by n_time_bins or n_event_bins, the overlap is defined by the fraction of a bin between 0 and 1.

  • include_incomplete (bool) – If True, includes overhang slice when time_window or event_count is specified. Not valid for bin_count methods.

Example

>>> from tonic.transforms import ToFrame
>>> transform1 = ToFrame(time_window=10000, overlap=1000, include_incomplete=True)
>>> transform2 = ToFrame(event_count=3000, overlap=100, include_incomplete=True)
>>> transform3 = ToFrame(n_time_bins=100, overlap=0.1)