Skip to content

a proposal to add a method to iterators for producing an iterator of its subsequences

Notifications You must be signed in to change notification settings

bakkot/proposal-iterator-chunking

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Iterator Chunking

A TC39 proposal to add a method to iterators for producing an iterator of its subsequences.

Stage: 0

See the January 2024 presentation to committee.

motivation

It can be useful to consume a stream by more than one value at a time. For example, certain algorithms require looking at adjacent elements.

A common solution for this is a chunks method that works like the following:

let digits = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].values();

let chunksOf2 = Array.from(digits.chunks(2));
// [ [0, 1], [2, 3], [4, 5], [6, 7], [8, 9] ]

let chunksOf3 = Array.from(digits.chunks(3));
// [ [0, 1, 2], [3, 4, 5], [6, 7, 8], [9] ]

let chunksOf4 = Array.from(digits.chunks(4));
// [ [0, 1, 2, 3], [4, 5, 6, 7], [8, 9] ]

A more flexible solution is a sliding window method, usually named windows:

let windowsOf3 = Array.from(digits.windows(3));
// [ [0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7], [6, 7, 8], [7, 8, 9] ]

let windowsOf2AdvancingBy3 = Array.from(digits.windows(2, 3));
// [ [0, 1], [3, 4], [6, 7], [9] ]

chunks is just a specialisation of windows where chunks(n) is equivalent to windows(n, n).

let chunksOf4 = Array.from(digits.windows(4, 4));
// [ [0, 1, 2, 3], [4, 5, 6, 7], [8, 9] ]

design space

  • sliding windows sliding in/out past ends instead of sliding within boundaries?
  • default chunk size to 2?
  • chunking by 0 length?
  • windows advancing by 0?
  • should truncated chunks be returned?
    • filler/padding elements?
  • should windows ever be truncated?
    • should preserve equivalence with chunks, so answer should match chunks
    • only when full iterator doesn't fill a single window?
    • just lock step to 1 for windowing?

prior art

other languages

language library chunks windows chunks of 0? truncates windows?
Clojure core partition partition infinite empty lists when insufficient padding;
terminates after 1
Elm List.Extra groupsOf groupsOfWithStep empty list no
Haskell split chunksOf divvy infinite empty lists yes
Java Stream -- -- N/A N/A
Kotlin Iterable chunked windowed throws configurable via parameter
.NET System.Linq Enumerable.Chunk -- throws N/A
PHP array array_chunk -- throws N/A
Python itertools (3.12) batched -- ?? N/A
Python more-itertools grouper windowed empty iterator no, mandatory fill value
Ruby Enumerable each_slice each_cons throws no, step not configurable
Rust Iterator array_chunks map_windows panics no, step not configurable
Rust slice chunks windows panics no, step not configurable
Scala Seq grouped sliding throws yes
Swift Sequence -- -- N/A N/A

JS libraries

library chunks windows chunks of 0? truncates windows?
chunk chunk -- coerces 0 to false 😞 N/A
extra-iterable chunk chunk infinite empty arrays yes
iter-ops page -- throws N/A
iter-tools batch window, windowAhead, windowBehind throws optionally
iterablefu chunk -- collects everything into a single array N/A
itertools-ts chunkwise chunkwiseOverlap throws yes
Lodash / Underscore chunk -- infinite empty arrays N/A
Ramda splitEvery aperture infinite empty arrays no
sequency chunk -- throws N/A
wu chunk -- collects everything into a single array N/A

About

a proposal to add a method to iterators for producing an iterator of its subsequences

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 50.1%
  • JavaScript 49.9%