Repository navigation
Differences in Base.Iterators.partition and IterTools.partition #67
Description
There are some differences between Base.Iterators.partition and IterTools.partition. I brought it up on Slack and @oxinabox asked me to file an issue.
Base.Iterators.partitionreturns each chunk as anArray, whereasIterTools.partitionreturns aTuple- In the case that the last chunk is smaller than the requested partition size,
Base.Iterators.partitionreturns a shorter chunk, whereasIterTools.partitiondrops the last values. IterTools.partitionaccepts astepargument that determines the hop size for each chunk.
Activity
ssfrr commentedon Mar 5, 2020
update - I think on
masterBase.Iterators.partitionnow returns views into the original array, notArrays.This might (?) be the last inconsistency that would need to be addressed before something like #30 could be resolved. How do folks want to resolve the differences listed above? I'd propose using the
Base.Iteratorsbehavior for (1) and (2), but keeping thestepargument.One path forward would be to add the
stepargument as a PR toBase, then just reexportBase.Iterators.partition. This would also take advantage of the recent improvements that @mbauman made.omus commentedon Mar 5, 2020
Change to make
Base.Iterators.partitionreturn views was done in: JuliaLang/julia#33533goretkin commentedon May 19, 2020
I'm in favor of keeping the current behavior of
IterTools.partition, perhaps under a different name, to avoid confusion withBase.For example, implementing
Base.diff, but for an iterableitinstead of justAbstractVectors is very natural in terms ofIterTools.partition(it, 2, 1), and the fact that it produces a stream of tuples, which have a length known at compile time is (I'm guessing) beneficial for efficient code generation.ssfrr commentedon May 24, 2020
Yeah, it's sort of a shame that the implementation needs to be different depending on whether it's designed for long or short partitions (which would use Arrays or Tuples, respectively). For instance, in my use case (streaming audio) I often want to chop the stream into chunks of 512 or 1024 elements, so Tuples aren't good for that. Maybe the partition type should be provided as an optional argument to
partition?goretkin commentedon May 24, 2020
Oh, good. That's an important use too.
Base.Iterators.partitionreturns views into the array, instead of copies, now. Does that fit the audio chunking need?ssfrr commentedon May 24, 2020
Yeah, array views are good for me
pepijndevos commentedon Jun 14, 2023
Difference 2 is handled in Clojure by having partition-all and partition, where base
partitionmaps to Clojure'spartition-allbehavior.Difference 1 & 2 are grouped together in Transducers under Consecutive vs Partition. So you either get fixed length tuples or variable length arrays.
I think this roughly aligns with common usage patterns where you either want to have small fixed length groups, or process large chunks of an array.
In my ideal universe both
consecutive(IterToolspartition) andpartitionwould be in base, both supporting astepargument.nsajko commentedon Mar 8, 2026
FYI PR #140 adds some docs to point out the similarity and differences.