A Source can declare that its rows arrive sorted. The planner follows that promise through every stage and uses it to run an Aggregate in streaming mode, one group at a time, instead of holding every group in memory. Some stages keep the promise and some break it, often without telling you. Build a pipeline and watch where it holds.
keeps it
…unless it writes a field the order is on, or uses distinct. Writing the same value back counts: emit department = department drops an order on department, because the planner looks at which fields a Transform writes, not at what it writes into them. Leave sort fields out of your emits; every input field is carried through anyway.
keeps it
A Route branch keeps its input's order, and so does a composition call.
drops it
Always, even when every input is sorted the same way: two sorted inputs one after the other are not one sorted stream. It also joins the files of a multi-file Source into one stream.
drops it
A join's output has no defined order, whichever join method runs. Reshape and Cull reorder rows within each group.
adds one
The Source's rows are sorted by the key first, unless its sort_order already starts with the key, so a later Aggregate grouped by the key can stream.
per file
With glob:, regex: or paths:, each file is checked and sorted on its own; there is no order across files. An Aggregate can still stream: it works through each file separately.
sort_order is verified for each file as it's read. With on_unsorted: warn (the default), a file that is out of order is sorted and a W307 warning is written; with error the file is rejected. See Source Nodes.Take as many fields from the front of the order as there are group_by fields. If they are the same fields, in any order, the Aggregate can stream; its group_by is put in the order's order. Otherwise it uses hash aggregation, which holds every group until its input ends.
[department, day] streams group_by: [department] and [day, department], but not [day]: the first field is department.group_by (one total) always streams.strategy: auto (the default) falls back to hash quietly. strategy: streaming turns that fallback into an error (CXL0419) when the pipeline is checked, which is how you catch a change that broke the order.strategy: hash always uses hash aggregation.