You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #10624 we're building a new type of incremental model strategy, microbatch. To support that we need a concept an batch_size. The batch_size is essentially how "big" the incrementing of the model is. It should support the values day, month, and year. The effect in practice is that if your batch_size is day then when the event_start_time is generated in #10636, we can properly calculate what the "start" of a batch should be. Additionally, the lookback which is introduced in #10662 will essentially offset the event_start_time by an integer multiple of the batch_size
QMalcolm
changed the title
Allow setting of event_time_granularity as a top-level property of a model
[Feature] Allow setting of batch_size as a top-level property of a model
Sep 4, 2024
Housekeeping
Short description
In #10624 we're building a new type of incremental model strategy,
microbatch
. To support that we need a concept anbatch_size
. Thebatch_size
is essentially how "big" the incrementing of the model is. It should support the valuesday
,month
, andyear
. The effect in practice is that if yourbatch_size
isday
then when theevent_start_time
is generated in #10636, we can properly calculate what the "start" of a batch should be. Additionally, thelookback
which is introduced in #10662 will essentially offset theevent_start_time
by an integer multiple of thebatch_size
If defining it in the model sql it'd look like
If defining it in the model yaml it'd look like:
Acceptance criteria
batch_size
can be set asday
,month
, oryear
for an incremental microbatch modelSuggested Tests
batch
size makes it into the python representation of the modelImpact to Other Teams
Cloud artifacts, new property in the config of models
Will backports be required?
No
Context
No response
The text was updated successfully, but these errors were encountered: