You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #10624 we're building a new type of incremental model strategy, microbatch. In #10635 we allowed people to specify event_time_start and event_time_end via the CLI. However, the more common situation will be that neither are specified (or perhaps only one of them is specified).
In the case where event_time_end isn't specified, then the value of event_time_end should become now. In the case where event_time_start isn't specified, then the value of event_time_start should become the value of event_time_end truncated to the start of the batch_size definition.
In the case where neither is specified, what is listed above still applies. That is to say that if now is 2024-09-04 13:11:00 and my batch_size is day, then event_time_end would be 2024-09-04 13:11:00 and event_time_start would be 2024-09-04 00:00:00.
Acceptance criteria
event_time_end and event_time_start provided from the CLI is preferred
if event_time_end is not specified, then the value of event_time_end becomes now
if event_time_start is not specified, then the value of event_time_start becomes event_time_end truncated by the batch_size
Suggested Tests
Values of event_time_end and event_time_end provided from the CLI are preffered
If event_time_end is not provided, it is correctly defaulted
If event_time_start is not provided it is correctly defaulted
QMalcolm
changed the title
Automatically set --event-start-time and --event-end-time values if not provided
[Feature] Automatically set --event-start-time and --event-end-time values if not provided
Sep 4, 2024
Housekeeping
Short description
In #10624 we're building a new type of incremental model strategy,
microbatch
. In #10635 we allowed people to specifyevent_time_start
andevent_time_end
via the CLI. However, the more common situation will be that neither are specified (or perhaps only one of them is specified).In the case where
event_time_end
isn't specified, then the value ofevent_time_end
should become now. In the case whereevent_time_start
isn't specified, then the value ofevent_time_start
should become the value ofevent_time_end
truncated to the start of thebatch_size
definition.In the case where neither is specified, what is listed above still applies. That is to say that if now is
2024-09-04 13:11:00
and mybatch_size
isday
, thenevent_time_end
would be2024-09-04 13:11:00
andevent_time_start
would be2024-09-04 00:00:00
.Acceptance criteria
event_time_end
andevent_time_start
provided from the CLI is preferredevent_time_end
is not specified, then the value ofevent_time_end
becomes nowevent_time_start
is not specified, then the value ofevent_time_start
becomesevent_time_end
truncated by thebatch_size
Suggested Tests
event_time_end
andevent_time_end
provided from the CLI are prefferedevent_time_end
is not provided, it is correctly defaultedevent_time_start
is not provided it is correctly defaultedImpact to Other Teams
N/A
Will backports be required?
No
Context
Depends on:
--event-start-time
and--event-end-time
as CLI flags for specifying microbatch start and end times #10635batch_size
as a top-level property of a model #10637The text was updated successfully, but these errors were encountered: