Trial_timeout not working properly for hyperparameter sweep in pipeline

[morrissharp]

• Package Name: azure-ai-ml
• Package Version: 1.24.0
• Operating System: Windows
• Python Version: 3.11

Describe the bug
I am trying to submit a hyperparameter sweep job in a pipeline with a trial_timeout, but the trial_timeout does not get set (and the job continues to run past the trial_timeout time). It appears that when submitting from the python sdk, the maxRunDurationSeconds does not get set properly for the individual trial (see screenshot below).

To Reproduce
Steps to reproduce the behavior:

Run the code below:

benchmark_run_component_func = load_component(source="./benchmark.yml")
score_component_func = load_component(source="./aggregation.yml")

# define a pipeline
@pipeline()
def pipeline_with_hyperparameter_sweep():
    """Tune hyperparameters using sample components."""
    benchmark_run = benchmark_run_component_func(
        scenario=Choice(
            [
                44,
                45,
                46,
            ]
        ),
    )

    sweep_step = benchmark_run.sweep(
        primary_metric="correctness_score",
        goal="maximize",
        sampling_algorithm="grid",
        compute="small-cluster-low-priority",
    )

    sweep_step.set_limits(max_total_trials=200, max_concurrent_trials=3, timeout=100000, trial_timeout=1800)

    score_data = score_component_func(results_dir=sweep_step.outputs.benchmark_results_dir)


pipeline_job = pipeline_with_hyperparameter_sweep()

ml_client.jobs.create_or_update(pipeline_job)

Expected behavior
A trial should get cancelled if it goes past the trial_timeout time.

Screenshots
I have checked the raw json for the individual trials, and it appears that maxRunDurationSeconds is not set.

Job running for 2+ hrs when trial_timeout is set to 1800 seconds (30 min)

Additional context
Add any other context about the problem here.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/azure-ml-sdk @azureml-github.

[kingernupur]

Thank you for reporting the issue @morrissharp. Our team is looking into it.