Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Pollux scheduler #9

Open
wants to merge 51 commits into
base: main
Choose a base branch
from
Open

Implement Pollux scheduler #9

wants to merge 51 commits into from

Commits on Apr 10, 2024

  1. create pollux.py pollux_scheduler.py framework

    @yuxianshi
    yuxianshi committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    e88465d View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Add code for tracking and using job_infos and allocations as in Pollu… 

    …x (need to run and debug)
    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    98fd651 View commit details
    Browse the repository at this point in the history

  2. add cluster_state.allocations

    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    e3a3910 View commit details
    Browse the repository at this point in the history

  3. wrap some code into function update_allocation_info

    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    9b8a395 View commit details
    Browse the repository at this point in the history

  4. minor changes

    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    a0ef639 View commit details
    Browse the repository at this point in the history

  5. minor changes

    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    ff209d5 View commit details
    Browse the repository at this point in the history

  6. obtain to_suspend

    @yuxianshi
    yuxianshi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    cdf3c9c View commit details
    Browse the repository at this point in the history

  7. Convert node_info of Blox to nodes map of Pollux

    @manasdtrivedi
    manasdtrivedi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    c05ed2d View commit details
    Browse the repository at this point in the history

  8. Merge job_info and node_info changes

    @manasdtrivedi
    manasdtrivedi committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    322ebc0 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. allow sending model name in workload.py

    @yuxianshi
    yuxianshi committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    f1f710d View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Fix imports, cluster state bug

    @manasdtrivedi
    manasdtrivedi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    7488bb9 View commit details
    Browse the repository at this point in the history

  2. add pollux trace to workload_pollux

    @yuxianshi
    yuxianshi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    90cfb88 View commit details
    Browse the repository at this point in the history

  3. change default scheduler in simulator_simple.py to Pollux

    @yuxianshi
    yuxianshi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    895b4af View commit details
    Browse the repository at this point in the history

  4. Add requirements.txt

    @manasdtrivedi
    manasdtrivedi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    5543c4a View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. add calculations of "to_suspend" and "to_launch" in pollux.py

    @yuxianshi
    yuxianshi committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    926deff View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. add simulation args to match Pollux simulation setting

    @yuxianshi
    yuxianshi committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    a2d9529 View commit details
    Browse the repository at this point in the history

  2. minor change

    @yuxianshi
    yuxianshi committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    ece995f View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. Add workload files

    @manasdtrivedi
    manasdtrivedi committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    71890e3 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. debugging pollux.py

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    bbf3873 View commit details
    Browse the repository at this point in the history

  2. minor change

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    6660fa3 View commit details
    Browse the repository at this point in the history

  3. debugging: job_ids_to_track & parse_philly_jobs

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    a80477c View commit details
    Browse the repository at this point in the history

  4. set job completion by target epoch for pollux

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    97dc150 View commit details
    Browse the repository at this point in the history

  5. minor change

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    232493c View commit details
    Browse the repository at this point in the history

  6. Use numCPUcores parameter when initializing Pollux's nodes

    @manasdtrivedi
    manasdtrivedi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    df69651 View commit details
    Browse the repository at this point in the history

  7. run.sh

    @yuxianshi
    yuxianshi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    95a80b1 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. debugging: prune "allocations"

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    44737af View commit details
    Browse the repository at this point in the history

  2. debugging: solve keyerror in calculation of "to_launch" in pollux.py

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    4fc7cde View commit details
    Browse the repository at this point in the history

  3. debugging: address inifinite loop issue

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    b778378 View commit details
    Browse the repository at this point in the history

  4. debugging: address infinit loop

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    a759d43 View commit details
    Browse the repository at this point in the history

  5. debugging: remove Job object from job_state.job_runtime_stats

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    c455a82 View commit details
    Browse the repository at this point in the history

  6. result: interval = 1min

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    a07e126 View commit details
    Browse the repository at this point in the history

  7. result: 2min and 8min

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    ef2daa6 View commit details
    Browse the repository at this point in the history

  8. result: 4 min

    @manasdtrivedi
    manasdtrivedi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    06d235b View commit details
    Browse the repository at this point in the history

  9. result: 30s interval

    @yuxianshi
    yuxianshi committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    f05dbd0 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Add adaptdl JCT data

    @manasdtrivedi
    manasdtrivedi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    83d2e49 View commit details
    Browse the repository at this point in the history

  2. Add plotting, update requirements

    @manasdtrivedi
    manasdtrivedi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    d8789fe View commit details
    Browse the repository at this point in the history

  3. pollux reproduce results: 4m & 8m

    @yuxianshi
    yuxianshi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    253689f View commit details
    Browse the repository at this point in the history

  4. plot: using reproduced result of pollux-author version 4m and 8m

    @yuxianshi
    yuxianshi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    5a39fe6 View commit details
    Browse the repository at this point in the history

  5. plot: match color in poster

    @yuxianshi
    yuxianshi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    3a30ccc View commit details
    Browse the repository at this point in the history

  6. minor change

    @yuxianshi
    yuxianshi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    bf227fe View commit details
    Browse the repository at this point in the history

  7. reprocuding pollux result: 1m & 2m

    @yuxianshi
    yuxianshi committed May 1, 2024
    Configuration menu
    Copy the full SHA
    d07f94a View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. reprocuding pollux result: 30s

    @yuxianshi
    yuxianshi committed May 2, 2024
    Configuration menu
    Copy the full SHA
    630c395 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. minor change

    @yuxianshi
    yuxianshi committed May 7, 2024
    Configuration menu
    Copy the full SHA
    40ed3c0 View commit details
    Browse the repository at this point in the history

  2. update plot to use reproduced Pollux author implementation result

    @yuxianshi
    yuxianshi committed May 7, 2024
    Configuration menu
    Copy the full SHA
    95cb545 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2024

  1. minor changes

    @yuxianshi
    yuxianshi committed May 12, 2024
    Configuration menu
    Copy the full SHA
    a7e21b4 View commit details
    Browse the repository at this point in the history

  2. debugging: fix current time in Pollux Job object (inside job_state)

    @yuxianshi
    yuxianshi committed May 12, 2024
    Configuration menu
    Copy the full SHA
    af9d32e View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. simulation: update result

    @yuxianshi
    yuxianshi committed May 13, 2024
    Configuration menu
    Copy the full SHA
    9b21dfa View commit details
    Browse the repository at this point in the history

  2. minor change

    @yuxianshi
    yuxianshi committed May 13, 2024
    Configuration menu
    Copy the full SHA
    9921089 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. Remove debug comments

    @manasdtrivedi
    manasdtrivedi committed May 21, 2024
    Configuration menu
    Copy the full SHA
    3bda855 View commit details
    Browse the repository at this point in the history

  2. Refactoring: file names and directory structure

    @manasdtrivedi
    manasdtrivedi committed May 21, 2024
    Configuration menu
    Copy the full SHA
    9aed55c View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Update installation and execution instructions

    @manasdtrivedi
    manasdtrivedi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    e69871b View commit details
    Browse the repository at this point in the history