[OLD] M3SA - Multi-Meta-Model Simulation Analyzer #240
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rq1 multi model
…ent window size - time to generate analysis (description + implementation)
…ent window size - time to generate analysis (description + implementation)
…ement-and-experiments #7 feat opendc analyze enhancement and experiments
…ation files can be found on the branch dedicated for experiments
…ation files can be found on the branch dedicated for experiments
…lotted individually and in the same plot
…d 1000, plotted individually and in the same plot
downlaoding dependencies for the website may take more time than initially allowed
|
@DanteNiewenhuis, when merging, please choose from the dropdown "Squash and merge." |
* metamodel & multimodel integrated in the failure branch * untracked demo * .gitignore updated to untrack demo * code improved towards better readability * metamodel functional for scenarios with the same configurations * docstring added * code structure improved * demo folder moved in opendc/demo * an unnecessarily commed removed from Radu's tests * demo removed (temporarily) from gitignore * demo files added to git * towards a margin-of-error-capable Multi-Model * [RQ1] simulation data; RQ1 first draft finished * Multi-Model and Meta-Model are now clean * expriment folder setup, Experiment 1 update, add in a folder, Experiment window size - time to generate analysis (description + implementation) * expriment folder setup, Experiment 1 update, add in a folder, Experiment window size - time to generate analysis (description + implementation) * use case experiment 1 results * use case experiment 1 - code base * multimodel now handles cpu_utilization and outputs statistics * rq1 multi-model analysis by window_size, experiments and plots * gitignore updated to ignore the experiment-related files - experimentation files can be found on the branch dedicated for experiments * gitignore updated to ignore the experiment-related files - experimentation files can be found on the branch dedicated for experiments * experiment 1 data, windows analyed by size of 1, 10, 100, and 1000, plotted individually and in the same plot * experiment 1 functionality, windows analyed by size of 1, 10, 100, and 1000, plotted individually and in the same plot * renamings for better in-paper linkings * branch test 1 * branch test 2 * demo folder untracked * resources (currently only traces, soon also others e.g., prebuilts) moved in their own folder * experiment 1 renamed more specifically, experiment 2 coming soon * gitignore updated to ignore demo * demo.zip removed... * resources folder, with traces (for now) * raw linking between the python backend, and then opendc-analyzer json interface * (feat) kotlin backend gives 2-parameter input to python backend, one for outputFolderPath, one for multiModelSetup * (feat) user-input parser file * (feat) multi-model linked with user provided configuration/input * (testing) input provided for testing the program - after a brief testing, it works * (documentation) input doc for MultiMetaModel * (documentation) scenario doc updated to match the MultiMetaModel documentation * (documentation) added export rate in MultiMetaModel input * (feat) MultiModel is now capable of reading and parsing user input, along side its main functionalities * (fix) experiment python scripts rescued * experiment 2 simulation files added, files for sample-rate=3s were omitted (larger than 100mb limit) * sample-rate=3s simulation files added via GIT LFS * (feat) counted the number of samples in experiment 2, per sampling rate * (feat) .gitattributes updated accordingly * Update .gitattributes and remove certain files from Git LFS * Committing local changes before merging * test * gittest * gittest * Remove LFS tracking from files * gittest * gittest * order of the arguments repaired * (fix) order of the arguments (only now) repaired * (feat) Multi-Model now handles, based on user's input, both time-series and cumulative plots * (misc) commit before temporarily changing the branch * (fix) experiment file names improved * (fix) multi-model window outputting problem solved * (documentation) updated to match user input * (fix) updated to match user input * (fix) experiment 2 - user-experience improved * demo setup for meeting * (feat&fix) MultiModel handles vertical and horizontal limits from input, window accuracy was improved, useless content deleted * (feat) Multi-Model cumulative time series plot almost developed; other Multi-Model plots refined; demo for the Multi-Model * (feat) Multi-Model, almost reaching final of iteration 2, user input updated, efficiency improved * (feat) documentation updated, experiment 2 updated, and re-ran for multiple types of plots, many plots + multimodel towards the end of iteration 2 * (feat) towards finishing rq1 * (feat) experiment 4 and misc * (fix) opendc-analyzer folder cleaned * (feat) parser updated to match the meta-model input * (feat) plotting integration of the metamodel * (feat) MetaModel very-basic functionality initialization, computation with only one function (mean), multi-model appending and linking, plotting tested (and works) fot time series graph yet * (feat) better naming and little, temporary, ducktape, to disable a feature and allow meta-model to function * (misc) output results, files -- with this commit, the first sub-iteration of the MVP is complete * (feat) meta-model handles also cumulative * (feat) last commit before major refactoring * (feat) MVP of metamodel done 3 functions handled, integrated with multimodel, able to plot all the types of graphs the multimodel plots, the code is in a relatively good shape, still requires documentation. up next: iteration 2! * (feat) Meta-Model outputs the meta-model simulation results (feat) experiment 5, accuracy, currently setting up * (feat) bitbrains small analysis * (feat) surf-sara and carbon-trace workloads analyzed * (feat) accuracy evaluator setup * (doc) code documentation * (fix) removed use_cases (not used anymore) * (doc) following the industry standards * (feat) accuracy evaluator, part of experiment 5, implemented MVP future elements: have a better output, save into a file, with specific details per model / metamodel * (feat) accuracy_evaluator outputs in an external file + some accuracy analysis for individual models, metamodel with mean, and metamodel with median * experiment 5 addded * (feat) seeking meta_equations with lower error rate(s) than mean/median (still searching) * (feat) Warning: Cannot have a Meta-Model without a Multi-Model. No computation made." * experiment 6 added * (fix) clarified: aggregation_function changed to window_function * (fix) sample rate added in the reproducibility.md file for exp 6 * (feat) aggregation_function -> window_function; vertical&horizontal ticks can now be selected by the user; metamodel more proeminent in the plots * (feat) fr6 plots * (feat) experiment 7 report files * (feat) gitignore ignores large files * (feat) gitignore * (feat) experiment 7 results * Experiment 5 adjusted for RMSLE in %age * experiment 7 and experiment 8 data * (feat) experiment data updated * (feat) data plots for the experiment * (feat) large data files ignored * . * (feat) documentation update * (feat) documenting code * (fix) demo folder removed * (fix) meta_model now outputs straight after computation * (feat) md file setup, folder setup * (feat) host renamed to "sim". this makes the tool more inclusive. MultiModel also handles seed every variable name containing the word host, and the corresponding documentation, we rename to "sim" (simulation). for example raw-host-data -> raw-sim-data. Also, the Multi-Model now handles the seed, which can be given by user, or is 0 by default. * (feat) added the seed, as a non-mandatory value, defaulting to 0 * (feat) added the seed, as a non-mandatory value, defaulting to 0 * (feat) host -> sim (refactoring consequences) * (feat) multi-meta-model integration file, started, linked with main for testing * (feat) corresponding integration & analysis (testing) files * (feat) multi-meta-model-integration file - towards draft 1 * (feat) multi-meta-model-integration file - draft 1 * (feature) multimodel has now ID -101 (instead of 101) this "feature" does not limit anymore the multimodel to "only" 100 models (hence clashing when the id increases to 100) * (fix) exit codes no more relevant * (feat) simulation data files * (feat) minor change * (feat) Simulator Specifics * accuracy and performance validation for RQ3 * (feat) new branding - Multi-Meta-Model is now "M3SA" M3SA - the short of Multi-Meta-Model Simulation Analysis * (feat) new branding - Multi-Meta-Model is now "M3SA" M3SA - the short of Multi-Meta-Model Simulation Analysis * (fix) universality simulation folder removed from local-master * (feat) experiment being set up * (feat) experiment being set up * (feat) analyzr now is M3SA * (feat) use case - usability evaluation * (feat) use case - usability evaluation * (fix) removed unused files * (fix) analyzer renamed to "m3sa" * (fix) analyzer renamed to "m3sa" * (fix) m3sa linked with opendc * (fix) splotless applied * (fix) preparing PR * (fix) preparing PR - updating website * (feat) PR template nicer * (feat) CI: Add disk cleanup and usage checks to github actions * (feat) CI: increased timeout time for documentation downlaoding dependencies for the website may take more time than initially allowed * (feat) CI: Enhance GitHub Pages setup and optimize Node.js caching in CI. * testing website workflow without m3sa documentation * build workflow reverted to initial * website files back here * (fix) <br> now as <br /> * (fix) docusaurus 'broken links' solved * (fix) experimental components removed from the master branch * gitignore removes pyc and pycache * removed all pyc and pycache
monitor does not have anymore serversPending attribute which causes test (hence workflows) failures
Radu-Nicolae
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
M3SA - Multi-Meta-Model Simulation Analysis. Full functionality implemented and integrated into OpenDC.
Multi-Model
Meta-Model
M3SA
-a
when running OpenDC, via
ScenarioCli.kt
.
Paper
M3SA is part of the research efforts of AtLarge Research Team, towards massivizing computer systems. We conduct scientific research and follow the AtLarge Design Vision. We design, analyze, and engineer the M3SA system, towards performance, accuracy, universality, and usability.
M3SA: Exploring the Performance and Climate Impact of Datacenters by Multi-Model Simulation and Analysis
Authors: Radu Nicolae (@Radu-Nicolae), Alexandru Iosup (@AlexandruIosup), Dante Niewenhuis (@DanteNiewenhuis)
External Dependencies 🍀
N/A
Breaking API Changes⚠️
N/A
Simply specify none (N/A) if not applicable.