Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slm #421

Open
wants to merge 460 commits into
base: loader
Choose a base branch
from
Open

Slm #421

wants to merge 460 commits into from

Conversation

gparmer
Copy link
Collaborator

@gparmer gparmer commented Jun 16, 2021

Summary of this Pull Request (PR)

Initial slm implementation that is designed to

  1. simplify the current sl codebase by removing hierarchical scheduling,
  2. removing allocation from the core code, and
  3. decomposing the system into modules more cleanly.

This is a work in progress, and the tests are in implementation/tests/slm/main.c. Trello encodes many of the next steps, but they include:

  1. getting multicore working again which involves bootup, and communication (queues, and asnds),
  2. determine the proper design of the periodic blocking API,
  3. adding the blockpoint logic, and
  4. test the heck out of it with existing tests.

Intent for your PR

Choose one (Mandatory):

  • This PR is for a code-review and is intended to get feedback, but not to be pulled yet.
  • This PR is mature, and ready to be integrated into the repo.

Reviewers (Mandatory):

(Specify @<github.com username(s)> of the reviewers. Ex: @user1, @user2)

@WenyuanShao @msdx321 @betahxy

Code Quality

As part of this pull request, I've considered the following:

Style:

  • Comments adhere to the Style Guide (SG)
  • Spacing adhere's to the SG
  • Naming adhere's to the SG
  • All other aspects of the SG are adhered to, or exceptions are justified in this pull request
  • I have run the auto formatter on my code before submitting this PR (see doc/auto_formatter.md for instructions)

Code Craftsmanship:

  • I've made an attempt to remove all redundant code
  • I've considered ways in which my changes might impact existing code, and cleaned it up
  • I've formatted the code in an effort to make it easier to read (proper error handling, function use, etc...)
  • I've commented appropriately where code is tricky
  • I agree that there is no "throw-away" code, and that code in this PR is of high quality

Testing

I've tested the code using the following test programs (provide list here):

  • slmtests

@gparmer gparmer requested a review from WenyuanShao June 16, 2021 17:06
Copy link
Collaborator Author

@gparmer gparmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some notes on things to fix.

@@ -0,0 +1,5 @@
#ifndef SLM_POLICY_TIMER_H
#define SLM_POLICY_TIMER_H

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this file.

#ifndef SLM_TIMER_H
#define SLM_TIMER_H

struct slm_timer_thd {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REmove this file.

@@ -0,0 +1,9 @@
## tests

This is the skeleton interface used by the `mkcomponent.sh` script to aid in the creation of a new component.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: add documentation here.

};

struct slm_timer_thd *
slm_thd_timer_policy(struct slm_thd *t)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace with slm_container.h (see below).

Create a set of macros to enable the scheduler to easily mix and match different policies (edf/fprr/fifo), timers (quantum/oneshot), and memory allocators (static/dynamic).

@@ -0,0 +1,9 @@
## slmin

This is the skeleton library used by the `mklib.sh` script to aid in the creation of a new library.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add actual documentation.


/* Thread allocation and indexing API */

struct slm_thd *slm_policy_thd_alloc(void);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FIXME: make all of this consistent with the above fprr APIs, or remove.

* `ret`. This assumes that ret == 0 is success, otherwise jump to the
* label.
*/
#define ERR_CHECK(expr, label) \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix indentation.

betahxy and others added 27 commits October 8, 2021 09:36
- -no-pic and -no-pie are not valid flags in ld
- update needed env deps
- it does not make sense to specify this section manually and set a PHDR (.interp segment) for it, this may cause ld reporting errors because some Alloc type sections might be tried to put into the interp segment by the linker, but we don't have this segment in the final PHDR set.
- fix composer allocating captbl_end using BOOT_CAPTBL_FREE, it is 52 now. This can be used by both i386 and x86_64
- fix pgtbl_lkup_lvl to return/set page table entry flags correctly
- fix cos compinfo initialiaztion problem
still needs higher level API/library to call
still needs to be tested
- x86_64 ABI requires a 16-byte alignment for fetching data and when calling a function, because some instructions like movabs and sse require data to be 16-byte aligned. Otherwise, it will case a #GP
- .boottext section needs to be aligned to generate correct bootable image
- the stack in loader.S needs to be set to .data section, the original .comm will generate a .bss section that could have random address after compiling
- fix the linker script so it can correctly compile a bootable image
- add -nmagic flag to $(LD) in the Makefile so $(LD) does not add additional blank information to the bootable image
- enable smp booting into long mode and jump into kmain
betahxy and others added 30 commits May 2, 2022 00:51
…nd validation of composition scripts, and added access to AS representation in the composer's state.

The overall goal is to add full initargs support for address spaces, going all the way back to a specification for address spaces from the composition scripts.

Steps:

1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. Order ASes by the parent relationship.
4. Assign virtual addresses based on ASes and the parent relation.
5. Synthesize the initargs based on the ASes and addresses.
6. Use the initargs and the AS specification in the booter.
…zation (with containing components, and exclusive AS components)

Steps:
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. -> DONE: Order ASes by the parent relationship.
4. Assign virtual addresses based on ASes and the parent relation.
5. Synthesize the initargs based on the ASes and addresses.
6. Use the initargs and the AS specification in the booter.
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. DONE: Order ASes by the parent relationship.
4. -> DONE: Assign virtual addresses based on ASes and the parent relation.
    This is *not yet integrated into the output*. That's next commit.

5. Synthesize the initargs based on the ASes and addresses.
6. Use the initargs and the AS specification in the booter.
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. DONE: Order ASes by the parent relationship.
4. DONE: Assign virtual addresses based on ASes and the parent relation.
    -> Now integrated into the output!
5. Synthesize the initargs based on the ASes and addresses.
6. Use the initargs and the AS specification in the booter.
…fication.

Steps:
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. DONE: Order ASes by the parent relationship.
4. DONE: Assign virtual addresses based on ASes and the parent relation.
5. -> DONE: Synthesize the initargs based on the ASes and addresses.
6. Use the initargs and the AS specification in the booter.
Steps:
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. DONE: Order ASes by the parent relationship.
4. DONE: Assign virtual addresses based on ASes and the parent relation.
5. DONE: Synthesize the initargs based on the ASes and addresses.
6. -> DONE: Use the initargs and the AS specification in the booter.

Pending: BUG for split ASes that don't properly maintain non-overlapping ASes.
Steps:
1. DONE: parsing and validation of composition scripts.
2. DONE: add an AS representation to the pass structure of the composer for later pass access.
3. DONE: Order ASes by the parent relationship.
4. DONE: Assign virtual addresses based on ASes and the parent relation.
5. DONE: Synthesize the initargs based on the ASes and addresses.
6. DONE: Use the initargs and the AS specification in the booter.
Official NIC/DPDK component support to Composite
Virtual address space specification, assignment, and booting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants