Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add document to p4c repo explaining to developers a little bit of what they need to know to add new tests #32

Open
jafingerhut opened this issue Dec 20, 2024 · 0 comments
Labels
smalltask A task that appears to require a small amount of work

Comments

@jafingerhut
Copy link
Collaborator

At the very least, it should explain:

  • for "positive" tests, i.e. ones where there is no error from the P4 compiler, what directory should you put the source file in, and what expected output files should be added (and in what directory for those)? Also how to update those if a change to p4c changes the expected outputs.

  • for "negative" tests, i.e. the ones where you expect the P4 compiler to catch a syntax or semantic error, what directory should you put the source file in, and what expected output files should be added?

  • How to mark a test as "expected to fail" (Xfail), and what does that mean? Note: At least for some existing CI tests, marking a positive test as Xfail means that it must fail, or else the overall CI run will fail. Marking a negative test as Xfail means that it must not give an error, or else the overall CI run will fail. However, because this is a confusing and subtle point, maybe not all p4c CI tests treat negative tests this way. It would be nice to make them all consistent, but at least we should document how they operate now, even if there is a TODO indicating the desire to change it in the future.

@fruffy fruffy added the smalltask A task that appears to require a small amount of work label Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
smalltask A task that appears to require a small amount of work
Projects
None yet
Development

No branches or pull requests

2 participants