Skip to content

Latest commit

 

History

History
508 lines (374 loc) · 27.5 KB

CONTRIBUTING.md

File metadata and controls

508 lines (374 loc) · 27.5 KB
Raw

Cocotb Contribution Guidelines

Welcome to the cocotb development! We are an inclusive community with the common goal of improving the cocotb, a coroutine based cosimulation library for writing VHDL and Verilog testbenches in Python. This guide explains how to contribute to cocotb, and documents the processes we agreed on to manage the project. All processes in this document are designed to streamline the development effort, to avoid bottlenecks, and to ultimately give a pleasant experience to all involved.

Getting Help

Cocotb is a diverse and challenging project to contribute to. If you ever feel lost, out of your depth, or simply want to know more, the cocotb Gitter channel is actively watched by many cocotb users, contributors, and maintainers. It is a good idea if you are unsure whether your problem or question is worthy of a Github Issue to first post it to the Gitter channel. You may also ask questions in Github issues. If you don't receive any response on the Gitter channel or a Github issue, or you want help privately, you may directly contact a maintainer.

What to Work On

There is a lot of work to do on this project, no matter your area of expertise or skill level.

If you are a beginner there are several Github issues marked good first issue you can look at. There are also a number of things you can work on that aren't in Github issues and don't require in-depth knowledge of the cocotb internals. They including the following:

  • Documentation improvements
  • Managing Github issues and the Gitter channel
  • Testing and coverage improvements

Cocotb is still not perfect. There are plenty of bug fixes and features that can be worked on. Most of these are recorded as Github issues.

Documentation

Cocotb's documentation is always open to improvements. Improving documentation will help users better understand and use cocotb; and may decrease the number of questions the Gitter channel and Github issue page. Updating documentation requires knowledge of:

Some documentation should be located in the official documentation on Read the Docs/RTD, while the rest belongs on the Wiki. There are several ways to improve the documentation:

  • Better documenting core functionality (RTD)
  • Documenting common "gotchas" (RTD)
  • Documenting difficult and niche use cases (Wiki)
  • Documenting common design patterns (Wiki)
  • Documenting internal components (Wiki)

See the documentation on building the documentation and the guidelines on submitting pull requests. Documentation improvements typically require no maintainer pre-approval; you can simply work on the documentation and open a pull request. Documentation on the Wiki does not require a pull request; any user with a Github account can contribute to it. Please be responsible with that power.

Project Management

The cocotb project is fairly popular and the Gitter channel and Github issues page receive a fair amount of traffic; which is only expected to increase. People are needed to categorize issues and pull requests, and respond to questions. Working this task is the quickest way to learn how cocotb works. Tending to this task requires the following:

  • people skills
  • an understanding of the scope of cocotb
  • general understanding about how cocotb works

Someone working this task should set notifications on the Gitter channel to be notified of any new comments. They should also "watch" the Github repo by selecting the "Watching" notification level button in the upper right corner of the main Github page. Finally, they should notify the maintainers that they are able and willing to accept questions.

To be able to add labels and close issues and PRs you will need special permissions. Contact a maintainer if you are interested in receiving these permissions. They will be granted according to the project's need and the requestor's familiarity with cocotb. Once you have those permissions, see the guidelines on managing issues and pull requests.

This task can also be done without special repo permissions, by just commenting on the issue or PR. This is especially helpful for Github issues about bugs. If you can duplicate the bug or confirm the bug report is invalid, that helps maintainers a lot.

Tests and Coverage

Cocotb has a suite of unit tests (located in the tests directory) and examples (located in the examples directory) which are functional acceptance tests. If a pull request cannot pass all of these tests, it will likely be rejected. To ensure cocotb only includes the highest quality code, these test should be exhaustive. We use code coverage as a quantifiable metric of the "exhaustiveness" of these tests, and wish to improve this metric.

Working on this task requires a familiarity with:

  • Cocotb's core functionality
  • How to write Verilog and VHDL
  • How to write cocotb tests in Python
  • (Optionally) codecov; coverage aggregator and Github bot
  • (Optionally) the coverage module, for Python code coverage
  • (Optionally) gcov, for C++ code coverage
  • (Optionally) Github Actions, for automatic acceptance testing

Cocotb's regression tests can be improved by:

  • Testing more of cocotb's core functionality
  • Testing corner cases left out of the current set of tests (identified by looking at code coverage)
  • Increasing the matrix of simulators, operating system, and Python installations tested in CI

Testing improvements don't require maintainer pre-approval, but require a pull request. Please see the guidelines on submitting pull requests.

Features

Cocotb is still in development and new features are still welcome and appreciated; as long as they stay in scope. Cocotb is comprised of several major codebases, each requiring different sets of skills and development process. Instead of including that breakdown here, it is done in the internal documentation.

Small improvements to existing features generally do not require maintainer pre-approval. Large changes, approximately >150 LOC changed, and new features generally require maintainer pre-approval. If a change is deemed too large for the main repo, or out of scope, please feel free to make it an extension.

New features must not break existing features.

Feature changes require full coverage of the added feature. This likely requires adding new unit tests to the tests directory. Issue-specific test directories will not be accepted, unless a special HDL entity is required. Instead, place the test in an existing test suite (test_cocotb, test_discovery, etc.).

Features should generally follow the following design principles:

  • Something the user cannot do without assistance of cocotb-specific code
  • Orthogonal to existing features
  • Easily composed with existing features
  • Limited in scope and impervious to scope creep

Bugfixes

!WARNING! Bugfixing cocotb is not for the faint of heart!

Bugs happen. cocotb supports many simulators that have inconsistent support for the procedural interfaces cocotb depends on, and it has a number of features that aren't wholly tested yet. There are likely many bugs lurking, waiting to be found; which is why increasing testing and code coverage is important. Working on bugfixing can be very challenging, depending on the cause of the bug. In general, bugfixing requires knowledge of:

  • How cocotb works
  • cocotb's debugging utilities
  • (Optional) Simulator interfaces (VPI, VHPI, and FLI)
  • (Optional) Python debugging tools (pdb, dowser)
  • (Optional) C/C++ debugging tools (gdb, valgrind)
  • (Optional) Specific simulators (sometimes the bug exists in the simulator and not cocotb)

Fixing a bug follows the procedure:

  1. Locate buggy behavior, make a Github issue
    • Maintainers may be able to offer more information, confirm it as a bug, or confirm it as expected behavior
  2. Make a Minimum Reproducible Failing Example (MRFE, pronounced like Murphy, like the law :)
  3. Open a new draft pull request with the MRFE test
    • It should cause CI to fail
  4. Determine scope of the bug, and add that detail to the pull request
    • Which simulators/interfaces are affected?
    • Which Python versions?
    • Which operating systems?
  5. Determine the cause of the bug, and add that detail to the pull request
    • May require Python or C debugging, or the builtin cocotb debugging utilities
  6. Make a fix, and push it up on the PR branch
    • It should cause the CI to pass
    • The fix should not break other existing functionality

Details on how to debug cocotb can be found on the Wiki.

Deprecations and Removals

Cocotb's treatment of deprecations and removal follows guidelines laid out here. Deprecations serve the following purposes:

  • Remove legacy code that has been deemed out of scope
  • Remove support for a simulator, OS, or Python version that is past end-of-life
  • Remove potentially dangerous, broken, and misunderstood interfaces (usually accompanied with a superior alternative)

Deprecations can be incorporated at any time. They are typically implemented by issuing a DeprecationWarning in Python code; or issuing a LOG_WARN with DEPRECATED in the message in C++ code.

Removals only occur on major version bumps. One can create removal pull requests at any time, on the condition they will not be accepted until the next release is known to be a major version release.

Breaking Changes

Breaking changes are changes to the interface or behavior of a user-facing entity. They are necessary when a user-facing interfaces are broken in a way that cannot be changed without changing the behavior of user's code. In these situations it is ideal to be able to implement a switch between new better behavior and the old broken behavior. On major version bumps, this switch will be deprecated and the new behavior will become the default.

In cases where behavioral switches are not easy to implement, breaking changes will attempt to be broadcasted to user by issuing a DeprecationWarning when the to-be-changed behavior is invoked. Before major releases, pending breaking changes will be incorporated.

One can create pull requests with breaking changes at any time, on the condition they will not be accepted until the next release is known to be a major version release.

Setting Up a Development Environment

Assuming you have used cocotb prior to reading this guide, you will already have the cocotb installation prerequisites and standard development tools (editor, shell, git, etc.) installed.

Additionally, you will need doxygen, for building documentation, and tox, for building documentation and running regression tests.

We recommend if you are using a Linux distribution to use your system package manager to install doxygen. Likewise, doxygen can be installed using the homebrew package manager on Mac OS. Windows contributors should download a binary distribution installer from the main website.

tox is a Python project and can be installed with pip:

pip install tox

Finally, you should fork and clone the cocotb repo. This will allow you to make changes to the cocotb source code, and run regressions and build documentation locally.

Now you are ready to contribute!

Running Tests Locally

First, set up your development environment.

Our tests are managed by tox, which runs both pytest tests and our system of makefiles. The regression does not end on the first failure, but continues until all tests in the /tests and /examples directories have been run.

To run the tests locally with tox, you will need to select an appropriate test environment. Valid test environments are formatted as {your python version}-{your OS}. Valid python version values are py35, py36, py37, py38, or py39; and valid OS values are linux, macos, or windows. For example, a valid test environment is py38-linux. You can see the list of valid test environments by running the below command:

tox -l

Once you know the test environment you wish to use, call tox.

tox -e py38-linux

At the end of the regression, if there were any test failures, the tests that failed will be printed. Otherwise, tox will print a green :).

Selecting a Language and Simulator for Regression

tox supports the usage of the environment variables SIM and TOPLEVEL_LANG to select a simulator and language to run the regression. By default the tests will attempt to run with the Icarus Verilog simulator.

For example, if you wanted to run tests with GHDL on Linux with Python 3.8, you would issue the following command:

SIM=ghdl TOPLEVEL_LANG=vhdl tox -e py38-linux

Running Individual Tests Locally

Each test under /tests/test_cases/*/ and /examples/*/tests/ can be run individually. This is particularly useful if you want to run a particular test that fails the regression.

First you must install cocotb from source by navigating to the project root directory and issuing the following command:

python -m pip install .

On Windows, you must instead install cocotb from source like so:

python -m pip install --global-option build_ext --global-option --compiler=mingw32 .

Once that has been done, you can navigate to the directory containing the test you wish to run. Then you may issue an appropriate make command. For example, if you want to test with Icarus using Verilog sources:

make SIM=icarus TOPLEVEL_LANG=verilog

Building Documentation Locally

First, set up your development environment.

Documentation is built locally using tox. The last message in the output will contain a URL to the documentation you just built. Simply copy and paste the link into your browser to view it. The documentation will be built in the same location on your hard drive on every run, so you only have to refresh the page to see new changes.

To build the documentation locally on Linux or Mac, issue the following command:

tox -e docs

Building the documentation is not currently supported on Windows.

Architecture and Scope of Cocotb

Cocotb has seen adoption in a wide variety of scenarios with sometimes conflicting requirements. To foster experimentation and to decentralize the development process the architecture of cocotb is highly modular. A solid core forms the foundation upon which extensions can provide higher-level functionality.

The core of cocotb are

  • the infrastructure to write testbenches with coroutines, threads, etc.,
  • the abstraction and interaction with simulators through interfaces like VPI, GPI, etc.,
  • tooling to run tests, and
  • core primitives to interact with the simulation: triggers, data access classes, etc.

As a general rule, functionality beyond this core set should go into extensions. However, none of these rules are set in stone. They can and should be challenged at times to ensure the project stays relevant to the majority of its users.

Maintainer Pre-approval

After making changes to cocotb, changes must be approved by at least one maintainer before being included. Out-of-scope and breaking changes will not be accepted. Also a maintainer could object to a change due to implementation approach or code quality reasons. To potentially save you frustration and time, it is a good idea to get maintainer pre-approval on the task before starting it.

The best way to get maintainer pre-approval is to make a Github issue. These issues can be a place for maintainers, as well as other users, to voice opinions on a proposed change before the task is worked. You may also propose changes on the Gitter channel or by directly contacting a maintainer.

How to Get Changes Merged

Have you fixed a bug in cocotb, or want to add new functionality to it? Cocotb follows the typical GitHub flow and makes use of pull requests and reviews. Follow the steps below to get your changes merged, i.e. integrated into the main cocotb codebase.

  1. Create an issue ticket on cocotb's GitHub issue tracker describing the problem. Issues are also a good place to discuss design options with others before writing code.
  2. Fork the cocotb GitHub repository into your personal namespace.
  3. Create a new branch off the master branch for your set of changes. Use one branch per "topic," i.e. per set of changes which belong together.
  4. Create one or multiple commits to address the issue. Make sure to read and follow the Patch Requirements when preparing your commits.
  5. Create new pull request (PR).
  6. When you submit (or update) the pull request, a suite of regression tests will run. If any of them turns "red," i.e. reports a failure, you most likely need to fix your code before it can be merged.
  7. The pull request needs to be reviewed by at least one maintainer. We aim to give feedback to all pull requests within a week, but as so often, life can get in the way. If you receive no feedback from a maintainer within that time, please contact them directly (e.g. on Gitter or email). You can find a list of all maintainers below. If a maintainer asks you to explain or modify code, try to do so.
  8. Once your code has at least one positive review from a maintainer and no maintainer strongly objects it your code is ready to be merged into the master branch.

Patch Requirements

All changes which should go into the main codebase of cocotb must follow this set of requirements.

  • The code must be within the scope of cocotb.
  • All code must be licensed under the Revised BSD License. By contributing to this project you signal your agreement with these license terms.
  • All code must follow the established coding standards. For Python code, follow the PEP 8 style guide.
  • All code must pass existing tests. New functionality must be accompanied by tests, and bug fixes should add tests to increase the test coverage and prevent regressions.
  • If code changes or enhances documented behavior the documentation should be updated.
  • If a change is user-visible, a newsfragment should be added to documentation/source/newsfragments.
  • All pull requests must be accepted by at least one maintainer, with no maintainer strongly objecting. Reviews must be performed by a person other than the primary author of the code.
  • All commits should follow established best practices when creating a commit message:
    • The first line of the commit message is the short summary of what the code change does. Keep this line below 50 characters.
    • Then have one blank line.
    • Now comes the long description of the commit. Use this text to discuss things which are not obvious from the code, especially why changes were made. Include the GitHub issue number (if one exists) in the form "Fixes #nnn" (read more about that). Keep each description line below 72 characters.
  • Use the following header for new non-example files: 
    # Copyright cocotb contributors
# Licensed under the Revised BSD License, see LICENSE for details.
# SPDX-License-Identifier: BSD-3-Clause
  • Use the following header for new example files: 
    # This file is public domain, it can be freely copied without restrictions.
# SPDX-License-Identifier: CC0-1.0

Managing of Issues and Pull Requests

The cocotb project makes use of GitHub labels attached to issues and pull requests to structure the development process. Each issue and pull request can have multiple labels assigned.

The type labels define the type of issue or PR:

  • type:bug: a bug in existing functionality
  • type:feature: new functionality
  • type:question: a support question
  • type:cleanup: cleanup or refactoring on code, documentation, or other areas
  • type:deprecation: API that should warn and eventually be removed

The status labels give a quick impression of the current status of the issue or PR:

  • status:worksforme: the issue it not reproducible, or intended behavior (i.e. not a bug)
  • status:blocked: further progress is blocked by a dependency, e.g. other code which must be commited first.
  • status:needs-info: feedback from someone is required. The issue/PR text gives more details.
  • status:duplicate: the same issue is already being handled in another issue/PR.
  • status:close?: issues which can probably be closed, but need a second pair of eyes
  • status:needs-proprietary-testing: Help needed testing on a proprietary tool
  • status:out-of-scope: An issue or PR that was closed because the feature or bug was deemed to be out of scope

For the use in pull requests the following additional status labels are defined:

  • status:needs-review: this PR needs at least one review
  • status:changes-requested: changes are requested to the code
  • status:ready-for-merge: this PR is ready (according to the Patch Requirements) to be merged
  • status:needs-rebase: needs a git rebase
  • status:needs-newsfragment: Needs a towncrier newsfragment for the changelog

The category labels help maintainers to filter issues which are relevant to their area of expertise:

  • category:OS:MacOS: Mac OS/OS X specific issues
  • category:OS:Linux: Linux specific issues
  • category:OS:Windows: Microsoft Windows-specific issues
  • category:simulators: simulator support, including VPI/GPI/etc.
  • category:simulators:activehdl: Aldec Active-HDL
  • category:simulators:cvc: Tachyon CVC
  • category:simulators:ghdl: GHDL
  • category:simulators:icarus: Icarus Verilog (iverilog)
  • category:simulators:ius: Cadence Incisive (IUS)
  • category:simulators:modelsim: Mentor Modelsim
  • category:simulators:questa: Mentor Questa
  • category:simulators:riviera: Aldec Riviera-PRO
  • category:simulators:vcs: Synopsys VCS
  • category:simulators:verilator: Verilator
  • category:simulators:xcelium: Cadence Xcelium
  • category:codebase:gpi: relating to the GPI or one of the implementation
  • category:codebase:pygpi: relating to the Python wrapper around the GPI (embed library and simulator module)
  • category:codebase:scheduler: relating to the coroutine scheduler, triggers, or coroutine objects
  • category:codebase:test-runner: relating to code for automating test runs (regression manager)
  • category:codebase:handle: relating to handles or handle types (BinaryValue)
  • category:codebase:project-automation: relating to included project automation (makefiles)
  • category:codebase:testbenching: relating to testbenching components (Drivers, Monitors, etc.)
  • category:building: relating to build C/C++ libraries and extension modules
  • category:packaging: issues related to (PyPi) packaging, etc.
  • category:docs: documentation issues and fixes
  • category:extensions: cocotb extensions
  • category:performance: performance topics
  • category:tests-ci: continuous integration and unit tests

To help new contributors find a good issue to work on one more label is used (following GitHub standard practices):

  • good first issue: this issue is a good starting point for new contributors. The issue should give an actionable description of what to do to complete this task, along with contact information of a mentor for this task.

cocotb explicitly uses no priority labels, as experience indicates that they provide little value.

Issues and pull requests which are invalid, or where feedback is lacking for four weeks, should be closed.

Cocotb Releases

cocotb aims to keep the master branch always in a releasable state. At least four times a year an official release should be created. It is the job of the maintainers to find a suitable time for a release, to communicate it to the community, and to coordinate it.

Maintainers

Cocotb uses a shared maintainer model. Most maintainers are experts in part of the cocotb codebase, and are primarily responsible for reviews in this area.

  • Kaleb Barrett (@ktbarrett)
  • Tomasz Hemperek (@themperek)
  • Marlon James (@marlonjames)
  • Colin Marquardt (@cmarqu)
  • Philipp Wagner (@imphil)
  • Eric Wieser (@eric-wieser)

Founders

  • Chris Higgs (@chiggs)
  • Stuart Hodgson (@stuarthodgson)

Getting in Contact with a Maintainer

All of the maintainers are active on the Gitter channel. They prefer inquiries go through direct messages on Gitter, or by mentioning them in the main cocotb Gitter channel using @{maintainer name}. Maintainers are unpaid volunteers, so it might take a while for a maintainer to get back to you.

Code of Conduct

The cocotb development community aims to be welcoming to everyone. The FOSSi Foundation Code of Conduct applies. Please contact any of the maintainers if you feel uncomfortable in the cocotb development community.