[RFC] Adopt the standard for public code

[bonjourmauko]

Adopt the standard for public code

Status	Proposed
Author	Mauko Quiroga ([email protected])
Sponsor	Foundation for Public Code ([email protected])
Created	2021-04-14
Updated	2021-07-01
Closing on	2021-10-01

Intent

See if we would like to adopt the Standard for Public Code, after discussing and identifying what we aspire to and what does and doesn't apply to OpenFisca.

The questions as I see them are:

• Do we aspire for OpenFisca to be public code to write better public algorithms (see definitions below)?
• Would the standard for public code help us to get there?

Definitions

The Standard for Public Code is a set of criteria that supports public organizations in developing and maintaining software and policy together.

Public Code is both computer source code (such as software and algorithms) and public policy executed in a public context, by humans or machines, that can be considered vital public infrastructure, not just the expression of existing policy but the originator of new policy, and should therefore be subject to the principles of democratic governance.

OpenFisca is an open source platform to write rules as code. It allows to describe a tax & benefit system, provide a situation as input (i.e income), ask for a calculation as output (i.e. income tax), and get the results. Current uses of OpenFisca include simulating individual situation tax & benefits, populations tax & benefits, and the impact of public policy reforms.

Rules as Code rethinks one of the core functions of government: rulemaking. It proposes to create an official, machine-consumable version of some types of government rules, to exist alongside the existing natural language counterpart. It represents a transformational shift in how governments create rules, and how third parties consume them.

Democratic Accountability states that governments and the people they serve need technological sovereignty – allowing them to set and control the functioning of public software, just like they are able to set and control policy that is legally formulated in laws.

Public Algorithm is an algorithm that operates in the service of the general interest, are used to execute the law, to enforce the law, and are often unavoidable, that is to say that there are no alternatives for users. They are forms of public action and as such are subject to the same form of algorithmic accountability.

Algorithmic Accountability means for public administrations to signal when an algorithm is used, to describe it, to justify its use, to explain its effects, both individual and general, to make it accessible, by publishing the source code and the associated documentation, and to allow for the algorithm to be challenged, indicating the way for users to appeal.

Adoption process

From the Standard for Public Code:

Every time a contribution is suggested to a codebase – through for instance a merge request – the codebase stewards of our organization will audit the contribution for compliance with the Standard for Public Code.

New contributions can only be adopted into the codebase after they have been approved as compliant with the Standard for Public Code, and have been reviewed by another contributor.

The audit is presented as a review of the contribution. The codebase steward gives line by line feedback and compliance, helping the contributor to improve their contribution. The merge request cannot be fulfilled until the codebase stewards have approved the contribution.

Nomenclature

✅ - Complies
❌ - Does not comply
🚫 - Does not apply
❓ - Not sure how to apply

Criteria

Code in the open

• ✅ All source code for any policy and software in use (unless used for fraud detection) MUST be published and publicly accessible.
• ✅ Contributors MUST NOT upload sensitive information regarding users, their organization or third parties to the repository.
• ✅ Any source code not currently in use (such as new versions, proposals or older versions) SHOULD be published.
• 🚫 The source code MAY provide the general public with insight into which source code or policy underpins any specific interaction they have with your organization.
  • The core of OpenFisca is an abstraction to allow writing and executing legislation as code to any country, so it is not specific to any organisation's policy.

Bundle policy and source code

• 🚫 A codebase MUST include the policy that the source code is based on.
  • The core of OpenFisca is an abstraction to allow writing and executing legislation as code to any country, so it is not specific to any organisation's policy.
• 🚫 A codebase MUST include all source code that the policy is based on.
  • The core of OpenFisca is an abstraction to allow writing and executing legislation as code to any country, so it is not specific to any organisation's policy.
• ❌ All policy and source code that the codebase is based on MUST be documented, reusable and portable.
  • Several important parts of the source code are not yet properly documented.
  • Most of the source code is reusable.
  • All of the source code is portable.
• 🚫 Policy SHOULD be provided in machine readable and unambiguous formats.
  • The core of OpenFisca is an abstraction to allow writing and executing legislation as code to any country, so it is not specific to any organisation's policy.
• 🚫 Continuous integration tests SHOULD validate that the source code and the policy are executed coherently.
  • The core of OpenFisca is an abstraction to allow writing and executing legislation as code to any country, so it is not specific to any organisation's policy.

Create reusable and portable code

• ✅ The codebase MUST be developed to be reusable in different contexts.
• ✅ The codebase MUST be independent from any secret, undisclosed, proprietary or non-open licensed code or services for execution and understanding.
• ✅ The codebase SHOULD be in use by multiple parties.
• ❌ The roadmap SHOULD be influenced by the needs of multiple parties.
  • There's not yet a clear and stakeholder-representative roadmap.
• ✅ Configuration SHOULD be used to make code adapt to context specific needs.
• ❌ Codebases SHOULD include a publiccode.yml metadata description so that they’re easily discoverable.
  • It does not.
• ✅ Code and its documentation SHOULD NOT contain situation-specific information.

Welcome contributors

• ✅ The codebase MUST allow anyone to submit suggestions for changes to the codebase.
• ✅ The codebase MUST include contribution guidelines explaining how contributors can get involved, for example in a file.
• ❌ The codebase SHOULD advertise the committed engagement of involved organizations in the development and maintenance.
  • It does not.
• ❌ The codebase SHOULD document the governance of the codebase, contributions and its community, for example in a GOVERNANCE file.
  • It does not.
• ❌ The codebase SHOULD have a publicly available roadmap.
  • It does not.
  • See Have a clear product/technical roadmap for OpenFisca Core #942
• ❌ The codebase MAY include a code of conduct for contributors.
  • It does not.

Make contributing easy

• ✅ The codebase MUST have a public issue tracker that accepts suggestions from anyone.
• 🚫 The codebase MUST include instructions for how to privately report security issues for responsible disclosure.
  • There are no potential security issues related to the codebase that would require private report.
• ✅ The documentation MUST link to both the public issue tracker and submitted codebase changes, for example in a file.
• ✅ The codebase MUST have communication channels for users and developers, for example email lists.
• 🚫 The documentation SHOULD include instructions for how to report potentially security sensitive issues on a closed channel.
  • There are no potential security issues related to the codebase that would require private report.

Maintain version control

• ✅ You MUST have a way to maintain version control for the code.
• ✅ All files in a codebase MUST be version controlled.
• ✅ All decisions MUST be documented in commit messages.
• ❌ Every commit message MUST link to discussions and issues wherever possible.
  • There is no consistency in terms of commits, they tend to be atomic but it is not always the case.
  • All merge commits are linked to discussions.
  • Not all merge commits are linked to issues.
• ✅ You SHOULD use a distributed version control system.
• ✅ You SHOULD group relevant changes in commits.
• ✅ You SHOULD mark different versions of the codebase, for example using revision tags or textual labels.
• ✅ You SHOULD prefer file formats where the changes within the files can be easily viewed and understood in the version control system.

Require review of contributions

• ✅ All contributions that are accepted or committed to release versions of the codebase MUST be reviewed by another contributor.
• ✅ Reviews MUST include source, policy, tests and documentation.
• ✅ Reviewers MUST provide feedback on all decisions to not accept a contribution.
• ❌ Contributions SHOULD conform to the standards, architecture and decisions set out in the codebase in order to pass review.
  • There are no standards nor maintenance guidelines so review decisions are basically arbitrary.
  • See Have a clear process for maintenance/contribution #1004
• ✅ Reviews SHOULD include running both the code and the tests of the codebase.
• ❌ Contributions SHOULD be reviewed by someone in a different context than the contributor.
  • It is not always the case as there is no explicit list of maintainers.
  • In order to get a review it happens more often than not to reach people privately in the context of the contributor in order to get a review.
• ✅ Version control systems SHOULD not accept non-reviewed contributions in release versions.
• ❌ Reviews SHOULD happen within two business days.
  • They could take years.
  • Because of lack of governance and clear guidelines there are contributions that never get reviewed.
  • See Have a clear process for maintenance/contribution #1004
• ✅ Reviews MAY be performed by multiple reviewers.

Document your objectives

• ✅ The codebase MUST contain documentation of its objectives – like a mission and goal statement – that is understandable by designers and developers so that they can use or contribute to the codebase.
• ❌ The codebase SHOULD contain documentation of its objectives understandable by policy makers and management.
  • Some work has been done but documentation is still very technical and hard to grasp for policy-makers and management.
• ❌ The codebase MAY contain documentation of its objectives for the general public.
  • Some work has been done but documentation is still very technical and hard to grasp for the general public.

Document the code

• ❌ All of the functionality of the codebase – policy as well as source – MUST be described in language clearly understandable for those that understand the purpose of the code.
  • There are parts of the code that are very cryptic, untested and undocumented.
• The documentation of the codebase MUST contain:
  • ✅ a description of how to install and run the source code,
  • ✅ examples demonstrating the key functionality.
• The documentation of the codebase SHOULD contain:
  • ❌ a high level description that is clearly understandable for a wide audience of stakeholders, like the general public and journalists,
    • Some work has been done but documentation is still very technical and hard to grasp for the general public and for journalists.
  • ✅ a section describing how to install and run a standalone version of the source code, including, if necessary, a test dataset,
  • ❌ examples for all functionality.
    • A great deal has been done but it is still incomplete.
    • Some examples are outdated and/or incorrect (invalid syntax, bad signatures, and so on...)
• ❓ There SHOULD be continuous integration tests for the quality of your documentation.
  • Documentation loosely is checked for validity to avoid broken releases.
  • Documentation quality is not checked.
• ❓ The documentation of the codebase MAY contain examples that make users want to immediately start using the codebase.
  • The documentation does contain ready to use examples.
• ✅ You MAY use the examples in your documentation to test the code.

Use plain English

• ❌ All code and documentation MUST be in English.
  • There's still some French.
• 🚫 Any translation MUST be up to date with the English version and vice versa.
• ❌ There SHOULD be no acronyms, abbreviations, puns or legal/domain specific terms in the codebase without an explanation preceding it or a link to an explanation.
  • There are some.
• ✅ The name of the codebase SHOULD be descriptive and free from acronyms, abbreviations, puns or branding.
• ❌ Documentation SHOULD aim for a lower secondary education reading level, as recommended by the Web Content Accessibility Guidelines 2.
  • It does not.
• 🚫 Any code, documentation and tests MAY have a translation.

Use open standards

• ✅ For features of a codebase that facilitate the exchange of data the codebase MUST use an open standard that meets the Open Source Initiative Open Standard Requirements.
• 🚫 If no existing open standard is available, effort SHOULD be put into developing one.
• Standards that are machine testable SHOULD be preferred over those that are not.
• 🚫 Functionality using features from a standard that is not an open standard MAY be provided if necessary, but only in addition to compliant features.
• 🚫 All non-compliant standards used MUST be recorded clearly in the documentation.
• ✅ The codebase SHOULD contain a list of all the standards used with links to where they are available.

Use continuous integration

• ❌ All functionality in the source code MUST have automated tests.
  • It does not.
• ✅ Contributions MUST pass all automated tests before they are admitted into the codebase.
• ❓ Contributions MUST be small.
  • How do you define small?
  • Contributions are not always atomic.
• ✅ The codebase MUST have active contributors.
• ❌ Source code test and documentation coverage SHOULD be monitored.
  • Source code coverage is monitored but there is no coverage policy.
  • Documentation coverage is not monitored.
• 🚫 Policy and documentation MAY have testing for consistency with the source and vice versa.
• ✅ Policy and documentation MAY have testing for style and broken links.

Publish with an open license

• ✅ All code and documentation MUST be licensed such that it may be freely reusable, changeable and redistributable.
• ✅ Software source code MUST be licensed under an OSI-approved open source license.
• ✅ All code MUST be published with a license file.
• ✅ Contributors MUST NOT be required to transfer copyright of their contributions to the codebase.
• ❓ All source code files in the codebase SHOULD include a copyright notice and a license header that are machine readable.
  • Why?
• 🚫 Codebases MAY have multiple licenses for different types of code and documentation.
  • All of the codebase and by transitivity the codebases depended upon are AGPL-3.0 licensed.
  • As there's no CLA or any other type of contributor's copyright transfer it does not seem even feasible at this point.

Use a coherent style

• ❌ Contributions MUST adhere to either a coding or writing style guide, either your own or an existing one that is advertised in or part of the codebase.
  • There's some style enforced but it is very loose which creates a lot of frustration at review time.
  • There are some explicit and implicit styles that are not enforced.
  • See https://github.com/openfisca/openfisca-core#style
  • See https://github.com/openfisca/openfisca-core/blob/master/STYLEGUIDE.md
• ✅ Contributions SHOULD pass automated tests on style.
• ❌ Your codebase SHOULD include inline comments and documentation for non-trivial sections.
  • Not enforced.
• ✅ You MAY include sections in your style guide on understandable English.

Document codebase maturity

• ✅ A codebase MUST be versioned.
• ✅ A codebase that is ready to use MUST only depend on other codebases that are also ready to use.
• ✅ A codebase that is not yet ready to use MUST have one of these labels:
  • prototype - to test the look and feel, and to internally prove the concept of the technical possibilities,
  • alpha - to do guided tests with a limited set of users,
  • beta - to open up testing to a larger section of the general public, for example to test if the codebase works at scale,
  • pre-release version - code that is ready to be released but hasn’t received formal approval yet.
• ✅ A codebase SHOULD contain a log of changes from version to version, for example in the CHANGELOG .

TODO

• Stated intent
• Provided definitions
• Described the adoption process
• Run the checklist to see what already complies and what doesn't
• Of what doesn't, check what do actually apply
• Make a proper RFC of those elements
• Summarise discussions with the Foundation for Public Code
• Propose a long-term roadmap to make the RFC more actionable

[ericherman]

This is an excellent analysis.

Having reviewed the gaps and question marks, it seems that the codebase is already in pretty good shape and that the gaps are largely in areas where it is easy to imagine how to close them. Easy to imagine does not mean easy to do, I'm sure some of these represent a lot of work.

If it would help the community to make a decision on this topic, I'm happy to host a call where we discuss the open questions and the gaps.

[bonjourmauko]

Review in progress here: https://hackmd.io/Kny0vCAbRu-wTyBuhKj2Gg?both

[ericherman]

Existing open issues related to Use Plain English:

• Add nuance on where non-English is acceptable #223
• Clarify intersection of policy-defined vocabulary and Use Plain English #308

[ericherman]

Two pull requests on the Standard for Public Code have been opened:

• https://github.com/publiccodenet/standard/pull/527
• https://github.com/publiccodenet/standard/pull/528

Reviews welcome.

Ping: @maukoquiroga

[Ainali]

Two pull requests on the Standard for Public Code have been opened:

• Issue #223: Add nuance on where non-English is acceptable publiccodenet/standard#527
• Issue #308: Clarify intersection of policy-defined vocabulary and Use Plain English publiccodenet/standard#528

Just a note that we will discuss these PR's on the community call on Thursday, 14.00 UTC.

[Ainali]

Just a note that we will discuss these PR's on the community call on Thursday, 14.00 UTC.

Here are the notes from the community call. We had a good discussion and think we addressed the concerns in good spirit and have implemented the changes we developed to the pull requests.

[ericherman]

Happy New Year!

Is there a way in which the Foundation for Public Code can help move this issue forward?
If so, please ping me or @Ainali and we can help.

Regardless, when is a good time to have the next conf-call in which this issue could be discussed?
@maukoquiroga will you be the one to send a new invite? Or should I be pinging someone else?

[bzg]

Hi! Just allowing myself to ping @MattiSG on this one, if that helps.

[Ainali]

We have now released version 0.2.3 that includes the changes mentioned above. Find it here: https://github.com/publiccodenet/standard/releases/tag/0.2.3-1

Does this alleviate the concerns about English in policy and code?

[bonjourmauko]

Hi all! As I understand this RFC came to a halt. I'm closing the issue but please do not hesitate to reopen it if I'm closing it by mistake or if there are new forthcoming elements.

[MattiSG]

We are still very interested in adopting the Standard. Your work @maukoquiroga in assessing through this checklist has been extremely relevant and useful, and @Ainali and @ericherman have been very supportive.

We currently have a good understanding of how we could proceed further with @publiccodenet:

1. Update checklist.
2. Keep track of missing points and obtain community consensus on fulfilling them in the future.
3. Officially adopt the Standard in the sense of the above.
4. Benefit from @publiccodenet support, at least in the form of participation to pull requests reviews for governance conformity, and as a third party for checking compliance with the Standard.

I have maintained the connection throughout this year and am happy with the offer outlined above as I believe it can be a useful tool in ensuring governance is enforced and improved over time, no matter who operates OpenFisca.

The main difficulty encountered, which explains the (more visible than it actually is, but still true) stagnation on this topic, is the lack of resources secured for setting up and maintaining this partnership. I have been working significantly on fundraising in the last 2 months and hope to secure a budget for several topics, including this one, in 2023.

At this stage, this topic is in my view very slow, but not abandoned. I will reopen this issue to enable tracking its progress 🙂

[bzg]

At this stage, this topic is in my view very slow, but not abandoned.

Music to my ears! Of tangential/potentiel interest: - https://code.gouv.fr/#/repos now allows to filter repositories containing a publiccode.yml file. - The good people at https://publiccode.net published this interactive "eligibility assessment" that may also help: https://publiccodenet.github.io/assessment-eligibility/ 2 cts.

[ericherman]

The Standard for Public Code has been updated to version 0.7.1, and we have updated the review template, which is also available in PDF

We would like to fill in the template for OpenFisca as best we can, and then get someone more familiar with the codebase to join us for a 1/2 hour (approximately) call to complete it. Who could be available for that kind of call?

[MattiSG]

@ericherman thank you for this suggestion! Is that half an hour or one to two hours? :)

In any case, I would happily join this call later this month.

[Ainali]

That is meant as 0,5 hours!

[ericherman]

Sorry! A 30-minutes (approximately) call.