Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Certificate bundle upgrade (IDFGH-12148) #13204

Closed
wants to merge 3 commits into from
Closed

Certificate bundle upgrade (IDFGH-12148) #13204

wants to merge 3 commits into from

Conversation

BitsForPeople
Copy link
Contributor

Modified gen_crt_bundle.py and esp_crt_bundle.c to build/use a static 'index' of the certificates in a bundle and include this index data in the bundle data.
This obviates the need for esp_crt_bundle.c to allocate heap memory and build the index at runtime, saving ~120x4 bytes of heap for the full/'unfiltered' bundle.
Changed "mbedtls_x509_crt parent" in esp_crt_check_signature() to "mbedtls_pk_context pubkey" to save some 200 bytes of stack requirement.

@BitsForPeople
Reduced RAM & stack use of cert bundle
8d50f09
@BitsForPeople
Reduced RAM & stack use of cert bundle
263a16a
@BitsForPeople
Merge branch 'crt_bundle_upgrade' of https://github.com/BitsForPeople…
c38b75f 
…/esp-idf into crt_bundle_upgrade

# Conflicts:
#	components/mbedtls/esp_crt_bundle/esp_crt_bundle.c
#	components/mbedtls/esp_crt_bundle/gen_crt_bundle.py
@github-actions GitHub Actions
Copy link

Warnings
⚠️

Some issues found for the commit messages in this PR:

  • the commit message "Reduced RAM & stack use of cert bundle":
    • summary looks empty
    • type/action looks empty
  • the commit message "Reduced RAM & stack use of cert bundle":
    • summary looks empty
    • type/action looks empty

Please fix these commit messages - here are some basic tips:

  • follow Conventional Commits style
  • correct format of commit message should be: <type/action>(<scope/component>): <summary>, for example fix(esp32): Fixed startup timeout issue
  • allowed types are: change,ci,docs,feat,fix,refactor,remove,revert,test
  • sufficiently descriptive message summary should be between 20 to 72 characters and start with upper case letter
  • avoid Jira references in commit messages (unavailable/irrelevant for our customers)

TIP: Install pre-commit hooks and run this check when committing (uses the Conventional Precommit Linter).
Messages
📖 You might consider squashing your 3 commits (simplifying branch history).

👋 Hello BitsForPeople, we appreciate your contribution to this project!

📘 Please review the project's Contributions Guide for key guidelines on code, documentation, testing, and more.
🖊️ Please also make sure you have read and signed the Contributor License Agreement for this project.
Click to see more instructions ...


This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Resolve all warnings (⚠️ ) before requesting a review from human reviewers - they will appreciate it.
- Addressing info messages (📖) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...


We do welcome contributions in the form of bug reports, feature requests and pull requests via this public GitHub repository.

This GitHub project is public mirror of our internal git repository

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved, we synchronize it into our internal git repository.
4. In the internal git repository we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
5. If the change is approved and passes the tests it is merged into the default branch.
5. On next sync from the internal git repository merged change will appear in this public GitHub repository.

Generated by 🚫 dangerJS against c38b75f

@espressif-bot espressif-bot added the Status: Opened Issue is new label Feb 18, 2024
@github-actions github-actions bot changed the title Certificate bundle upgrade Certificate bundle upgrade (IDFGH-12148) Feb 18, 2024
@mahavirj mahavirj requested a review from ESP-Marius February 19, 2024 07:11
@espressif-bot espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Feb 28, 2024
@Harshal5
Copy link
Collaborator

Harshal5 commented Mar 5, 2024

Hello @BitsForPeople,

It looks like you have made some changes in the mbedtls submodule too. Could you please confirm the changes that were needed in it? Also, it would be preferred if the submodule changes could be avoided as I think this certificate bundle format change would not need changes in the submodule itself.

@BitsForPeople
Copy link
Contributor Author

BitsForPeople commented Mar 7, 2024 •
edited
Loading

Hello @Harshal5 .
You're right in that no changes to mbedtls itself are needed. - And nothing in mbedtls should actually be included in this PR.
Is the PR salvagable? What can I do?

@Harshal5
Copy link
Collaborator

@BitsForPeople Also I think we could squash the three commits into one. You could follow the below steps:

  1. Soft reset the three commits using the command: git reset --soft c38b75fa31b2861ff8a037597c7abd2b05a5b7e1^.
  2. Then restore the staged mbedtls changes using the command: git restore --staged components/mbedtls/mbedtls
  3. Now, the mbedtls changes need to be reset. Change directory to the mbedtls submodule and reset the changes using the following commands:
cd components/mbedtls/mbedtls
git reset --hard HEAD
  1. Finally move back to the ESP-IDF git directory and commit the changes using git commit and maybe you could use the commit message as "feat(mbedtls/crt_bundle): Store cert indices to reduce memory usage".

Let me know in case you feel this is a bit complicated, I would be happy to help you by carrying this out myself.

@BitsForPeople
Copy link
Contributor Author

@Harshal5, thank you for providing the steps. Didn't work unfortunately, been having issues with git on my (Windows) machine for some time.
Would you be so kind and 'fix' the PR?

@Harshal5
Copy link
Collaborator

sha=c38b75fa31b2861ff8a037597c7abd2b05a5b7e1

@Harshal5 Harshal5 added PR-Sync-Merge Pull request sync as merge commit and removed PR-Sync-Merge Pull request sync as merge commit labels Mar 13, 2024
Harshal5
Harshal5 reviewed Aug 19, 2024
View reviewed changes
components/mbedtls/esp_crt_bundle/esp_crt_bundle.c
Comment on lines +207 to +214
// Issuers are in DER encoding, with lengths encoded in the content; if valid DER, differing lengths
// are reflected in differing content.
// Still, we won't try to memcmp beyond the given length:
int cmp_res = memcmp(issuer, esp_crt_get_name(cert), min(issuer_len, cert_name_len) );

if ( unlikely( cmp_res == 0 ) ) {
cmp_res = (int)issuer_len - cert_name_len;
if( likely( cmp_res >= 0 ) ) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @BitsForPeople, could you please explain why we allow the issuer name to be longer than the found certificate name? Do you have any example wherein this would be needed?

We think due to this, an incorrect certificate could get matched if the name of another certificate is a prefix of the name of the certificate we're looking for. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Harshal5.
I'm actually not sure anymore why I did a >= 0 here, t.b.h. Specifically, I can't remember seeing a case where this would be needed.
Reviewing it now, I believe you are right: If one cert's name would be a prefix of another cert's name in a bundle, we might (in theory) pick the wrong cert.
I think changing it to == 0 should work.

Sorry for seemingly not getting back to this PR until now. I spent a few hours trying to fix the issue of the unrelated mbedtls files in the PR but could not for the life of me get my working tree back to a consistent state with a clean PR.
Is there something you can do with the PR on your end?

@espressif-bot espressif-bot added Status: Done Issue is done internally Resolution: NA Issue resolution is unavailable and removed Status: In Progress Work is in progress labels Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Harshal5 Harshal5 Harshal5 left review comments

@ESP-Marius ESP-Marius Awaiting requested review from ESP-Marius

Assignees

@Harshal5 Harshal5

Labels
PR-Sync-Merge Pull request sync as merge commit Resolution: NA Issue resolution is unavailable Status: Done Issue is done internally
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@BitsForPeople @Harshal5 @espressif-bot