Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matter 1 3 upmerge #435

Merged
merged 1,601 commits into from
May 15, 2024
Merged

Conversation

kkasperczyk-no
Copy link
Contributor

Regular Matter fork upmerge to the 1.3.0.0 tag

bzbarsky-apple and others added 30 commits March 14, 2024 18:45
This fixes the endpoint_config representation of "endpoint with no specific
parent endpoint set" to match the SDK internal representation, so we can remove
some code that was working around the representation mismatch.
…. (#32581)

Because we were storing the PeerAddress in the session when getting a message,
we effectively pinned sessions to particular interface ids at that point.

This can lead to routing failures.  We should only be pinning to interface IDs
for link-local addresses, just like we do for initial IP resolution via DNS-SD.
…rs-app. (#32573)

The cluster implementation does those checks.
* Normalize BT MAC letter case passed to DiscoverAsync

* Remove obsolete docstrings

* [Python] Delete BLE scanner handle in an explicit way

* Update callback names

* Get rid of obsolete comments

* Unref manager client on glib thread
* Implement Android check in delegate

* Restyled by whitespace

* Restyled by google-java-format

* Restyled by clang-format

* Update documentation

* Restyled by whitespace

* Restyled by google-java-format

* Fix jni object reference issue, add chiptool callback

* Restyled by google-java-format

* Restyled by clang-format

* Update kotlin codestyle

* remove public

* fix typo

* Restyled by clang-format

* remove chip

* Modify from comment

---------

Co-authored-by: Restyled.io <[email protected]>
* Update Chef Lock to enable user PIN and more

* Remove unused clusters (Groups). Add Power Source

* Refer to the latest lock-app codes

* Update device PowerSource attributes

* Enable PrivacyMode, Remove debug message

* Fix typo

* Restyled by whitespace

* Restyled by clang-format

* Add macro to doorlock to avoid compilation of light

* [Chef] support doorlock in esp32 & nrfconnect

* Update the date from 2023 to 2024

* Use CharSpan instead of "char *". Refine ifdef

* Restyled by clang-format

---------

Co-authored-by: Restyled.io <[email protected]>
…. (#32578)

This way when the next DST transition happens, the device won't be confused
about the local time.
* Add ruff as Python linter

* Add ruff to linter workflow

* Remove flake8

* Set default line-length and explicitly select rules

* Address REPL Test issue

* Explicitly set line lenght for autopep8

* Fix restyled config
…e README.md (#32254)

* Update Matter Linux Air Quality Example README.md

* restyled

* Update README.md

* Update README.md

* restyled
* Implement a Vector-based TLVBackingStore for converting from stream-based input to TLV format

* :Address comments

* Fix for esp32 qemu build
* Switch subscription resumption test app to lit-icd-app

* review changes
* [Darwin] Adding metrics for MTRDeviceController APIs

- Adding initial set of metrics for MTRDeviceController APIs around
  device commissioning
- Added uniqueIdentifer to MTRMetrics
- Changed MetricsCollector to removed the end event for begin / instant
  event

* Update src/darwin/Framework/CHIPTests/MTRMetricsTests.m

Co-authored-by: Karsten Sperling <[email protected]>

* Addressed code review feedback

* Fixing namespace naming to be consistent

* Re-styler fixes

---------

Co-authored-by: Karsten Sperling <[email protected]>
* Wifi event loop implementation

* Applied suggestions for event values, added comments for further todos to look into

* Addressed comments about status and pointer checks, added helper functions to reduce repeated code blocks
* [Linux] Do not report BLE scan error if BlueZ is gone

* Wrap GDBusObjectManager with GAutoPtr

* Wrap glib callbacks with lambda

* Wrap start/stop implementations with lambda

* Fix signal handler type
* Update github action versions

actions/upload-artifact@v4
Wandalen/[email protected]

* Make bootstrap-logs artifacts unique
…races. (#32035)

* Added a config option to set the permit list size for esp32 traces

- Fixes project-chip/connectedhomeip#31536

* Addressed review comments
Bumps [third_party/imgui/repo](https://github.com/ocornut/imgui) from `fbf45ad` to `3c435c0`.
- [Release notes](https://github.com/ocornut/imgui/releases)
- [Commits](ocornut/imgui@fbf45ad...3c435c0)

---
updated-dependencies:
- dependency-name: third_party/imgui/repo
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [third_party/ot-br-posix/repo](https://github.com/openthread/ot-br-posix) from `9bdaa91` to `bd15f71`.
- [Release notes](https://github.com/openthread/ot-br-posix/releases)
- [Commits](openthread/ot-br-posix@9bdaa91...bd15f71)

---
updated-dependencies:
- dependency-name: third_party/ot-br-posix/repo
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [third_party/pigweed/repo](https://github.com/google/pigweed) from `ea93eb5` to `1f12d06`.
- [Commits](google/pigweed@ea93eb5...1f12d06)

---
updated-dependencies:
- dependency-name: third_party/pigweed/repo
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The grammar for how command directions are inidcated changed in
the underlying files. This affected derived clusters. Test updated
to use the new grammar, meta-test updated.
* Spec parsing: Use new ID table

The ID table has now been added for every cluster, including alias
clusters. This means we can now just make direct copies of the alias
clusters as we parse the cluster file. It also means we can ignore
the top-level cluster tag, so we don't need to worry about the
new weird naming.

Not yet handled in the XML - pics for aliased clusters.

* Restyled by isort

---------

Co-authored-by: Restyled.io <[email protected]>
* TC-IDM-10.4: Catch feature mask errors and log

* Add comment
* Implement Android validate Device attestation

* Restyled by google-java-format

* Restyled by clang-format

* Update VendorId to uint16_t

* Update from comment

* Restyled by whitespace

* Restyled by google-java-format

* Restyled by clang-format

* Update kotlin codestyle

* Modify type casting

* Restyled by clang-format

---------

Co-authored-by: Restyled.io <[email protected]>
kkasperczyk-no and others added 19 commits May 8, 2024 10:26
Set for CI build NCS recommended version to origin/main
Add a manually triggered workflow for building Android
CHIPTool packages for arm and arm64 architectures,
CHIP Tool for Linux x64 and arm64, and OTA provider for
Linux.
The packages are uploaded both as github artifacts and
release packages (as the latter require a release tag).

The doxygen workflow pushes doxygen HTML files to another
branch which is undesired on the sdk-connectedhomeip fork
as it causes significant growth of the repository size.

Signed-off-by: Damian Krolik <[email protected]>
Signed-off-by: Kamil Kasperczyk <[email protected]>

Remove Android Chip Tool from release workflow

Android Chip Tool is no longer maintained so it must be
removed from the release workflow.

Remove dependency to openssl from CHIP tool build for linux

Currently the chip-tool that is released as a part of
sdk-connectedhomeip has dependency to openssl.
The issue is that Ubuntu 20.04 and Ubuntu 22.04 use different version
of OpenSSL and version built under 20.04 does not work under
22.04 (without some hacks to install the old openssl version)
and vice-verse.

- Removed the dependency to the dynamic library (OpenSSL) to build
Chip-Tool with the mbedTLS.

[nrf noup] Use "continue-on-error" for examples in CI

Make building nRF Connect examples optional in github
workflows in a sense that they do not fail the workflow
if they do not compile. The reason is that the CI is often
run for downstream patches, that are required to proceed
with changes in NCS or Zephyr and we do not want to force
developers to update examples in sdk-connectedhomeip at
that moment - they will be updated in the upstream later.

Signed-off-by: Damian Krolik <[email protected]>

[nrf noup] Conditionally disable all unnecessary workflows

Currently, in sdk-connectedhomeip we don't need to run
upstream's workflows for all platforms because we use
sdk-nrf workflows to verify our PRs.

We decided to remove all of them except Lint Code Base and
Unit Tests from the 'pull request' and 'push' triggers and
leave the possibility to run them manually.

This commit cannot be pushed to the Matter upstream repository!

[nrf noup] Disable dependabot

It runs on the upstream repository and updates dependencies accordingly.

[nrf noup] Repaired unit tests

Last changes and upmerges cause instabilities in our workflows
actions. This commit repairs unit tests.

[nrf noup] [nrfconnect] Fix release tools workflow

[nrf noup] Aligned release tools to Matter 1.2

Added all changes to release tools github workflow needed to work
with Matter 1.2.
Make chip_data_model.cmake correctly determine CHIP_ROOT
constant if that is not defined in the application CMake.

Also, use Python3 selected by CMake.

Signed-off-by: Damian Krolik <[email protected]>

[nrf noup] Bypassed code generation at build time

Allow to bypass codegen at build time.
Re-add removed callback-stub.cpp and plugin callback templates

Signed-off-by: Kamil Kasperczyk <[email protected]>
Signed-off-by: Damian Krolik <[email protected]>

[nrf noup] Fix the external cluster injection mechanism

The necessary change was reverted during the last upmerge.
Without this fix, the external cluster injection does not work.

Signed-off-by: Marcin Kajor <[email protected]>
pigweed_environment.gni file is included by GN scripts
used by Matter build system. The file is generated by
bootstrap.sh script but in nRF Connect SDK we do not want
to force user to run that script. Therefore, include a copy
of that file in tree.

Signed-off-by: Damian Krolik <[email protected]>
The helper script is going to find recommended ZAP version in the
Matter repository, and download and extract the package from the
ZAP releases.

Signed-off-by: Kamil Kasperczyk <[email protected]>
Signed-off-by: Michał Szablowski <[email protected]>

[nrf noup] Fixed output generated by get_zap.py on macOS

The zap executable location in script and printed path are not
correct.

[nrf noup] Fixed package names for zap binaries

The URLs for the ZAP tool binaries changed, so script for
downloading it had to be aligned.

Signed-off-by: Kamil Kasperczyk <[email protected]>

[nrf noup] Fix zap instalation for MacOs

Fix updating get_zap.py script, which ensure zap
will work on the MacOs systems.

Signed-off-by: Patryk Lipinski <[email protected]>
This commit fixes building on Windows PowerShell by removing usage of
"bash" command.

Signed-off-by: Lukasz Duda <[email protected]>
Usage of generate.py script is not possible on windows
due to fcntl package available onlu on unix pyhton packages
using I/O control on descriptor file is not needed on windows.
This change enhance script to use fcntl on non-window system.
Stop filtering out gdwarf-4 when passing flags from Zephyr
to Matter's GN build system. Older pyelftools versions are
not able to parse DWARF5 format, so ram and rom report
would not be generated.

Signed-off-by: Damian Krolik <[email protected]>
Signed-off-by: Lukasz Duda <[email protected]>
Fix the build of Matter samples with sys_heap malloc enabled.
The linker errors would occur because:
1. The realloc() function is not used outside the LTO-
   optimized code.
2. Thus, the LTO linker plugin would mark __wrap_realloc
   symbol as PREVAILING_DEF_IRONLY, which would enable
   the compiler to optimize away the function.
3. As a result, undefined references to realloc() could not
   be resolved by the linker.

Mark malloc/calloc/realloc/free as externally visible.

Signed-off-by: Damian Krolik <[email protected]>
libCHIPShell.a is already part of libCHIP.a but the former
had to be linked additionally with --whole-archive flag to
process the shell and init objects defined with SHELL_XXX
and SYS_INIT Zephyr macros and, in turn, register Matter
shell commands properly.

This symbol duplication between the two libraries causes
issues when building Matter with LTO, so replace the current
approach with explicitly pulling in one symbol from
MainLoopZephyr.cpp file.

Signed-off-by: Damian Krolik <[email protected]>
When CONFIG_CHIP_MALLOC_SYS_HEAP_DEBUG is enabled, log
every malloc/realloc/free to trace the source of memory
leaks in the application.
We need a mechanism to move the DAC private key from the
factory data set to PSA ITS NVM storage during the first
boot of the device.
Then the DAC private key must be removed from the factory
data set and protected by overwriting.

In this commit:
- Added a method to FactoryDataProvider for moving and
removing DAC from the factory data set.
- Aligned the SignWithDeviceAttestationKey method to work with
stored DAC priv key in ITS NVM instead of raw data.
- Added a Matter config to determine base address of ITS storage
for Matter purposes.
- x509 MBedTLS support seems to be not needed anymore
we can disable it and save ~20kB of FLASH.
- Prevent the DAC private key from removal during the
factory reset - for now, disable
the CHIP_FACTORY_RESET_ERASE_NVS config by default.
- Remove the DAC private key from the factory data set
only if the CONFIG_CHIP_CRYPTO_PSA_MIGRATE_DAC_PRIV_KEY
config is set to 'y'.
This fixes the uninitialized variable warning
appearing when compiling the door lock app.

Signed-off-by: Patryk Lipinski <[email protected]>
The recvmsg() is now implemented natively in Zephyr,
so we are not supposed to define a custom one.

Signed-off-by: Marcin Kajor <[email protected]>
Add "zap-gui" west command that runs ZAP GUI to edit
the requested ZAP file.

Add "zap-generate" west command that generates the cluster
code out of the requested ZAP file.

Both commands verify if a correct version of ZAP package
is installed in .zap-install directory in the Matter SDK
module, and update the installed version accordingly.

The commands also try to find the ZAP file in the current
directory if the ZAP file path is not explicitly provided.
The current 800ms is not enough in real setups, where Thread
routers must serve as intermediate hops for many parallel
conversations. Bump this to 2s.

Signed-off-by: Damian Krolik <[email protected]>
GitHub action to automatically create Pull Requests in
the manifest repo when updating revisions

Signed-off-by: Jakub Ciesla <[email protected]>
Wi-Fi unicast tx/rx counters were calculated based on a wrong
values, what could lead to getting negative results.

Recent changes allow to use dedicated unicast structure field
instead of obtaining the values based on calculation.

Signed-off-by: Kamil Kasperczyk <[email protected]>
Pointer to the ThreadDiagnosticsDelegate is used without a null
check. It leads to the usage fault, if ThreadNetworkDiagnostics
cluster is disabled and pointer is set to nullptr.
Added undefined CHIP_NXP_PLATFORM config to pass compliance check.

Signed-off-by: Kamil Kasperczyk <[email protected]>
@kkasperczyk-no kkasperczyk-no merged commit a497f79 into nrfconnect:master May 15, 2024
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.