Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add database and tests #32

Merged
merged 28 commits into from
Sep 14, 2023
Merged

Add database and tests #32

merged 28 commits into from
Sep 14, 2023

Commits on Sep 14, 2023

  1. Implement database cache handling 

    The cache package is responsible for handling a database of users/groups
cached for nss.
This allows preventing calling the broker for each nss requests.
Handle update/get different modules, while keeping the database small
and performant.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    ee3248f View commit details
    Browse the repository at this point in the history

  2. Allow serialize/deserialize database with redacted time. 

    This is only for tests to allow loading/unloading with known time in
advance.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    2cc3690 View commit details
    Browse the repository at this point in the history

  3. Ensure that we don't use the database while cleaning the underlying f… 

    …iles

We were basically allowing cleaning up dirty while db transaction were
in progress. Prevent from this possibility by having all database
transactions are read ones, and changing the files/reopening the database
is write.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    bee270f View commit details
    Browse the repository at this point in the history

  4. Ensure that close finishes once the cleanup goroutine is done 

    We want to avoid a race between Close() and cleaning up the associated
database goroutine.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    64141b6 View commit details
    Browse the repository at this point in the history

  5. Use Remove instead of RemoveAll to clean dirty database. 

    We only remove one file, let’s remove it directly and ignore unexisting
ones.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    47abc7b View commit details
    Browse the repository at this point in the history

  6. Derive dirty flag name from constant 

    This we be reused and exported in the tests.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    4cef0b4 View commit details
    Browse the repository at this point in the history

  7. Error enhancements 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8653427 View commit details
    Browse the repository at this point in the history

  8. First set of TestNew() subtests 

    * Export needed arguments.
* First set of subtests, including cleaning.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    4a1419d View commit details
    Browse the repository at this point in the history

  9. Refresh golden files for TestNew() 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    e501b8b View commit details
    Browse the repository at this point in the history

  10. Make initial database clear up more robust 

    Ensure that we treat invalid database content right away, clearing as
necessary.
Factor out cleaning code in the goroutine, ensuring that we are giving
it some retry period.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    f148853 View commit details
    Browse the repository at this point in the history

  11. Adapt and add tests to new clear up behaviour in New() 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    a4de4c0 View commit details
    Browse the repository at this point in the history

  12. Ensure the cleanup goroutine has started when returning a new cache 

    We had the case where the new cache was returned, then an update is
called BEFORE the cleanup goroutine was started, resulting in any
invalid entries triggering the cleanup process but dropping the request.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d65e34a View commit details
    Browse the repository at this point in the history

  13. Relax now the timeouts in test as we are more consistent for cleanup 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d98f777 View commit details
    Browse the repository at this point in the history

  14. Test helper for no dirty flag 

    This will be shared across tests.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    aba093b View commit details
    Browse the repository at this point in the history

  15. Fix wrong bucket name in update 

    Thanks to the tests!

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d375b9e View commit details
    Browse the repository at this point in the history

  16. Small commentary updates 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    87814ec View commit details
    Browse the repository at this point in the history

  17. Add UpdateFromUserInfo database tests 

    Illustrate cases where we can recover and those were we can’t.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    6e7a73b View commit details
    Browse the repository at this point in the history

  18. Update golden files for TestUpdateFromUserInfo 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    2db01ca View commit details
    Browse the repository at this point in the history

  19. Switch to Go 1.21.0 

    We are using slog

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    06c38b7 View commit details
    Browse the repository at this point in the history

  20. Factor out initCache as a helper for tests 

    This will be reused across tests.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    493a60f View commit details
    Browse the repository at this point in the history

  21. Delete all unknown buckets in database 

    Let’s ensure the database only contains expected and known buckets.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    ec9f67e View commit details
    Browse the repository at this point in the history

  22. Additional TestNew() case handling bucket recreations. 

    Buckets are automatically added to existing database, add a test case
for this.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    fe7766e View commit details
    Browse the repository at this point in the history

  23. Tests for getting users from database 

    Add tests for getting users by id/name/all.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    e60240f View commit details
    Browse the repository at this point in the history

  24. Golden files for get user tests 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8a92e85 View commit details
    Browse the repository at this point in the history

  25. Tests for getting groups from database 

    Tests for getting them by gid/name/all.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d8bf0b2 View commit details
    Browse the repository at this point in the history

  26. Golden file for getting group tests 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    ee5f84d View commit details
    Browse the repository at this point in the history

  27. Run with latest golangci-lint 

    Supports go 1.21.0.

Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    b0e5b8e View commit details
    Browse the repository at this point in the history

  28. Fix linting with new golangci-lint 

    Co-authored-by: Jean-Baptiste Lallement <[email protected]>
    @didrocks @jibel
    didrocks and jibel committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    b8cbcf0 View commit details
    Browse the repository at this point in the history