From 2fd2a07742f1cf3c175e9682ff3f3f425ffc2c75 Mon Sep 17 00:00:00 2001 From: Oskar Dudycz Date: Fri, 6 Aug 2021 12:40:40 +0200 Subject: [PATCH 1/3] Added samples for Protecting Sensitive Data in Event-Sourced Systems with Crypto Shredding --- CQRS_Flow/.NET/README.md | 12 +- Crypto_Shredding/.NET/.gitignore | 291 +++++++++++++++++ Crypto_Shredding/.NET/CryptoShredding.sln | 38 +++ .../.NET/CryptoShredding.sln.DotSettings | 12 + Crypto_Shredding/.NET/README.md | 54 +++ Crypto_Shredding/.NET/docker-compose.yml | 36 ++ .../CryptoShredding.IntegrationTests.csproj | 26 ++ .../EventStoreTests/GetEventsTests.cs | 307 ++++++++++++++++++ .../TestSupport/Given_When_Then.cs | 33 ++ .../Attributes/DataSubjectIdAttribute.cs | 12 + .../Attributes/PersonalDataAttribute.cs | 12 + .../src/CryptoShredding/Contracts/IEvent.cs | 6 + .../CryptoShredding/CryptoShredding.csproj | 13 + .../src/CryptoShredding/EventConverter.cs | 37 +++ .../.NET/src/CryptoShredding/EventStore.cs | 62 ++++ .../Repository/CryptoRepository.cs | 44 +++ .../Repository/EncryptionKey.cs | 16 + .../DeserializationContractResolver.cs | 52 +++ .../SerializationContractResolver.cs | 58 ++++ .../Serialization/EncryptorDecryptor.cs | 63 ++++ .../JsonConverters/DecryptionJsonConverter.cs | 41 +++ .../JsonConverters/EncryptionJsonConverter.cs | 41 +++ .../FieldEncryptionDecryption.cs | 92 ++++++ .../Serialization/JsonSerializer.cs | 92 ++++++ .../JsonSerializerSettingsFactory.cs | 54 +++ .../Serialization/SerializedEvent.cs | 16 + README.md | 9 + 27 files changed, 1522 insertions(+), 7 deletions(-) create mode 100644 Crypto_Shredding/.NET/.gitignore create mode 100644 Crypto_Shredding/.NET/CryptoShredding.sln create mode 100644 Crypto_Shredding/.NET/CryptoShredding.sln.DotSettings create mode 100644 Crypto_Shredding/.NET/README.md create mode 100644 Crypto_Shredding/.NET/docker-compose.yml create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/TestSupport/Given_When_Then.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Attributes/DataSubjectIdAttribute.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Attributes/PersonalDataAttribute.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Contracts/IEvent.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/EventConverter.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/EventStore.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Repository/CryptoRepository.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Repository/EncryptionKey.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/DeserializationContractResolver.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/SerializationContractResolver.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/EncryptorDecryptor.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/DecryptionJsonConverter.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/EncryptionJsonConverter.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializer.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializerSettingsFactory.cs create mode 100644 Crypto_Shredding/.NET/src/CryptoShredding/Serialization/SerializedEvent.cs diff --git a/CQRS_Flow/.NET/README.md b/CQRS_Flow/.NET/README.md index 2fc55ec..201dc2b 100644 --- a/CQRS_Flow/.NET/README.md +++ b/CQRS_Flow/.NET/README.md @@ -6,16 +6,15 @@ This sample is showing a typical flow of the Event Sourcing pattern with [Event ## Prerequisities -1. Install git - https://git-scm.com/downloads. -2. Install .NET Core 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. -3. Install Visual Studio 2019, Rider or VSCode. -4. Install docker - https://docs.docker.com/docker-for-windows/install/. -5. Open `ECommerce.sln` solution. +1. Install .NET Core 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. +2. Install Visual Studio 2019, Rider or VSCode. +3. Install docker - https://docs.docker.com/docker-for-windows/install/. +4. Open `ECommerce.sln` solution. ## Running 1. Run: `docker-compose up`. -2. Wait until all dockers got are downloaded and running. +2. Wait until all dockers are up and running. 3. You should automatically get: - EventStoreDB UI: http://localhost:2113/ - ElasticSearch running at http://localhost:9200 @@ -23,7 +22,6 @@ This sample is showing a typical flow of the Event Sourcing pattern with [Event 4. Open, build and run `ECommerce.sln` solution. - Swagger should be available at: http://localhost:5000/index.html - ## Overview It uses: diff --git a/Crypto_Shredding/.NET/.gitignore b/Crypto_Shredding/.NET/.gitignore new file mode 100644 index 0000000..69b54c5 --- /dev/null +++ b/Crypto_Shredding/.NET/.gitignore @@ -0,0 +1,291 @@ +## Ignore Visual Studio temporary files, build results, and +## files generated by popular Visual Studio add-ons. +## +## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore + +# User-specific files +*.suo +*.user +*.userosscache +*.sln.docstates + +# User-specific files (MonoDevelop/Xamarin Studio) +*.userprefs + +# Build results +[Dd]ebug/ +[Dd]ebugPublic/ +[Rr]elease/ +[Rr]eleases/ +x64/ +x86/ +bld/ +[Bb]in/ +[Oo]bj/ +[Ll]og/ + +# Visual Studio 2015 cache/options directory +.vs/ +# Uncomment if you have tasks that create the project's static files in wwwroot +#wwwroot/ + +# MSTest test Results +[Tt]est[Rr]esult*/ +[Bb]uild[Ll]og.* + +# NUNIT +*.VisualState.xml +TestResult.xml + +# Build Results of an ATL Project +[Dd]ebugPS/ +[Rr]eleasePS/ +dlldata.c + +# .NET Core +project.lock.json +project.fragment.lock.json +artifacts/ +# **/Properties/launchSettings.json + +*_i.c +*_p.c +*_i.h +*.ilk +*.meta +*.obj +*.pch +*.pdb +*.pgc +*.pgd +*.rsp +*.sbr +*.tlb +*.tli +*.tlh +*.tmp +*.tmp_proj +*.log +*.vspscc +*.vssscc +.builds +*.pidb +*.svclog +*.scc + +# Chutzpah Test files +_Chutzpah* + +# Visual C++ cache files +ipch/ +*.aps +*.ncb +*.opendb +*.opensdf +*.sdf +*.cachefile +*.VC.db +*.VC.VC.opendb + +# Visual Studio profiler +*.psess +*.vsp +*.vspx +*.sap + +# TFS 2012 Local Workspace +$tf/ + +# Guidance Automation Toolkit +*.gpState + +# ReSharper is a .NET coding add-in +_ReSharper*/ +*.[Rr]e[Ss]harper +*.DotSettings.user + +# JustCode is a .NET coding add-in +.JustCode + +# TeamCity is a build add-in +_TeamCity* + +# DotCover is a Code Coverage Tool +*.dotCover + +# Visual Studio code coverage results +*.coverage +*.coveragexml + +# NCrunch +_NCrunch_* +.*crunch*.local.xml +nCrunchTemp_* + +# MightyMoose +*.mm.* +AutoTest.Net/ + +# Web workbench (sass) +.sass-cache/ + +# Installshield output folder +[Ee]xpress/ + +# DocProject is a documentation generator add-in +DocProject/buildhelp/ +DocProject/Help/*.HxT +DocProject/Help/*.HxC +DocProject/Help/*.hhc +DocProject/Help/*.hhk +DocProject/Help/*.hhp +DocProject/Help/Html2 +DocProject/Help/html + +# Click-Once directory +publish/ + +# Publish Web Output +*.[Pp]ublish.xml +*.azurePubxml +# TODO: Comment the next line if you want to checkin your web deploy settings +# but database connection strings (with potential passwords) will be unencrypted +*.pubxml +*.publishproj + +# Microsoft Azure Web App publish settings. Comment the next line if you want to +# checkin your Azure Web App publish settings, but sensitive information contained +# in these scripts will be unencrypted +PublishScripts/ + +# NuGet Packages +*.nupkg +# The packages folder can be ignored because of Package Restore +**/packages/* +# except build/, which is used as an MSBuild target. +!**/packages/build/ +# Uncomment if necessary however generally it will be regenerated when needed +#!**/packages/repositories.config +# NuGet v3's project.json files produces more ignorable files +*.nuget.props +*.nuget.targets + +# Microsoft Azure Build Output +csx/ +*.build.csdef + +# Microsoft Azure Emulator +ecf/ +rcf/ + +# Windows Store app package directories and files +AppPackages/ +BundleArtifacts/ +Package.StoreAssociation.xml +_pkginfo.txt + +# Visual Studio cache files +# files ending in .cache can be ignored +*.[Cc]ache +# but keep track of directories ending in .cache +!*.[Cc]ache/ + +# Others +ClientBin/ +~$* +*~ +*.dbmdl +*.dbproj.schemaview +*.jfm +*.pfx +*.publishsettings +orleans.codegen.cs + +# Since there are multiple workflows, uncomment next line to ignore bower_components +# (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) +#bower_components/ + +# RIA/Silverlight projects +Generated_Code/ + +# Backup & report files from converting an old project file +# to a newer Visual Studio version. Backup files are not needed, +# because we have git ;-) +_UpgradeReport_Files/ +Backup*/ +UpgradeLog*.XML +UpgradeLog*.htm + +# SQL Server files +*.mdf +*.ldf +*.ndf + +# Business Intelligence projects +*.rdl.data +*.bim.layout +*.bim_*.settings + +# Microsoft Fakes +FakesAssemblies/ + +# GhostDoc plugin setting file +*.GhostDoc.xml + +# Node.js Tools for Visual Studio +.ntvs_analysis.dat +node_modules/ + +# Typescript v1 declaration files +typings/ + +# Visual Studio 6 build log +*.plg + +# Visual Studio 6 workspace options file +*.opt + +# Visual Studio 6 auto-generated workspace file (contains which files were open etc.) +*.vbw + +# Visual Studio LightSwitch build output +**/*.HTMLClient/GeneratedArtifacts +**/*.DesktopClient/GeneratedArtifacts +**/*.DesktopClient/ModelManifest.xml +**/*.Server/GeneratedArtifacts +**/*.Server/ModelManifest.xml +_Pvt_Extensions + +# Paket dependency manager +.paket/paket.exe +paket-files/ + +# FAKE - F# Make +.fake/ + +# JetBrains Rider +.idea/ +*.sln.iml + +# CodeRush +.cr/ + +# Python Tools for Visual Studio (PTVS) +__pycache__/ +*.pyc + +# Cake - Uncomment if you are using it +# tools/** +# !tools/packages.config + +# Telerik's JustMock configuration file +*.jmconfig + +# BizTalk build output +*.btp.cs +*.btm.cs +*.odx.cs +*.xsd.cs + +NDependOut +.idea/ diff --git a/Crypto_Shredding/.NET/CryptoShredding.sln b/Crypto_Shredding/.NET/CryptoShredding.sln new file mode 100644 index 0000000..7948042 --- /dev/null +++ b/Crypto_Shredding/.NET/CryptoShredding.sln @@ -0,0 +1,38 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{53B51C6D-8443-4EFC-BD9B-F7310F317D50}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CryptoShredding", "src\CryptoShredding\CryptoShredding.csproj", "{9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docker", "docker", "{70E20215-387D-455E-9C6E-28166F7A4EC0}" +ProjectSection(SolutionItems) = preProject + docker-compose.yml = docker-compose.yml +EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{7E02C6B3-2B38-4A1F-99AC-B8FBD6973A96}" +ProjectSection(SolutionItems) = preProject + README.md = README.md +EndProjectSection +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CryptoShredding.IntegrationTests", "src\CryptoShredding.IntegrationTests\CryptoShredding.IntegrationTests.csproj", "{FA1C5197-1F19-4EB6-825C-08BB380283AB}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Release|Any CPU = Release|Any CPU + EndGlobalSection + GlobalSection(NestedProjects) = preSolution + {9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D} = {53B51C6D-8443-4EFC-BD9B-F7310F317D50} + {FA1C5197-1F19-4EB6-825C-08BB380283AB} = {53B51C6D-8443-4EFC-BD9B-F7310F317D50} + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D}.Debug|Any CPU.Build.0 = Debug|Any CPU + {9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D}.Release|Any CPU.ActiveCfg = Release|Any CPU + {9C35F2D6-5E9A-4A8C-8F5C-0EF0B165221D}.Release|Any CPU.Build.0 = Release|Any CPU + {FA1C5197-1F19-4EB6-825C-08BB380283AB}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {FA1C5197-1F19-4EB6-825C-08BB380283AB}.Debug|Any CPU.Build.0 = Debug|Any CPU + {FA1C5197-1F19-4EB6-825C-08BB380283AB}.Release|Any CPU.ActiveCfg = Release|Any CPU + {FA1C5197-1F19-4EB6-825C-08BB380283AB}.Release|Any CPU.Build.0 = Release|Any CPU + EndGlobalSection +EndGlobal diff --git a/Crypto_Shredding/.NET/CryptoShredding.sln.DotSettings b/Crypto_Shredding/.NET/CryptoShredding.sln.DotSettings new file mode 100644 index 0000000..8343f12 --- /dev/null +++ b/Crypto_Shredding/.NET/CryptoShredding.sln.DotSettings @@ -0,0 +1,12 @@ + + <Policy Inspect="True" Prefix="" Suffix="" Style="AaBb_AaBb" /> + True + True + True + True + True + True + True + True + True + True \ No newline at end of file diff --git a/Crypto_Shredding/.NET/README.md b/Crypto_Shredding/.NET/README.md new file mode 100644 index 0000000..520f3de --- /dev/null +++ b/Crypto_Shredding/.NET/README.md @@ -0,0 +1,54 @@ +# Protecting Sensitive Data in Event-Sourced Systems with Crypto Shredding + +This sample is showing an example of using the Crypto Shredding pattern with [EventStoreDB](https://developers.eventstore.com). This can be a solution for handling e.g. [European General Data Protection Regulation](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation). + +Read more in the [Diego Martin](https://github.com/diegosasw) article ["Protecting Sensitive Data in Event-Sourced Systems with Crypto Shredding"](https://www.eventstore.com/blog/protecting-sensitive-data-in-event-sourced-systems-with-crypto-shredding-1); + +## Prerequisities + +1. Install .NET Core 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. +2. Install Visual Studio 2019, Rider or VSCode. +3. Install docker - https://docs.docker.com/docker-for-windows/install/. +4. Open `ECommerce.sln` solution. + +## Running + +1. Run: `docker-compose up`. +2. Wait until all dockers are up and running. +3. You should automatically get: + - EventStoreDB UI: http://localhost:2113/ +4. Open, build and run [tests](./src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs) in [CryptoShredding.sln](CryptoShredding.sln) solution. + +## Overview + +The general flow for using Crypto Shredding patern: + +1. Identify sensitive data in an event. See: + - [PersonalDataAttribute](./src/CryptoShredding/Attributes/PersonalDataAttribute.cs). +2. Associate sensitive data to a subject. See: + - [DataSubjectIdAttribute ](./src/CryptoShredding/Attributes/DataSubjectIdAttribute.cs), + - and the usage in [ContactAdded event](./src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs#L274). +3. Store private encryption keys. See: + - [EncryptionKey](./src/CryptoShredding/Repository/EncryptionKey.cs), + - [CryptoRepository](./src/CryptoShredding/Repository/CryptoRepository.cs). +4. Get rid of the private encryption key when desired. See: + - `DeleteEncryptionKey` method in [CryptoRepository](./src/CryptoShredding/Repository/CryptoRepository.cs). +5. Cryptographic algorithm to use when encrypting and decrypting. See: + - [EncryptorDecryptor](./src/CryptoShredding/Serialization/EncryptorDecryptor.cs). +6. Encrypt text and other data types. See: + - [FieldEncryptionDecryption](./src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs). +7. Upstream serialization with encryption mechanism. See: + - [SerializationContractResolver](./src/CryptoShredding/Serialization/ContractResolvers/SerializationContractResolver.cs), + - [EncryptionJsonConverter](./src/CryptoShredding/Serialization/JsonConverters/EncryptionJsonConverter.cs), + - [JsonSerializerSettingsFactory](./src/CryptoShredding/Serialization/JsonSerializerSettingsFactory.cs), + - [SerializedEvent](./src/CryptoShredding/Serialization/SerializedEvent.cs). +8. Decrypt text and masking mechanism when it cannot be decrypted. See: + - [FieldEncryptionDecryption](./src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs). +9. Downstream deserialization with decryption mechanism. See: + - [DeserializationContractResolver](./src/CryptoShredding/Serialization/ContractResolvers/DeserializationContractResolver.cs), + - [DecryptionJsonConverter](./src/CryptoShredding/Serialization/JsonConverters/DecryptionJsonConverter.cs). +10. Wire up together with an [EventStoreDB](https://developers.eventstore.com) gRPC client. See: + - [EventConverter](./src/CryptoShredding/EventConverter.cs), + - [EventStore](./src/CryptoShredding/EventStore.cs). +11. Test everything with an [EventStoreDB](https://developers.eventstore.com). See: + - [GetEventsTests](./src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs). diff --git a/Crypto_Shredding/.NET/docker-compose.yml b/Crypto_Shredding/.NET/docker-compose.yml new file mode 100644 index 0000000..25833bf --- /dev/null +++ b/Crypto_Shredding/.NET/docker-compose.yml @@ -0,0 +1,36 @@ +version: "3" +services: + ####################################################### + # EventStoreDB - Event Store + ####################################################### + eventstore.db: + image: eventstore/eventstore:21.6.0-buster-slim + environment: + - EVENTSTORE_CLUSTER_SIZE=1 + - EVENTSTORE_RUN_PROJECTIONS=All + - EVENTSTORE_START_STANDARD_PROJECTIONS=true + - EVENTSTORE_EXT_TCP_PORT=1113 + - EVENTSTORE_HTTP_PORT=2113 + - EVENTSTORE_INSECURE=true + - EVENTSTORE_ENABLE_EXTERNAL_TCP=true + - EVENTSTORE_ENABLE_ATOM_PUB_OVER_HTTP=true + ports: + - '1113:1113' + - '2113:2113' + volumes: + - type: volume + source: eventstore-volume-data + target: /var/lib/eventstore + - type: volume + source: eventstore-volume-logs + target: /var/log/eventstore + networks: + - eventstore.db + +networks: + eventstore.db: + driver: bridge + +volumes: + eventstore-volume-data: + eventstore-volume-logs: diff --git a/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj new file mode 100644 index 0000000..03533d7 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj @@ -0,0 +1,26 @@ + + + + net5.0 + false + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + diff --git a/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs new file mode 100644 index 0000000..fcd3fdc --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs @@ -0,0 +1,307 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading.Tasks; +using CryptoShredding.Attributes; +using CryptoShredding.Contracts; +using CryptoShredding.IntegrationTests.TestSupport; +using CryptoShredding.Repository; +using CryptoShredding.Serialization; +using EventStore.Client; +using FluentAssertions; +using Xunit; + +namespace CryptoShredding.IntegrationTests.EventStoreTests +{ + public static class GetEventsTests + { + public class Given_A_ContactBookCreated_And_Two_Events_With_Personal_Data_Stored_When_Getting_Events + : Given_WhenAsync_Then_Test + { + private EventStore _sut; + private EventStoreClient _eventStoreClient; + private string _streamName; + private Guid _joeId; + private Guid _janeId; + private ContactAdded _expectedContactAddedOne; + private ContactAdded _expectedContactAddedTwo; + private IEnumerable _result; + + protected override async Task Given() + { + const string connectionString = "esdb://localhost:2113?tls=false"; + _eventStoreClient = new EventStoreClient(EventStoreClientSettings.Create(connectionString)); + + var supportedEvents = + new List + { + typeof(ContactBookCreated), + typeof(ContactAdded) + }; + + var cryptoRepository = new CryptoRepository(); + var encryptorDecryptor = new EncryptorDecryptor(cryptoRepository); + var jsonSerializerSettingsFactory = new JsonSerializerSettingsFactory(encryptorDecryptor); + var jsonSerializer = new JsonSerializer(jsonSerializerSettingsFactory, supportedEvents); + var eventConverter = new EventConverter(jsonSerializer); + + var aggregateId = Guid.NewGuid(); + _streamName = $"ContactBook-{aggregateId.ToString().Replace("-", string.Empty)}"; + + _sut = new EventStore(_eventStoreClient, eventConverter); + + var contactBookCreated = + new ContactBookCreated + { + AggregateId = aggregateId + }; + + _joeId = Guid.NewGuid(); + var contactAddedOne = + new ContactAdded + { + AggregateId = aggregateId, + Name = "Joe Bloggs", + Birthday = new DateTime(1984, 1, 1, 0, 0, 0, DateTimeKind.Utc), + PersonId = _joeId, + Address = + new Address + { + Street = "Blue Avenue", + Number = 23, + CountryCode = "ES" + } + }; + + _janeId = Guid.NewGuid(); + var contactAddedTwo = + new ContactAdded + { + AggregateId = aggregateId, + Name = "Jane Bloggs", + Birthday = new DateTime(1987, 12, 31, 0, 0, 0, DateTimeKind.Utc), + PersonId = _janeId, + Address = + new Address + { + Street = "Pink Avenue", + Number = 33, + CountryCode = "ES" + } + }; + + var eventsToPersist = + new List + { + contactBookCreated, + contactAddedOne, + contactAddedTwo + }; + + var aggregateVersion = eventsToPersist.Count; + + await _sut.PersistEvents(_streamName, aggregateVersion, eventsToPersist); + + _expectedContactAddedOne = contactAddedOne; + _expectedContactAddedTwo = contactAddedTwo; + } + + protected override async Task When() + { + _result = await _sut.GetEvents(_streamName); + } + + [Fact] + public void Then_It_Should_Retrieve_Three_Events() + { + _result.Should().HaveCount(3); + } + + [Fact] + public void Then_It_Should_Have_Decrypted_The_First_ContactAdded_Event() + { + _result.ElementAt(1).Should().BeEquivalentTo(_expectedContactAddedOne); + } + + [Fact] + public void Then_It_Should_Have_Decrypted_The_Second_ContactAdded_Event() + { + _result.ElementAt(2).Should().BeEquivalentTo(_expectedContactAddedTwo); + } + + protected override void Cleanup() + { + _eventStoreClient.Dispose(); + } + } + + public class Given_A_ContactBookCreated_And_Two_Events_With_Personal_Data_Stored_And_Encryption_Key_For_One_Is_Deleted_When_Getting_Events + : Given_WhenAsync_Then_Test + { + private EventStore _sut; + private EventStoreClient _eventStoreClient; + private string _streamName; + private Guid _joeId; + private Guid _janeId; + private ContactAdded _expectedContactAddedOne; + private ContactAdded _expectedContactAddedTwo; + private IEnumerable _result; + + protected override async Task Given() + { + const string connectionString = "esdb://localhost:2113?tls=false"; + _eventStoreClient = new EventStoreClient(EventStoreClientSettings.Create(connectionString)); + + var supportedEvents = + new List + { + typeof(ContactBookCreated), + typeof(ContactAdded) + }; + + var cryptoRepository = new CryptoRepository(); + var encryptorDecryptor = new EncryptorDecryptor(cryptoRepository); + var jsonSerializerSettingsFactory = new JsonSerializerSettingsFactory(encryptorDecryptor); + var jsonSerializer = new JsonSerializer(jsonSerializerSettingsFactory, supportedEvents); + var eventConverter = new EventConverter(jsonSerializer); + + var aggregateId = Guid.NewGuid(); + _streamName = $"ContactBook-{aggregateId.ToString().Replace("-", string.Empty)}"; + + _sut = new EventStore(_eventStoreClient, eventConverter); + + var contactBookCreated = + new ContactBookCreated + { + AggregateId = aggregateId + }; + + _joeId = Guid.NewGuid(); + var contactAddedOne = + new ContactAdded + { + AggregateId = aggregateId, + Name = "Joe Bloggs", + Birthday = new DateTime(1984, 1, 1, 0, 0, 0, DateTimeKind.Utc), + PersonId = _joeId, + Address = + new Address + { + Street = "Blue Avenue", + Number = 23, + CountryCode = "ES" + } + }; + + _janeId = Guid.NewGuid(); + var contactAddedTwo = + new ContactAdded + { + AggregateId = aggregateId, + Name = "Jane Bloggs", + Birthday = new DateTime(1987, 12, 31, 0, 0, 0, DateTimeKind.Utc), + PersonId = _janeId, + Address = + new Address + { + Street = "Pink Avenue", + Number = 33, + CountryCode = "ES" + } + }; + + var eventsToPersist = + new List + { + contactBookCreated, + contactAddedOne, + contactAddedTwo + }; + + var aggregateVersion = eventsToPersist.Count; + + await _sut.PersistEvents(_streamName, aggregateVersion, eventsToPersist); + cryptoRepository.DeleteEncryptionKey(_janeId.ToString()); + + _expectedContactAddedOne = contactAddedOne; + _expectedContactAddedTwo = + new ContactAdded + { + AggregateId = aggregateId, + Name = "***", + Birthday = default, + PersonId = _janeId, + Address = + new Address + { + Street = "***", + Number = default, + CountryCode = "ES" + } + }; + } + + protected override async Task When() + { + _result = await _sut.GetEvents(_streamName); + } + + [Fact] + public void Then_It_Should_Retrieve_Three_Events() + { + _result.Should().HaveCount(3); + } + + [Fact] + public void Then_It_Should_Have_Decrypted_The_First_ContactAdded_Event() + { + _result.ElementAt(1).Should().BeEquivalentTo(_expectedContactAddedOne); + } + + [Fact] + public void Then_It_Should_Have_Decrypted_The_Second_ContactAdded_Event() + { + _result.ElementAt(2).Should().BeEquivalentTo(_expectedContactAddedTwo); + } + + protected override void Cleanup() + { + _eventStoreClient.Dispose(); + } + } + } + + public class ContactAdded + : IEvent + { + public Guid AggregateId { get; set; } + + [DataSubjectId] + public Guid PersonId { get; set; } + + [PersonalData] + public string Name { get; set; } + + [PersonalData] + public DateTime Birthday { get; set; } + + public Address Address { get; set; } = new Address(); + } + + public class Address + { + [PersonalData] + public string Street { get; set; } + + [PersonalData] + public int Number { get; set; } + + public string CountryCode { get; set; } + } + + public class ContactBookCreated + : IEvent + { + public Guid AggregateId { get; set; } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/TestSupport/Given_When_Then.cs b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/TestSupport/Given_When_Then.cs new file mode 100644 index 0000000..31138c0 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/TestSupport/Given_When_Then.cs @@ -0,0 +1,33 @@ +using System; +using System.Threading.Tasks; + +namespace CryptoShredding.IntegrationTests.TestSupport +{ + public abstract class Given_WhenAsync_Then_Test + : IDisposable + { + protected Given_WhenAsync_Then_Test() + { + Task.Run((Func) (async () => await this.SetupAsync())).Wait(); + } + + private async Task SetupAsync() + { + await Given(); + await When(); + } + + protected abstract Task Given(); + + protected abstract Task When(); + + public void Dispose() + { + Cleanup(); + } + + protected virtual void Cleanup() + { + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/DataSubjectIdAttribute.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/DataSubjectIdAttribute.cs new file mode 100644 index 0000000..d15e9b3 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/DataSubjectIdAttribute.cs @@ -0,0 +1,12 @@ +using System; + +namespace CryptoShredding.Attributes +{ + /** + * Specifies the PII owner (e.g: the person Id) + */ + public class DataSubjectIdAttribute + : Attribute + { + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/PersonalDataAttribute.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/PersonalDataAttribute.cs new file mode 100644 index 0000000..26674d7 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Attributes/PersonalDataAttribute.cs @@ -0,0 +1,12 @@ +using System; + +namespace CryptoShredding.Attributes +{ + /** + * Specifies the property that holds PII + */ + public class PersonalDataAttribute + : Attribute + { + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Contracts/IEvent.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Contracts/IEvent.cs new file mode 100644 index 0000000..025d615 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Contracts/IEvent.cs @@ -0,0 +1,6 @@ +namespace CryptoShredding.Contracts +{ + public interface IEvent + { + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj b/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj new file mode 100644 index 0000000..a4b254e --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj @@ -0,0 +1,13 @@ + + + + .net5.0 + + + + + + + + + diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/EventConverter.cs b/Crypto_Shredding/.NET/src/CryptoShredding/EventConverter.cs new file mode 100644 index 0000000..b044147 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/EventConverter.cs @@ -0,0 +1,37 @@ +using CryptoShredding.Contracts; +using CryptoShredding.Serialization; +using EventStore.Client; + +namespace CryptoShredding +{ + public class EventConverter + { + private readonly JsonSerializer _jsonSerializer; + + public EventConverter(JsonSerializer jsonSerializer) + { + _jsonSerializer = jsonSerializer; + } + + public IEvent ToEvent(ResolvedEvent resolvedEvent) + { + var data = resolvedEvent.Event.Data; + var metadata = resolvedEvent.Event.Metadata; + var eventName = resolvedEvent.Event.EventType; + var persistableEvent = _jsonSerializer.Deserialize(data, metadata, eventName); + return persistableEvent; + } + + public EventData ToEventData(IEvent @event) + { + var eventTypeName = @event.GetType().Name; + var id = Uuid.NewUuid(); + var serializedEvent = _jsonSerializer.Serialize(@event); + var contentType = serializedEvent.IsJson ? "application/json" : "application/octet-stream"; + var data = serializedEvent.Data; + var metadata = serializedEvent.MetaData; + var eventData = new EventData(id, eventTypeName,data, metadata, contentType); + return eventData; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/EventStore.cs b/Crypto_Shredding/.NET/src/CryptoShredding/EventStore.cs new file mode 100644 index 0000000..004f12d --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/EventStore.cs @@ -0,0 +1,62 @@ +using System.Collections.Generic; +using System.Linq; +using System.Threading.Tasks; +using CryptoShredding.Contracts; +using EventStore.Client; + +namespace CryptoShredding +{ + public class EventStore + { + private readonly EventStoreClient _eventStoreClient; + private readonly EventConverter _eventConverter; + + public EventStore( + EventStoreClient eventStoreClient, + EventConverter eventConverter) + { + _eventStoreClient = eventStoreClient; + _eventConverter = eventConverter; + } + + public async Task PersistEvents(string streamName, int aggregateVersion, IEnumerable eventsToPersist) + { + var events = eventsToPersist.ToList(); + var count = events.Count; + if (count == 0) + { + return; + } + + var expectedRevision = GetExpectedRevision(aggregateVersion, count); + var eventsData = + events.Select(x => _eventConverter.ToEventData(x)); + if (expectedRevision == null) + await _eventStoreClient.AppendToStreamAsync(streamName, StreamState.NoStream, eventsData); + else + await _eventStoreClient.AppendToStreamAsync(streamName, expectedRevision.Value, eventsData); + } + + public async Task> GetEvents(string streamName) + { + const int start = 0; + const int count = 4096; + const bool resolveLinkTos = false; + var sliceEvents = + _eventStoreClient.ReadStreamAsync(Direction.Forwards, streamName, start, count, resolveLinkTos: resolveLinkTos); + var resolvedEvents = await sliceEvents.ToListAsync(); + var events = + resolvedEvents.Select(x => _eventConverter.ToEvent(x)); + return events; + } + + private StreamRevision? GetExpectedRevision(int aggregateVersion, int numberOfEvents) + { + var originalVersion = aggregateVersion - numberOfEvents; + var expectedVersion = originalVersion != 0 + ? StreamRevision.FromInt64(originalVersion - 1) + : (StreamRevision?)null; + return expectedVersion; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Repository/CryptoRepository.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Repository/CryptoRepository.cs new file mode 100644 index 0000000..26f119f --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Repository/CryptoRepository.cs @@ -0,0 +1,44 @@ +using System; +using System.Collections.Generic; + +namespace CryptoShredding.Repository +{ + public class CryptoRepository + { + private readonly IDictionary _cryptoStore; + + public CryptoRepository() + { + _cryptoStore = new Dictionary(); + } + + public EncryptionKey GetExistingOrNew(string id, Func keyGenerator) + { + var isExisting = _cryptoStore.TryGetValue(id, out var keyStored); + if (isExisting) + { + return keyStored; + } + + var newEncryptionKey = keyGenerator.Invoke(); + _cryptoStore.Add(id, newEncryptionKey); + return newEncryptionKey; + } + + public EncryptionKey GetExistingOrDefault(string id) + { + var isExisting = _cryptoStore.TryGetValue(id, out var keyStored); + if (isExisting) + { + return keyStored; + } + + return default; + } + + public void DeleteEncryptionKey(string id) + { + _cryptoStore.Remove(id); + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Repository/EncryptionKey.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Repository/EncryptionKey.cs new file mode 100644 index 0000000..eaf22dc --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Repository/EncryptionKey.cs @@ -0,0 +1,16 @@ +namespace CryptoShredding.Repository +{ + public class EncryptionKey + { + public byte[] Key { get; } + public byte[] Nonce { get; } + + public EncryptionKey( + byte[] key, + byte[] nonce) + { + Key = key; + Nonce = nonce; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/DeserializationContractResolver.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/DeserializationContractResolver.cs new file mode 100644 index 0000000..ee01151 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/DeserializationContractResolver.cs @@ -0,0 +1,52 @@ +using System; +using System.Collections.Generic; +using System.Security.Cryptography; +using CryptoShredding.Serialization.JsonConverters; +using Newtonsoft.Json; +using Newtonsoft.Json.Serialization; + +namespace CryptoShredding.Serialization.ContractResolvers +{ + public class DeserializationContractResolver + : DefaultContractResolver + { + private readonly ICryptoTransform _decryptor; + private readonly FieldEncryptionDecryption _fieldEncryptionDecryption; + + public DeserializationContractResolver( + ICryptoTransform decryptor, + FieldEncryptionDecryption fieldEncryptionDecryption) + { + _decryptor = decryptor; + _fieldEncryptionDecryption = fieldEncryptionDecryption; + NamingStrategy = new CamelCaseNamingStrategy(); + } + + protected override IList CreateProperties(Type type, MemberSerialization memberSerialization) + { + var properties = base.CreateProperties(type, memberSerialization); + foreach (var jsonProperty in properties) + { + var isSimpleValue = IsSimpleValue(type, jsonProperty); + if (isSimpleValue) + { + var jsonConverter = new DecryptionJsonConverter(_decryptor, _fieldEncryptionDecryption); + jsonProperty.Converter = jsonConverter; + } + } + return properties; + } + + private bool IsSimpleValue(Type type, JsonProperty jsonProperty) + { + var propertyInfo = type.GetProperty(jsonProperty.UnderlyingName); + if (propertyInfo is null) + { + return false; + } + var propertyType = propertyInfo.PropertyType; + var isSimpleValue = propertyType.IsValueType || propertyType == typeof(string); + return isSimpleValue; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/SerializationContractResolver.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/SerializationContractResolver.cs new file mode 100644 index 0000000..9e0ceb3 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/ContractResolvers/SerializationContractResolver.cs @@ -0,0 +1,58 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Security.Cryptography; +using CryptoShredding.Attributes; +using CryptoShredding.Serialization.JsonConverters; +using Newtonsoft.Json; +using Newtonsoft.Json.Serialization; + +namespace CryptoShredding.Serialization.ContractResolvers +{ + public class SerializationContractResolver + : DefaultContractResolver + { + private readonly ICryptoTransform _encryptor; + private readonly FieldEncryptionDecryption _fieldEncryptionDecryption; + + public SerializationContractResolver( + ICryptoTransform encryptor, + FieldEncryptionDecryption fieldEncryptionDecryption) + { + _encryptor = encryptor; + _fieldEncryptionDecryption = fieldEncryptionDecryption; + NamingStrategy = new CamelCaseNamingStrategy(); + } + + protected override IList CreateProperties(Type type, MemberSerialization memberSerialization) + { + var properties = base.CreateProperties(type, memberSerialization); + foreach (var jsonProperty in properties) + { + var isPersonalIdentifiableInformation = IsPersonalIdentifiableInformation(type, jsonProperty); + if (isPersonalIdentifiableInformation) + { + var serializationJsonConverter = new EncryptionJsonConverter(_encryptor, _fieldEncryptionDecryption); + jsonProperty.Converter = serializationJsonConverter; + } + } + return properties; + } + + private bool IsPersonalIdentifiableInformation(Type type, JsonProperty jsonProperty) + { + var propertyInfo = type.GetProperty(jsonProperty.UnderlyingName); + if (propertyInfo is null) + { + return false; + } + var hasPersonalDataAttribute = + propertyInfo.CustomAttributes + .Any(x => x.AttributeType == typeof(PersonalDataAttribute)); + var propertyType = propertyInfo.PropertyType; + var isSimpleValue = propertyType.IsValueType || propertyType == typeof(string); + var isSupportedPersonalIdentifiableInformation = isSimpleValue && hasPersonalDataAttribute; + return isSupportedPersonalIdentifiableInformation; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/EncryptorDecryptor.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/EncryptorDecryptor.cs new file mode 100644 index 0000000..8ac8a41 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/EncryptorDecryptor.cs @@ -0,0 +1,63 @@ +using System.Security.Cryptography; +using CryptoShredding.Repository; + +namespace CryptoShredding.Serialization +{ + public class EncryptorDecryptor + { + private readonly CryptoRepository _cryptoRepository; + + public EncryptorDecryptor(CryptoRepository cryptoRepository) + { + _cryptoRepository = cryptoRepository; + } + + public ICryptoTransform GetEncryptor(string dataSubjectId) + { + var encryptionKey = _cryptoRepository.GetExistingOrNew(dataSubjectId, CreateNewEncryptionKey); + var aesManaged = GetAesManaged(encryptionKey); + var encryptor = aesManaged.CreateEncryptor(); + return encryptor; + } + + public ICryptoTransform GetDecryptor(string dataSubjectId) + { + var encryptionKey = _cryptoRepository.GetExistingOrDefault(dataSubjectId); + if (encryptionKey is null) + { + // encryption key was deleted + return default; + } + + var aesManaged = GetAesManaged(encryptionKey); + var decryptor = aesManaged.CreateDecryptor(); + return decryptor; + } + + private EncryptionKey CreateNewEncryptionKey() + { + var aesManaged = + new AesManaged + { + Padding = PaddingMode.PKCS7 + }; + var key = aesManaged.Key; + var nonce = aesManaged.IV; + var encryptionKey = new EncryptionKey(key, nonce); + return encryptionKey; + } + + private AesManaged GetAesManaged(EncryptionKey encryptionKey) + { + var aesManaged = + new AesManaged + { + Padding = PaddingMode.PKCS7, + Key = encryptionKey.Key, + IV = encryptionKey.Nonce + }; + + return aesManaged; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/DecryptionJsonConverter.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/DecryptionJsonConverter.cs new file mode 100644 index 0000000..ec50b45 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/DecryptionJsonConverter.cs @@ -0,0 +1,41 @@ +using System; +using System.Security.Cryptography; +using Newtonsoft.Json; + +namespace CryptoShredding.Serialization.JsonConverters +{ + public class DecryptionJsonConverter + : JsonConverter + { + private readonly ICryptoTransform _decryptor; + private readonly FieldEncryptionDecryption _fieldEncryptionDecryption; + + public DecryptionJsonConverter( + ICryptoTransform decryptor, + FieldEncryptionDecryption fieldEncryptionService) + { + _decryptor = decryptor; + _fieldEncryptionDecryption = fieldEncryptionService; + } + + public override void WriteJson(JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer) + { + throw new NotImplementedException(); + } + + public override object ReadJson(JsonReader reader, Type objectType, object existingValue, Newtonsoft.Json.JsonSerializer serializer) + { + var value = reader.Value; + var result = _fieldEncryptionDecryption.GetDecryptedOrDefault(value, _decryptor, objectType); + return result; + } + + public override bool CanConvert(Type objectType) + { + return true; + } + + public override bool CanRead => true; + public override bool CanWrite => false; + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/EncryptionJsonConverter.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/EncryptionJsonConverter.cs new file mode 100644 index 0000000..b7f0299 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/EncryptionJsonConverter.cs @@ -0,0 +1,41 @@ +using System; +using System.Security.Cryptography; +using Newtonsoft.Json; + +namespace CryptoShredding.Serialization.JsonConverters +{ + public class EncryptionJsonConverter + : JsonConverter + { + private readonly ICryptoTransform _encryptor; + private readonly FieldEncryptionDecryption _fieldEncryptionDecryption; + + public EncryptionJsonConverter( + ICryptoTransform encryptor, + FieldEncryptionDecryption fieldEncryptionDecryption) + { + _encryptor = encryptor; + _fieldEncryptionDecryption = fieldEncryptionDecryption; + } + + public override void WriteJson(JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer) + { + var result = _fieldEncryptionDecryption.GetEncryptedOrDefault(value, _encryptor); + writer.WriteValue(result); + } + + public override object ReadJson(JsonReader reader, Type objectType, object existingValue, Newtonsoft.Json.JsonSerializer serializer) + { + throw new NotImplementedException(); + } + + public override bool CanConvert(Type objectType) + { + return true; + } + + public override bool CanRead => false; + + public override bool CanWrite => true; + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs new file mode 100644 index 0000000..83faf61 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonConverters/FieldEncryptionDecryption.cs @@ -0,0 +1,92 @@ +using System; +using System.ComponentModel; +using System.IO; +using System.Security.Cryptography; + +namespace CryptoShredding.Serialization.JsonConverters +{ + public class FieldEncryptionDecryption + { + private const string EncryptionPrefix = "crypto."; + + public object GetEncryptedOrDefault(object value, ICryptoTransform encryptor) + { + if (encryptor is null) + { + throw new ArgumentNullException(nameof(encryptor)); + } + var isEncryptionNeeded = value != null; + if (isEncryptionNeeded) + { + using var memoryStream = new MemoryStream(); + using var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write); + using var writer = new StreamWriter(cryptoStream); + var valueAsText = value.ToString(); + writer.Write(valueAsText); + writer.Flush(); + cryptoStream.FlushFinalBlock(); + + var encryptedData = memoryStream.ToArray(); + var encryptedText = Convert.ToBase64String(encryptedData); + var result = $"{EncryptionPrefix}{encryptedText}"; + return result; + } + + return default; + } + + public object GetDecryptedOrDefault(object value, ICryptoTransform decryptor, Type destinationType) + { + if (value is null) + { + throw new ArgumentNullException(nameof(value)); + } + var isText = value is string; + if (isText) + { + var valueAsText = (string)value; + var isEncrypted = valueAsText.StartsWith(EncryptionPrefix); + if (isEncrypted) + { + var isDecryptorAvailable = decryptor != null; + if (isDecryptorAvailable) + { + var startIndex = EncryptionPrefix.Length; + var valueWithoutPrefix = valueAsText.Substring(startIndex); + var encryptedValue = Convert.FromBase64String(valueWithoutPrefix); + using var memoryStream = new MemoryStream(encryptedValue); + using var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read); + using var reader = new StreamReader(cryptoStream); + var decryptedText = reader.ReadToEnd(); + var result = Parse(destinationType, decryptedText); + return result; + } + var maskedValue = GetMaskedValue(destinationType); + return maskedValue; + } + var valueParsed = Parse(destinationType, valueAsText); + return valueParsed; + } + + return value; + } + + private object Parse(Type outputType, string valueAsString) + { + var converter = TypeDescriptor.GetConverter(outputType); + var result = converter.ConvertFromString(valueAsString); + return result; + } + + private object GetMaskedValue(Type destinationType) + { + if (destinationType == typeof(string)) + { + const string templateText = "***"; + return templateText; + } + var defaultValue = Activator.CreateInstance(destinationType); + return defaultValue; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializer.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializer.cs new file mode 100644 index 0000000..2529c70 --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializer.cs @@ -0,0 +1,92 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using CryptoShredding.Attributes; +using CryptoShredding.Contracts; +using Newtonsoft.Json; + +namespace CryptoShredding.Serialization +{ + public class JsonSerializer + { + private const string MetadataSubjectIdKey = "dataSubjectId"; + + private readonly JsonSerializerSettingsFactory _jsonSerializerSettingsFactory; + private readonly IEnumerable _supportedEvents; + + public JsonSerializer( + JsonSerializerSettingsFactory jsonSerializerSettingsFactory, + IEnumerable supportedEvents) + { + _jsonSerializerSettingsFactory = jsonSerializerSettingsFactory; + _supportedEvents = supportedEvents; + } + + public SerializedEvent Serialize(IEvent @event) + { + var dataSubjectId = GetDataSubjectId(@event); + var metadataValues = + new Dictionary + { + {MetadataSubjectIdKey, dataSubjectId} + }; + + var hasPersonalData = dataSubjectId != null; + var dataJsonSerializerSettings = + hasPersonalData + ? _jsonSerializerSettingsFactory.CreateForEncryption(dataSubjectId) + : _jsonSerializerSettingsFactory.CreateDefault(); + + var dataJson = JsonConvert.SerializeObject(@event, dataJsonSerializerSettings); + var dataBytes = Encoding.UTF8.GetBytes(dataJson); + + var defaultJsonSettings = _jsonSerializerSettingsFactory.CreateDefault(); + var metadataJson = JsonConvert.SerializeObject(metadataValues, defaultJsonSettings); + var metadataBytes = Encoding.UTF8.GetBytes(metadataJson); + var serializedEvent = new SerializedEvent(dataBytes, metadataBytes, true); + return serializedEvent; + } + + public IEvent Deserialize(ReadOnlyMemory data, ReadOnlyMemory metadata, string eventName) + { + var metadataJson = Encoding.UTF8.GetString(metadata.Span); + var defaultJsonSettings = _jsonSerializerSettingsFactory.CreateDefault(); + var values = + JsonConvert.DeserializeObject>(metadataJson, defaultJsonSettings); + + var hasKey = values.TryGetValue(MetadataSubjectIdKey, out var dataSubjectId); + var hasPersonalData = hasKey && !string.IsNullOrEmpty(dataSubjectId); + + var dataJsonDeserializerSettings = + hasPersonalData + ? _jsonSerializerSettingsFactory.CreateForDecryption(dataSubjectId) + : _jsonSerializerSettingsFactory.CreateDefault(); + + var eventType = _supportedEvents.Single(x => x.Name == eventName); + var dataJson = Encoding.UTF8.GetString(data.Span); + var persistableEvent = + JsonConvert.DeserializeObject(dataJson, eventType, dataJsonDeserializerSettings); + return (IEvent)persistableEvent; + } + + private string GetDataSubjectId(IEvent @event) + { + var eventType = @event.GetType(); + var properties = eventType.GetProperties(); + var dataSubjectIdPropertyInfo = + properties + .FirstOrDefault(x => x.GetCustomAttributes(typeof(DataSubjectIdAttribute), false) + .Any(y => y is DataSubjectIdAttribute)); + + if (dataSubjectIdPropertyInfo is null) + { + return null; + } + + var value = dataSubjectIdPropertyInfo.GetValue(@event); + var dataSubjectId = value.ToString(); + return dataSubjectId; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializerSettingsFactory.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializerSettingsFactory.cs new file mode 100644 index 0000000..ebf22dc --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/JsonSerializerSettingsFactory.cs @@ -0,0 +1,54 @@ +using CryptoShredding.Serialization.ContractResolvers; +using CryptoShredding.Serialization.JsonConverters; +using Newtonsoft.Json; +using Newtonsoft.Json.Serialization; + +namespace CryptoShredding.Serialization +{ + public class JsonSerializerSettingsFactory + { + private readonly EncryptorDecryptor _encryptorDecryptor; + + public JsonSerializerSettingsFactory(EncryptorDecryptor encryptorDecryptor) + { + _encryptorDecryptor = encryptorDecryptor; + } + + public JsonSerializerSettings CreateDefault() + { + var defaultContractResolver = new CamelCasePropertyNamesContractResolver(); + var defaultSettings = GetSettings(defaultContractResolver); + return defaultSettings; + } + + public JsonSerializerSettings CreateForEncryption(string dataSubjectId) + { + var encryptor = _encryptorDecryptor.GetEncryptor(dataSubjectId); + var fieldEncryptionDecryption = new FieldEncryptionDecryption(); + var serializationContractResolver = + new SerializationContractResolver(encryptor, fieldEncryptionDecryption); + var jsonSerializerSettings = GetSettings(serializationContractResolver); + return jsonSerializerSettings; + } + + public JsonSerializerSettings CreateForDecryption(string dataSubjectId) + { + var decryptor = _encryptorDecryptor.GetDecryptor(dataSubjectId); + var fieldEncryptionDecryption = new FieldEncryptionDecryption(); + var deserializationContractResolver = + new DeserializationContractResolver(decryptor, fieldEncryptionDecryption); + var jsonDeserializerSettings = GetSettings(deserializationContractResolver); + return jsonDeserializerSettings; + } + + private JsonSerializerSettings GetSettings(IContractResolver contractResolver) + { + var settings = + new JsonSerializerSettings + { + ContractResolver = contractResolver + }; + return settings; + } + } +} \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/SerializedEvent.cs b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/SerializedEvent.cs new file mode 100644 index 0000000..6760b2c --- /dev/null +++ b/Crypto_Shredding/.NET/src/CryptoShredding/Serialization/SerializedEvent.cs @@ -0,0 +1,16 @@ +namespace CryptoShredding.Serialization +{ + public class SerializedEvent + { + public byte[] Data { get; } + public byte[] MetaData { get; } + public bool IsJson { get; } + + public SerializedEvent(byte[] data, byte[] metaData, bool isJson) + { + Data = data; + MetaData = metaData; + IsJson = isJson; + } + } +} \ No newline at end of file diff --git a/README.md b/README.md index a67dea0..611a880 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,15 @@ Samples are organised by the specific topic. By going to the folder, you can fin - Read models are stored as ElasticSearch documents. - Shows how to unit and integration test solution. + +### **[Crypto Shredding](./Crypto_Shredding/)** +- [.NET](./Crypto_Shredding/.NET/) + +**Description**: +- shows how to Protecting Sensitive Data (e.g. for [European General Data Protection Regulation](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation)) in Event-Sourced Systems. +- shows how to use the .NET `System.Security.Cryptography` library with [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) algorithm to encrypt and decrypt events' data. +- uses EventStoreDB. + ## Running samples locally Check the `README.md` file in the specific sample folder for the detailed run instructions. From 7d05613fbb8a1c1806310b721480649c78c67250 Mon Sep 17 00:00:00 2001 From: Oskar Dudycz Date: Fri, 6 Aug 2021 12:51:53 +0200 Subject: [PATCH 2/3] Configured pipeline for Crypto Shredding .NET samples --- .github/workflows/build.cqrs_flow.dotnet.yml | 71 +++++++++---------- .../build.crypto_shredding.dotnet.yml | 47 ++++++++++++ .github/workflows/build.dotnet.testreport.yml | 21 ++++++ .../CryptoShredding.IntegrationTests.csproj | 16 ++--- .../CryptoShredding/CryptoShredding.csproj | 2 +- 5 files changed, 112 insertions(+), 45 deletions(-) create mode 100644 .github/workflows/build.crypto_shredding.dotnet.yml create mode 100644 .github/workflows/build.dotnet.testreport.yml diff --git a/.github/workflows/build.cqrs_flow.dotnet.yml b/.github/workflows/build.cqrs_flow.dotnet.yml index e4684dd..affa230 100644 --- a/.github/workflows/build.cqrs_flow.dotnet.yml +++ b/.github/workflows/build.cqrs_flow.dotnet.yml @@ -1,4 +1,4 @@ -name: Build +name: Build CQRS Flow .NET on: push: @@ -11,38 +11,37 @@ defaults: working-directory: ./CQRS_Flow/.NET/ jobs: - build: - runs-on: ubuntu-latest - - steps: - - name: Check Out Repo - uses: actions/checkout@v1 - - - name: Start containers - run: docker-compose -f "docker-compose.yml" up -d - - - name: Setup .NET Core - uses: actions/setup-dotnet@v1 - with: - dotnet-version: "5.0.x" - - - name: Restore NuGet packages - run: dotnet restore - - - name: Build - run: dotnet build --configuration Release --no-restore - - - name: Run tests - run: dotnet test --configuration Release --no-build --logger "trx;LogFileName=test-results.trx" - - - name: Publish test results - uses: dorny/test-reporter@v1 - if: success() || failure() - with: - name: Tests Results - reporter: dotnet-trx - path: '**/test-results.trx' - - - name: Stop containers - if: always() - run: docker-compose -f "docker-compose.yml" down \ No newline at end of file + build: + runs-on: ubuntu-latest + + steps: + - name: Check Out Repo + uses: actions/checkout@v1 + + - name: Start containers + run: docker-compose -f "docker-compose.yml" up -d + + - name: Setup .NET Core + uses: actions/setup-dotnet@v1 + with: + dotnet-version: "5.0.x" + + - name: Restore NuGet packages + run: dotnet restore + + - name: Build + run: dotnet build --configuration Release --no-restore + + - name: Run tests + run: dotnet test --configuration Release --no-build --logger "trx;LogFileName=test-results.trx" + + - name: Upload test results + uses: actions/upload-artifact@v2 + if: success() || failure() + with: + name: test-results + path: "**/test-results.trx" + + - name: Stop containers + if: always() + run: docker-compose -f "docker-compose.yml" down diff --git a/.github/workflows/build.crypto_shredding.dotnet.yml b/.github/workflows/build.crypto_shredding.dotnet.yml new file mode 100644 index 0000000..0ae4ea5 --- /dev/null +++ b/.github/workflows/build.crypto_shredding.dotnet.yml @@ -0,0 +1,47 @@ +name: Build Crypto Shredding .NET + +on: + push: + branches: + - main + pull_request: + +defaults: + run: + working-directory: ./Crypto_Shredding/.NET + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - name: Check Out Repo + uses: actions/checkout@v1 + + - name: Start containers + run: docker-compose -f "docker-compose.yml" up -d + + - name: Setup .NET Core + uses: actions/setup-dotnet@v1 + with: + dotnet-version: "5.0.x" + + - name: Restore NuGet packages + run: dotnet restore + + - name: Build + run: dotnet build --configuration Release --no-restore + + - name: Run tests + run: dotnet test --configuration Release --no-build --logger "trx;LogFileName=test-results.trx" + + - name: Upload test results + uses: actions/upload-artifact@v2 + if: success() || failure() + with: + name: test-results + path: "**/test-results.trx" + + - name: Stop containers + if: always() + run: docker-compose -f "docker-compose.yml" down diff --git a/.github/workflows/build.dotnet.testreport.yml b/.github/workflows/build.dotnet.testreport.yml new file mode 100644 index 0000000..f296f42 --- /dev/null +++ b/.github/workflows/build.dotnet.testreport.yml @@ -0,0 +1,21 @@ +name: Test Report + +on: + workflow_run: + workflows: ["Build CQRS Flow .NET", "Build Crypto Shredding .NET"] + types: + - completed + +jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Publish test results + uses: dorny/test-reporter@v1 + if: success() || failure() + with: + name: Tests Results + artifact: test-results + reporter: dotnet-trx + path: "**/test-results.trx" + fail-on-error: "false" diff --git a/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj index 03533d7..2878f9b 100644 --- a/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj +++ b/Crypto_Shredding/.NET/src/CryptoShredding.IntegrationTests/CryptoShredding.IntegrationTests.csproj @@ -2,25 +2,25 @@ net5.0 - false + true + trx%3bLogFileName=$(MSBuildProjectName).trx + $(MSBuildThisFileDirectory)/bin/TestResults/$(TargetFramework) + - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive + all + runtime; build; native; contentfiles; analyzers + - + \ No newline at end of file diff --git a/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj b/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj index a4b254e..9a8174b 100644 --- a/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj +++ b/Crypto_Shredding/.NET/src/CryptoShredding/CryptoShredding.csproj @@ -1,7 +1,7 @@ - .net5.0 + net5.0 From 8273d1bf9ff808bf23421ea30594e57969a2c2de Mon Sep 17 00:00:00 2001 From: Oskar Dudycz Date: Fri, 6 Aug 2021 14:37:44 +0200 Subject: [PATCH 3/3] Updated the READMEs based on comments --- CQRS_Flow/.NET/README.md | 19 +++++++++---------- Crypto_Shredding/.NET/README.md | 11 +++++------ 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/CQRS_Flow/.NET/README.md b/CQRS_Flow/.NET/README.md index 201dc2b..a916619 100644 --- a/CQRS_Flow/.NET/README.md +++ b/CQRS_Flow/.NET/README.md @@ -6,21 +6,20 @@ This sample is showing a typical flow of the Event Sourcing pattern with [Event ## Prerequisities -1. Install .NET Core 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. -2. Install Visual Studio 2019, Rider or VSCode. -3. Install docker - https://docs.docker.com/docker-for-windows/install/. -4. Open `ECommerce.sln` solution. +- .NET 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. +- Visual Studio 2019, Jetbrains Rider or VSCode. +- Docker - https://docs.docker.com/docker-for-windows/install/. ## Running 1. Run: `docker-compose up`. -2. Wait until all dockers are up and running. -3. You should automatically get: - - EventStoreDB UI: http://localhost:2113/ - - ElasticSearch running at http://localhost:9200 - - Kibana - UI for ElasticSearch . Available at: http://localhost:5601 +2. Wait until all Docker containers are up and running. +3. Check that you can access each started component the following URLs: + - EventStoreDB: http://localhost:2113/ (use the default user `admin` and password `changeit`) + - ElasticSearch: http://localhost:9200 + - Kibana: http://localhost:5601 4. Open, build and run `ECommerce.sln` solution. - - Swagger should be available at: http://localhost:5000/index.html + - The Swagger UI should be available at: http://localhost:5000/index.html ## Overview diff --git a/Crypto_Shredding/.NET/README.md b/Crypto_Shredding/.NET/README.md index 520f3de..7f5980f 100644 --- a/Crypto_Shredding/.NET/README.md +++ b/Crypto_Shredding/.NET/README.md @@ -6,16 +6,15 @@ Read more in the [Diego Martin](https://github.com/diegosasw) article ["Protecti ## Prerequisities -1. Install .NET Core 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. -2. Install Visual Studio 2019, Rider or VSCode. -3. Install docker - https://docs.docker.com/docker-for-windows/install/. -4. Open `ECommerce.sln` solution. +- .NET 5.0 - https://dotnet.microsoft.com/download/dotnet/5.0. +- Visual Studio 2019, Jetbrains Rider or VSCode. +- Docker - https://docs.docker.com/docker-for-windows/install/. ## Running 1. Run: `docker-compose up`. -2. Wait until all dockers are up and running. -3. You should automatically get: +2. Wait until all Docker containers are up and running. +3. Check that you can access each started component the following URL: - EventStoreDB UI: http://localhost:2113/ 4. Open, build and run [tests](./src/CryptoShredding.IntegrationTests/EventStoreTests/GetEventsTests.cs) in [CryptoShredding.sln](CryptoShredding.sln) solution.