Code

Encryption basics#1

Code

Draft

rok wants to merge 73 commits intomainrok/arrow-rs:mainfrom encryption-basicsrok/arrow-rs:encryption-basicsCopy head branch name to clipboard

Commits

Commits on Jan 21, 2025

• first commit

  

  ggershinsky

  authored and

  rok

  committed
  9c2ee0e

  Copy full SHA for 9c2ee0e

• Use ParquetMetaDataReader

  rok

  committed
  cd05623

  Copy full SHA for cd05623

• Fix CI

  rok

  committed
  a63b375

  Copy full SHA for a63b375

• test

  rok

  committed
  89aab9f

  Copy full SHA for 89aab9f

• save progress

  rok

  committed
  5412f89

  Copy full SHA for 5412f89

• work

  rok

  committed
  36a2e3b

  Copy full SHA for 36a2e3b

• Review feedback

  rok

  committed
  39f16df

  Copy full SHA for 39f16df

• page decompression issue

  rok

  committed
  513425f

  Copy full SHA for 513425f

• add update_aad

  rok

  committed
  d2e67b0

  Copy full SHA for d2e67b0

• Change encrypt and decrypt to return Results

  

  adamreeve

  authored and

  rok

  committed
  b5036a4

  Copy full SHA for b5036a4

• Use correct page ordinal and module type in AADs

  

  adamreeve

  authored and

  rok

  committed
  8e9b539

  Copy full SHA for 8e9b539

• Tidy up ordinal types

  

  adamreeve

  authored and

  rok

  committed
  9d09a61

  Copy full SHA for 9d09a61

• Lint

  rok

  committed
  8e3fbda

  Copy full SHA for 8e3fbda

• Fix regular deserialization path

  rok

  committed
  c09f9bd

  Copy full SHA for c09f9bd

• cleaning

  rok

  committed
  48cc9ae

  Copy full SHA for 48cc9ae

• Update data checks in test

  

  adamreeve

  authored and

  rok

  committed
  e96e519

  Copy full SHA for e96e519

• start non-uniform decryption

  rok

  committed
  ba4f5b3

  Copy full SHA for ba4f5b3

• Add missing doc comments

  

  adamreeve

  authored and

  rok

  committed
  e3d6b7b

  Copy full SHA for e3d6b7b

• Make encryption an optional feature

  

  adamreeve

  authored and

  rok

  committed
  82cfca7

  Copy full SHA for 82cfca7

• Handle when a file is encrypted but encryption is disabled or no decryption properties are provided

  

  adamreeve

  authored and

  rok

  committed
  5c5d8d9

  Copy full SHA for 5c5d8d9

• Allow for plaintext footer

  rok

  committed
  cec60c8

  Copy full SHA for cec60c8

• work

  rok

  committed
  baeef93

  Copy full SHA for baeef93

• Fix method name

  

  adamreeve

  authored and

  rok

  committed
  6638e12

  Copy full SHA for 6638e12

• work

  rok

  committed
  bec2f5d

  Copy full SHA for bec2f5d

• Minor

  rok

  committed
  afe0f36

  Copy full SHA for afe0f36

• work

  rok

  committed
  a3d3911

  Copy full SHA for a3d3911

• work

  rok

  committed
  1ef5dff

  Copy full SHA for 1ef5dff

• work

  rok

  committed
  bbaef12

  Copy full SHA for bbaef12

• Fix reading to end of file

  

  adamreeve

  authored and

  rok

  committed
  cb91a21

  Copy full SHA for cb91a21

• Refactor tests

  

  adamreeve

  authored and

  rok

  committed
  5b72569

  Copy full SHA for 5b72569

• Fix non-uniform encryption configuration

  

  adamreeve

  authored and

  rok

  committed
  322b4b7

  Copy full SHA for 322b4b7

• Don't use footer key for non-encrypted columns

  

  adamreeve

  authored and

  rok

  committed
  9947f6c

  Copy full SHA for 9947f6c

• Rebase and cleanup

  rok

  committed
  6c9f5c5

  Copy full SHA for 6c9f5c5

• Cleanup

  rok

  committed
  782cd85

  Copy full SHA for 782cd85

• Cleanup

  rok

  committed
  5ead306

  Copy full SHA for 5ead306

• Cleanup

  rok

  committed
  d70f44e

  Copy full SHA for d70f44e

• Cleanup

  rok

  committed
  8a8d99f

  Copy full SHA for 8a8d99f

• Cleanup

  rok

  committed
  2e52d13

  Copy full SHA for 2e52d13

• Cleanup

  rok

  committed
  c2497e6

  Copy full SHA for c2497e6

• lint

  rok

  committed
  951f2fa

  Copy full SHA for 951f2fa

Commits on Jan 22, 2025

• Remove encryption setup

  rok

  committed
  e6006bf

  Copy full SHA for e6006bf

• Fix building with ring on wasm

  rok

  committed
  669df5e

  Copy full SHA for 669df5e

• file_decryptor into a seperate module

  rok

  committed
  27f4112

  Copy full SHA for 27f4112

• lint

  rok

  committed
  6acb984

  Copy full SHA for 6acb984

• FileDecryptionProperties should have at least one key

  rok

  committed
  c4860da

  Copy full SHA for c4860da

Commits on Jan 23, 2025

• Move cyphertext reading into decryptor

  rok

  committed
  23375d1

  Copy full SHA for 23375d1

• More tidy up of footer key handling

  adamreeve

  committed
  d44c409

  Copy full SHA for d44c409

• Get column decryptors as RingGcmBlockDecryptor

  adamreeve

  committed
  5e8394c

  Copy full SHA for 5e8394c

Commits on Jan 24, 2025

• Use Arc<dyn BlockDecryptor>

  adamreeve

  committed
  11d2037

  Copy full SHA for 11d2037

• Fix file metadata tests

  adamreeve

  committed
  06cfe65

  Copy full SHA for 06cfe65

• Handle reading plaintext footer files without decryption properties

  adamreeve

  committed
  177d826

  Copy full SHA for 177d826

• Split up encryption modules further

  adamreeve

  committed
  0998b13

  Copy full SHA for 0998b13

• Error instead of panic for AES-GCM-CTR

  adamreeve

  committed
  6de6c35

  Copy full SHA for 6de6c35

Commits on Feb 5, 2025

• load_async

  rok

  committed
  925d86f

  Copy full SHA for 925d86f

• new_with_options

  rok

  committed
  729603a

  Copy full SHA for 729603a

• Add tests

  rok

  committed
  05a06f7

  Copy full SHA for 05a06f7

Commits on Feb 6, 2025

• get_metadata

  rok

  committed
  83fd6ca

  Copy full SHA for 83fd6ca

Commits on Feb 7, 2025

• Add CryptoContext to async_reader

  rok

  committed
  45e1007

  Copy full SHA for 45e1007

• Add row_group_ordinal to InMemoryRowGroup

  rok

  committed
  48ebfda

  Copy full SHA for 48ebfda

• Adjust docstrings

  rok

  committed
  0b55c8d

  Copy full SHA for 0b55c8d

Commits on Feb 10, 2025

• Apply suggestions from code review

  Show description for b960466
  

  rok

  and

  adamreeve

  authored
  b960466

  Copy full SHA for b960466

• Review feedback

  rok

  committed
  b18b099

  Copy full SHA for b18b099

• move file_decryption_properties into ArrowReaderOptions

  rok

  committed
  5a82128

  Copy full SHA for 5a82128

• make create_page_aad method of CryptoContext

  rok

  committed
  df99b9a

  Copy full SHA for df99b9a

• make create_page_aad method of CryptoContext

  rok

  committed
  f411465

  Copy full SHA for f411465

• first commit

  Show description for 2085557
  

  ggershinsky

  authored and

  rok

  committed
  2085557

  Copy full SHA for 2085557

• Add FileEncryptionProperties to WriterProperties

  rok

  committed
  2bd3069

  Copy full SHA for 2bd3069

• Update lib.rs

  Show description for b6ef006
  

  corwinjoy

  authored and

  rok

  committed
  b6ef006

  Copy full SHA for b6ef006

• Update paths. Add notes on where encryption is needed.

  

  corwinjoy

  authored and

  rok

  committed
  5de0a55

  Copy full SHA for 5de0a55

• Attempt to add decryption to ArrowReaderMetadata::load_async

  

  corwinjoy

  authored and

  rok

  committed
  27c578b

  Copy full SHA for 27c578b

• Update async_reader with latest from Rok

  

  corwinjoy

  authored and

  rok

  committed
  f6cd160

  Copy full SHA for f6cd160

• Add changes needed to access encryption from datafusion.

  

  corwinjoy

  authored and

  rok

  committed
  1b385c8

  Copy full SHA for 1b385c8

• post rebase

  rok

  committed
  8f94e2c

  Copy full SHA for 8f94e2c