parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From w...@apache.org
Subject [parquet-testing] branch master updated: PARQUET-1621: [C++] Add encrypted parquet files to apache parquet-testing repository (#7)
Date Thu, 18 Jul 2019 16:28:46 GMT
This is an automated email from the ASF dual-hosted git repository.

wesm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-testing.git


The following commit(s) were added to refs/heads/master by this push:
     new a277dc4  PARQUET-1621: [C++] Add encrypted parquet files to apache parquet-testing
repository (#7)
a277dc4 is described below

commit a277dc4e55ded3e3ea27dab1e4faf98c112442df
Author: revit1976 <51282986+revit1976@users.noreply.github.com>
AuthorDate: Thu Jul 18 19:28:41 2019 +0300

    PARQUET-1621: [C++] Add encrypted parquet files to apache parquet-testing repository (#7)
    
    * Add encrypted parquet files to apache parquet-testing repository
    
    * Update README.md
    
    Add encryption keys and explanation to data/README.md
    
    * Update README.md
    
    Additional fixes to encrypted files section.
---
 data/README.md                                     |  33 ++++++++++++++++++++-
 data/encrypt_columns_and_footer.parquet.encrypted  | Bin 0 -> 4930 bytes
 ...ncrypt_columns_and_footer_aad.parquet.encrypted | Bin 0 -> 4938 bytes
 ...ncrypt_columns_and_footer_ctr.parquet.encrypted | Bin 0 -> 4864 bytes
 ...nd_footer_disable_aad_storage.parquet.encrypted | Bin 0 -> 4930 bytes
 ...rypt_columns_plaintext_footer.parquet.encrypted | Bin 0 -> 5083 bytes
 data/uniform_encryption.parquet.encrypted          | Bin 0 -> 5483 bytes
 7 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/data/README.md b/data/README.md
index 9958630..b517afc 100644
--- a/data/README.md
+++ b/data/README.md
@@ -1,3 +1,34 @@
 # Test data files for Parquet compatibility and regression testing
 
-TODO: Document what each file is
\ No newline at end of file
+TODO: Document what each file is
+
+## Encrypted Files
+
+Tests files with .parquet.encrypted suffix are encrypted using Parquet Modular Encryption.
+
+A detailed description of the Parquet Modular Encryption specification can be found here:
+```
+ https://github.com/apache/parquet-format/blob/encryption/Encryption.md
+```
+
+Following are the keys and key ids (when using key\_retriever) used to encrypt the encrypted
columns and footer in the all the encrypted files:
+* Encrypted/Signed Footer:
+  * key:   {0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5}
+  * key_id: "kf"
+* Encrypted column named double_field:
+  * key:  {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,0}
+  * key_id: "kc1"
+* Encrypted column named float_field:
+  * key: {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,1}
+  * key_id: "kc2"
+
+The following files are encrypted with AAD prefix "tester":
+1. encrypt\_columns\_and\_footer\_disable\_aad\_storage.parquet.encrypted
+2. encrypt\_columns\_and\_footer\_aad.parquet.encrypted
+
+
+A sample that reads and checks these files can be found at the following tests:
+```
+cpp/src/parquet/encryption-read-configurations-test.cc
+cpp/src/parquet/test-encryption-util.h
+```
diff --git a/data/encrypt_columns_and_footer.parquet.encrypted b/data/encrypt_columns_and_footer.parquet.encrypted
new file mode 100644
index 0000000..460d05b
Binary files /dev/null and b/data/encrypt_columns_and_footer.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_aad.parquet.encrypted b/data/encrypt_columns_and_footer_aad.parquet.encrypted
new file mode 100644
index 0000000..863ccb2
Binary files /dev/null and b/data/encrypt_columns_and_footer_aad.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_ctr.parquet.encrypted b/data/encrypt_columns_and_footer_ctr.parquet.encrypted
new file mode 100644
index 0000000..0591df4
Binary files /dev/null and b/data/encrypt_columns_and_footer_ctr.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted b/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted
new file mode 100644
index 0000000..abd40e7
Binary files /dev/null and b/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted
differ
diff --git a/data/encrypt_columns_plaintext_footer.parquet.encrypted b/data/encrypt_columns_plaintext_footer.parquet.encrypted
new file mode 100644
index 0000000..0625080
Binary files /dev/null and b/data/encrypt_columns_plaintext_footer.parquet.encrypted differ
diff --git a/data/uniform_encryption.parquet.encrypted b/data/uniform_encryption.parquet.encrypted
new file mode 100644
index 0000000..048f35f
Binary files /dev/null and b/data/uniform_encryption.parquet.encrypted differ


Mime
View raw message