db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Thalamati (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-1156) allow the encrypting of an existing unencrypted db and allow the re-encrypting of an existing encrypted db
Date Sat, 05 Aug 2006 03:12:14 GMT
     [ http://issues.apache.org/jira/browse/DERBY-1156?page=all ]

Suresh Thalamati updated DERBY-1156:

    Attachment: reencrypt_6.diff

DERBY -1156 (partial) (re) encryption of an existing database.

This patch adds code required to correctly handle crash/recovery during 
encryption of an un-encrypted database and encryption of an encrypted 
database with a new key.

Crashes before updating the database with the new encryption key 
are handled by recovery, those log records are encrypted old key. 

Crashes after updates to the service.properties with a new encryption key 
is handled by using the new status flag (derby.storage.databaseEncryptionStatus)  
to track the changes and by writing the transaction log encrypted with 
new key to a new log file. 

Logging system can handle only one encryption key, which means the 
recovery can read the log with a new key or the old key. Checkpoint 
log record is also encrypted. So it is necessary for the system
to find a checkpoint in the log that it can decrypt using the encryption 
key stored in the service.properties or the one provided by the user. 
This is ensured by  by writing  COMMIT and CHECKPOINT log records
into a new log file and delete that file on-reboot if there is a crash 
before checkpoint records are updated. 

CHECKPOINT and COMMIT is done after setting the databaseEncryptionStatus to 
IN-PROGRESS in the service.properties.On a reboot if databaseEncryptionStatus 
is  IN-PROGRESS,  then engine first checks if checkpoint is in the last 
log file , it it is then (re) encryption is complete otherwise it deletes the 
last log file before doing recovery. Because the last log file also 
had the commit record , it is also gone; Now recovery sees log only encrypted 
with the old key and no end for re-encryption transaction, so it the (re) 
encryption work rolled back and database is brought to the state it was 
before (re) encryption started. 

If engine find a checkpoint in the last log file when databaseEncryptionStatus
is IN-PROGRESS , the it is clear that checkpoint is encrypted with the new 
key; so it does any cleanup required. 

Added new test cases using debug flags to crash at critical 
point during (re) encryption and recovery of failed (re)encryption. 

svn stat :

M      java\engine\org\apache\derby\impl\store\raw\log\ReadOnly.java
M      java\engine\org\apache\derby\impl\store\raw\log\LogToFile.java
M      java\engine\org\apache\derby\impl\store\raw\RawStore.java
M      java\engine\org\apache\derby\impl\store\raw\data\EncryptData.java
M      java\engine\org\apache\derby\impl\store\raw\data\BaseDataFileFactory.java

M      java\engine\org\apache\derby\iapi\services\io\FileUtil.java
M      java\engine\org\apache\derby\iapi\store\raw\log\LogFactory.java
M      java\engine\org\apache\derby\iapi\store\raw\data\DataFactory.java
M      java\engine\org\apache\derby\iapi\store\raw\RawStoreFactory.java
M      java\testing\org\apache\derbyTesting\functionTests\tests\store\ReEncryptCrashRecovery.java

> allow the encrypting of an existing unencrypted db and allow the re-encrypting of an
existing encrypted db
> ----------------------------------------------------------------------------------------------------------
>                 Key: DERBY-1156
>                 URL: http://issues.apache.org/jira/browse/DERBY-1156
>             Project: Derby
>          Issue Type: Improvement
>          Components: Store
>    Affects Versions:
>            Reporter: Mike Matrigali
>         Assigned To: Suresh Thalamati
>            Priority: Minor
>             Fix For:
>         Attachments: encryptspec.html, reencrypt_1.diff, reencrypt_2.diff, reencrypt_3.diff,
reencrypt_4.diff, reencrypt_5.diff, reencrypt_6.diff, reencryptspec_1.html
> encrypted database to be re-encrypted with a new password.
> Here are some ideas for an initial implementation.
> The easiest way to do this is to make sure we have exclusive access to the
> data and that no log is required in the new copy of the db.  I want to avoid
> the log as it also is encrypted.  Here is my VERY high level plan:
> 1) Force exclusive access by putting all the work in the low level store,
>    offline boot method.  We will do redo recovery as usual, but at the end
>    there will be an entry point to do the copy/encrypt operation.
> copy/encrypt process:
> 0) The request to encrypt/re-encrypt the db will be handled with a new set
>    of url flags passed into store at boot time.  The new flags will provide
>    the same inputs as the current encrypt flags.  So at high level the
>    request will be "connect db old_encrypt_url_flags; new_encrypt_url_flags".
>    TODO - provide exact new flag syntax.
> 1) Open a transaction do all logged work to do the encryption.  All logging
>    will be done with existing encryption.
> 2) Copy and encrypt every db file in the database.  The target files will
>    be in the data directory.  There will be a new suffix to track the new
>    files, similar to the current process used for handling drop table in
>    a transaction consistent manner without logging the entire table to the log.
>    Entire encrypted destination file is guaranteed synced to disk before
>    transaction commits.  I don't think this part needs to be logged.
>    Files will be read from the cache using existing mechanism and written
>    directly into new encrypted files (new encrypted data does not end up in
>    the cache).
> 3) Switch encrypted files for old files.  Do this under a new log operation
>    so the process can be correctly rolled back if the encrypt db operation
>    transaction fails.  Rollback will do file at a time switches, no reading
>    of encrypted data is necessary.
> 4) log a "change encryption of db" log record, but do not update
>    system.properties with the change.
> 5) commit transaction.
> 6) update system.properties and sync changes.
> 7) TODO - need someway to handle crash between steps 5 and 6.
> 6) checkpoint all data, at this point guaranteed that there is no outstanding
>    transaction, so after checkpoint is done there is no need for the log.
> o there probably should be something that catches a request to encrypt to
>   whatever db was already encrypted with.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message