ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Izhikov <nizhi...@apache.org>
Subject Re: data extractor
Date Sun, 01 Jul 2018 08:48:23 GMT
Hello, Dmitriy.

Should we support extraction of encrypted data?

There will be 2 type of keys we should load to successfully extract data:

* master key: keystore + password required.
* cache keys: masterkey + access to metastore required.

TDE task is almost done, please, take a look.

ticket - https://issues.apache.org/jira/browse/IGNITE-8485
prototype - https://github.com/apache/ignite/pull/4167
spi - https://github.com/apache/ignite/pull/4167/files#diff-9a792ab0e6971f202d22d530af0ac933

В Сб, 30/06/2018 в 22:37 +0300, Dmitriy Govorukhin пишет:
> Igniters,
> 
> I am working on IGNITE-7644 (export all key-value data from a persisted partition),
> it will be command line tool for extracting data from Ignite partition file without the
need to start node.
> The main motivation is to have a lifebuoy in case if a file has damage for some reason.

> 
> I suggest simple API and two commands for the first implementation:
> 
> -c
> --CRC [srcPath] - check CRC for all(or by type) pages in partition
> 
> -e
> --extract [srcPath] [outPath] - dump all survey data from partition to another file with
raw key/value pair format 
> (required graceful stop for a node, not necessary after --restore will be implemented)
> 
> Output file format see in attached, this format does not contain any index inside but
it is very simple and
> flexible for future works with raw key/value data.
> 
> Future features:
> -u
> --upload - reload raw key/value pairs to node
> 
> -s
> --status - check current node file status, need binary recovery or not (node crash on
the middle of a checkpoint)
> 
> -r
> --restore - restore binary consistency (finish checkpoint, required WAL file for recovery)
> 
> Let's start a discussion, any comments are welcome.
> 
Mime
View raw message