cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-9303) Match cassandra-loader options in COPY FROM
Date Tue, 01 Dec 2015 08:13:11 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015299#comment-15015299
] 

Stefania edited comment on CASSANDRA-9303 at 12/1/15 8:12 AM:
--------------------------------------------------------------

Thank you for your feedback, here is the -updated versions of the progress tables. I will
try and see if we can add support for importing multiple files as well- latest progress:

h3. Importing

||cassandra-loader||COPY FROM||description||status||
|-configFile filename|configfile|File with configuration options|done|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|done|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-skipRows skipRows|skiprows|Number of rows to skip|done|
|-skipCols columnsToSkip|skipcols|Comma-separated list of columsn to skip|done|
|-maxRows maxRows|maxrows|Maximum number of rows to read (-1 means all)|done|
|-maxErrors maxErrors|maxparseerrors|Maximum parse errors to endure|done|
|-badDir badDirectory|errFile|Directory for where to place badly parsed rows.|done|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|-consistencyLevel CL| |Consistency level|already available via cqlsh|
|-numFutures numFutures|jobs|Number of CQL futures to keep in flight|already available|
|-batchSize batchSize|minbatchsize, maxbatchsize|Number of INSERTs to batch together|alredy
available|
|-decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|-boolStyle boolStyleString|boolstyle|Style for booleans|done|
|-numThreads numThreads|numprocesses|Number of concurrent threads (files) to load|done|
|-queryTimeout # seconds|pagetimeout|Query timeout (in seconds)|already available|
|-numRetries numRetries|maxattempts|Number of times to retry the INSERT|already available|
|-maxInsertErrors # errors|maxinserterrors|Maximum INSERT errors to endure|done|
|-rate rows-per-second|ingestrate|Maximum insert rate|done|
|-progressRate num txns|reportfrequency|How often to report the insert rate|already available|
|-rateFile filename|ratefile|Where to print the rate statistics|done|
|-successDir dir| |Directory where to move successfully loaded files|won't implement|
|-failureDir dir| |Directory where to move files that did not successfully load|won't implement|


h3. Exporting

||cassandra-unloader||COPY TO||description||status||
|configFile filename|configfile|File with configuration options|done|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|already available|
|-nullString nullString|nullval|String that signifies NULL|already available|
| |skipcols|Comma-separated list of columsn to skip|done|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|consistencyLevel CL| |Consistency level|already available via cqlsh|
|decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|boolStyle boolStyleString|boolstyle|Style for booleans|done|
|numThreads numThreads|numprocesses|Number of concurrent threads to unload|done|
|beginToken tokenString|begintoken|Begin token|done|
|endToken tokenString|endtoken|End token|done|
| |ratefile|Where to print the rate statistics|done|

-Where it says _done_, I'm actually still working on automated tests.-


was (Author: stefania):
Thank you for your feedback, here is the -updated versions of the progress tables. I will
try and see if we can add support for importing multiple files as well- latest progress:

h3. Importing

||cassandra-loader||COPY FROM||description||status||
|-configFile filename|configfile|File with configuration options|done|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|done|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-skipRows skipRows|skiprows|Number of rows to skip|done|
|-skipCols columnsToSkip|skipcols|Comma-separated list of columsn to skip|done|
|-maxRows maxRows|maxrows|Maximum number of rows to read (-1 means all)|done|
|-maxErrors maxErrors|maxparseerrors|Maximum parse errors to endure|done|
|-badDir badDirectory|errFile|Directory for where to place badly parsed rows.|done|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|-consistencyLevel CL| |Consistency level|already available via cqlsh|
|-numFutures numFutures|jobs|Number of CQL futures to keep in flight|already available|
|-batchSize batchSize|minbatchsize, maxbatchsize|Number of INSERTs to batch together|alredy
available|
|-decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|-boolStyle boolStyleString|boolstyle|Style for booleans|done|
|-numThreads numThreads|numprocesses|Number of concurrent threads (files) to load|done|
|-queryTimeout # seconds|pagetimeout|Query timeout (in seconds)|already available|
|-numRetries numRetries|maxattempts|Number of times to retry the INSERT|already available|
|-maxInsertErrors # errors|maxinserterrors|Maximum INSERT errors to endure|done|
|-rate rows-per-second|ingestrate|Maximum insert rate|done|
|-progressRate num txns|reportfrequency|How often to report the insert rate|already available|
|-rateFile filename|ratefile|Where to print the rate statistics|done|
|-successDir dir| |Directory where to move successfully loaded files|won't implement|
|-failureDir dir| |Directory where to move files that did not successfully load|won't implement|


h3. Exporting

||cassandra-unloader||COPY TO||description||status||
|configFile filename|configfile|File with configuration options|done|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|already available|
|-nullString nullString|nullval|String that signifies NULL|already available|
| |skipcols|Comma-separated list of columsn to skip|done|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|consistencyLevel CL| |Consistency level|already available via cqlsh|
|decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|boolStyle boolStyleString|boolstyle|Style for booleans|done|
|numThreads numThreads|numprocesses|Number of concurrent threads to unload|done|
|beginToken tokenString|begintoken|Begin token|done|
|endToken tokenString|endtoken|End token|done|
| |ratefile|Where to print the rate statistics|done|

Where it says _done_, I'm actually still working on automated tests.

> Match cassandra-loader options in COPY FROM
> -------------------------------------------
>
>                 Key: CASSANDRA-9303
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9303
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Tools
>            Reporter: Jonathan Ellis
>            Assignee: Stefania
>            Priority: Critical
>             Fix For: 2.1.x
>
>
> https://github.com/brianmhess/cassandra-loader added a bunch of options to handle real
world requirements, we should match those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message