cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-9303) Match cassandra-loader options in COPY FROM
Date Tue, 24 Nov 2015 08:26:11 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015299#comment-15015299
] 

Stefania edited comment on CASSANDRA-9303 at 11/24/15 8:25 AM:
---------------------------------------------------------------

Thanks for your feedback, here is the updated versions of the progress tables. I will try
and see if we can add support for importing multiple files as well.

h3. Importing

||cassandra-loader||COPY FROM||description||status||
|-configFile filename| |File with configuration options|TODO (extend cqlsh config file)|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|done|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-skipRows skipRows| |Number of rows to skip|TODO|
|-skipCols columnsToSkip|column|Comma-separated list of columsn to skip|already available,
they can specify which columns in the cmd syntax|
|-maxRows maxRows| |Maximum number of rows to read (-1 means all)|TODO|
|-maxErrors maxErrors| |Maximum parse errors to endure|TODO|
|-badDir badDirectory| |Directory for where to place badly parsed rows.|TODO|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|-consistencyLevel CL| |Consistency level|already available via cqlsh|
|-numFutures numFutures|jobs|Number of CQL futures to keep in flight|already available|
|-batchSize batchSize|minbatchsize, maxbatchsize|Number of INSERTs to batch together|alredy
available|
|-decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|-boolStyle boolStyleString|boolstyle|Style for booleans|done|
|-numThreads numThreads|numProcesses|Number of concurrent threads (files) to load|done|
|-queryTimeout # seconds|pageTimeout|Query timeout (in seconds)|already available|
|-numRetries numRetries|maxattempts|Number of times to retry the INSERT|already available|
|-maxInsertErrors # errors| |Maximum INSERT errors to endure|TODO|
|-rate rows-per-second| |Maximum insert rate|TODO (unsure how)|
|-progressRate num txns|reportfrequency|How often to report the insert rate|already available|
|-rateFile filename| |Where to print the rate statistics|TODO|
|-successDir dir| |Directory where to move successfully loaded files|will implement only if
adding support for multi-file import|
|-failureDir dir| |Directory where to move files that did not successfully load|will implement
only if adding support for multi-file import|


h3. Exporting

||cassandra-unloader||COPY TO||description||status||
|configFile filename| |File with configuration options|TODO (extend cqlsh config file)|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|datetimeformat|Date format|already available|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|consistencyLevel CL| |Consistency level|already available via cqlsh|
|decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|boolStyle boolStyleString|boolstyle|Style for booleans|done|
|numThreads numThreads|numprocesses|Number of concurrent threads to unload|done|
|beginToken tokenString|begintoken|Begin token|done|
|endToken tokenString|endtoken|End token|done|

Where it says _done_, I'm actually still working on automated tests.


was (Author: stefania):
Thanks for your feedback, here is the updated versions of the progress tables. I will try
and see if we can add support for importing multiple files as well.

h3. Importing

||cassandra-loader||COPY FROM||description||status||
|-configFile filename| |File with configuration options|TODO (extend cqlsh config file)|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|dtformats|Date format|TODO|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-skipRows skipRows| |Number of rows to skip|TODO|
|-skipCols columnsToSkip|column|Comma-separated list of columsn to skip|already available,
they can specify which columns in the cmd syntax|
|-maxRows maxRows| |Maximum number of rows to read (-1 means all)|TODO|
|-maxErrors maxErrors| |Maximum parse errors to endure|TODO|
|-badDir badDirectory| |Directory for where to place badly parsed rows.|TODO|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|-consistencyLevel CL| |Consistency level|already available via cqlsh|
|-numFutures numFutures|jobs|Number of CQL futures to keep in flight|already available|
|-batchSize batchSize|minbatchsize, maxbatchsize|Number of INSERTs to batch together|alredy
available|
|-decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|-boolStyle boolStyleString|boolstyle|Style for booleans|done|
|-numThreads numThreads|numProcesses|Number of concurrent threads (files) to load|done|
|-queryTimeout # seconds|pageTimeout|Query timeout (in seconds)|already available|
|-numRetries numRetries|maxattempts|Number of times to retry the INSERT|already available|
|-maxInsertErrors # errors| |Maximum INSERT errors to endure|TODO|
|-rate rows-per-second| |Maximum insert rate|TODO (unsure how)|
|-progressRate num txns|reportfrequency|How often to report the insert rate|already available|
|-rateFile filename| |Where to print the rate statistics|TODO|
|-successDir dir| |Directory where to move successfully loaded files|will implement only if
adding support for multi-file import|
|-failureDir dir| |Directory where to move files that did not successfully load|will implement
only if adding support for multi-file import|


h3. Exporting

||cassandra-unloader||COPY TO||description||status||
|configFile filename| |File with configuration options|TODO (extend cqlsh config file)|
|-delim delimiter|delimiter|Delimiter to use|already available|
|-dateFormat dateFormatString|dtformats|Date format|already available|
|-nullString nullString|nullval|String that signifies NULL|already available|
|-port portNumber| |CQL Port Number|already available via cqlsh|
|-user username| |Cassandra username|already available via cqlsh|
|-pw password| |Password for user|already available via cqlsh|
|-ssl-truststore-path path| |Path to SSL truststore|already available via cqlsh|
|-ssl-truststore-pw pwd| |Password for SSL truststore|already available via cqlsh|
|-ssl-keystore-path path| |Path to SSL keystore|already available via cqlsh|
|-ssl-keystore-pw pwd| |Password for SSL keystore|already available via cqlsh|
|consistencyLevel CL| |Consistency level|already available via cqlsh|
|decimalDelim decimalDelim|decimalsep|Decimal delimiter|done|
| |thousandssep|Thousands delimiter|done|
|boolStyle boolStyleString|boolstyle|Style for booleans|done|
|numThreads numThreads|numprocesses|Number of concurrent threads to unload|done|
|beginToken tokenString|begintoken|Begin token|done|
|endToken tokenString|endtoken|End token|done|

Where it says _done_, I'm actually still working on automated tests.

> Match cassandra-loader options in COPY FROM
> -------------------------------------------
>
>                 Key: CASSANDRA-9303
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9303
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Tools
>            Reporter: Jonathan Ellis
>            Assignee: Stefania
>            Priority: Critical
>             Fix For: 2.1.x
>
>
> https://github.com/brianmhess/cassandra-loader added a bunch of options to handle real
world requirements, we should match those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message