cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hobbs (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
Date Tue, 24 Mar 2015 17:00:54 GMT


Tyler Hobbs commented on CASSANDRA-8970:

Hi [~aploetz], thanks for the patch!

It seems to me that the biggest problem raised in this ticket is that while {{COPY TO}} uses
the {{time_format}} property to encode timestamps when writing to a CSV, {{COPY FROM}} does
_not_ use that property.  It seems like {{COPY FROM}} should use {{time_format}} to convert
the date/timestamp to a long numeric timestamp (which Cassandra can interpret properly). 
Would you agree with that?  (I could open a separate ticket for that.)

Besides that, your patch to add a {{TIME_FORMAT}} option to {{COPY TO}} looks fine to me.

> Allow custom time_format on cqlsh COPY TO
> -----------------------------------------
>                 Key: CASSANDRA-8970
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Tools
>            Reporter: Aaron Ploetz
>            Priority: Trivial
>              Labels: cqlsh
>             Fix For: 2.1.4
>         Attachments: CASSANDRA-8970.patch
>   Original Estimate: 4h
>  Remaining Estimate: 4h
> When executing a COPY TO from cqlsh, the user is currently has no control over the format
of exported timestamp columns.  If the user has indicated a {{time_format}} in their cqlshrc
file, that format will be used.  Otherwise, the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is not valid
when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified in their
cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to formatted
date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit cqlsh,
modify the {{time_format}} property in cqlshrc, and restart cqlsh.  The ability to specify
a COPY option of TIME_FORMAT with a Python strftime format, would allow the user to quickly
alter the timestamp format for export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH DELIMITER='|'
AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}

This message was sent by Atlassian JIRA

View raw message