cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-9304) COPY TO improvements
Date Mon, 26 Oct 2015 07:32:27 GMT


Stefania commented on CASSANDRA-9304:

Actually it was bugging me too much and so I fixed it in [this commit|].

Pre-fetching the formatters helped a bit but by far the biggest culprit was this:

 def format_value_blob(val, colormap, **_):
-    bval = '0x' + ''.join('%02x' % c for c in val)
+    bval = '0x' + binascii.hexlify(val)
     return colorme(bval, colormap, 'blob')

Disabling coloring also helps (50 seconds vs 35 seconds for 2M records). This part is a bit
of a hack, let me know if you prefer to pass a new parameter to all formatters. 

> COPY TO improvements
> --------------------
>                 Key: CASSANDRA-9304
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Stefania
>            Priority: Minor
>              Labels: cqlsh
>             Fix For: 3.x, 2.1.x, 2.2.x
> COPY FROM has gotten a lot of love.  COPY TO not so much.  One obvious improvement could
be to parallelize reading and writing (write one page of data while fetching the next).

This message was sent by Atlassian JIRA

View raw message