Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 26 Oct 2015 07:32:27 +0000 (UTC)
From: "Stefania (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12827319.1430838774000.56819.1445844747993@Atlassian.JIRA>
In-Reply-To: <JIRA.12827319.1430838774000@Atlassian.JIRA>
References: <JIRA.12827319.1430838774000@Atlassian.JIRA>
 <JIRA.12827319.1430838774384@arcas>
Subject: [jira] [Commented] (CASSANDRA-9304) COPY TO improvements
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973855#comment-14973855 ] 

Stefania commented on CASSANDRA-9304:
-------------------------------------

Actually it was bugging me too much and so I fixed it in [this commit|https://github.com/stef1927/cassandra/commit/351901d65d4bcc9c03277a14a804cad996bb53a7].

Pre-fetching the formatters helped a bit but by far the biggest culprit was this:

{code}
 @formatter_for('bytearray')
 def format_value_blob(val, colormap, **_):
-    bval = '0x' + ''.join('%02x' % c for c in val)
+    bval = '0x' + binascii.hexlify(val)
     return colorme(bval, colormap, 'blob')
 formatter_for('buffer')(format_value_blob)
{code}

Disabling coloring also helps (50 seconds vs 35 seconds for 2M records). This part is a bit of a hack, let me know if you prefer to pass a new parameter to all formatters. 

> COPY TO improvements
> --------------------
>
>                 Key: CASSANDRA-9304
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9304
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Stefania
>            Priority: Minor
>              Labels: cqlsh
>             Fix For: 3.x, 2.1.x, 2.2.x
>
>
> COPY FROM has gotten a lot of love.  COPY TO not so much.  One obvious improvement could be to parallelize reading and writing (write one page of data while fetching the next).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)