impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3766: Applying LZ4 compression on buffers before spilling
Date Fri, 24 Jun 2016 16:01:09 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-3766:  Applying LZ4 compression on buffers before spilling
......................................................................


Patch Set 1:

It would be good to get some numbers on how this compares to the uncompressed version in terms
of bytes written to disk and end-to-end performance, I think we should get an idea of the
gains before doing the detailed review.

* Total uncompressed bytes versus compressed bytes written - you could add an extra profile
counter for this since we already have bytes_written_counter_.
* End-to-end query time on a release build.
* Disk space saved. This is a little tricky since we allocate 8mb file ranges, but don't write
the whole 8mb. The files will be sparse so we will save disk space, but you need to do something
like this to see it: http://extrabright.com/blog/2010/03/30/how-to-know-if-a-file-on-linux-is-sparse/

-- 
To view, visit http://gerrit.cloudera.org:8080/3478
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4d49bd8d6d7643c84cefd1274c18b52907ca1488
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: anujphadke <aphadke@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: anujphadke <aphadke@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message