Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
Message-Id: <201604200017.u3K0Hn4n007187@ip-10-146-233-104.ec2.internal>
Date: Wed, 20 Apr 2016 00:17:49 +0000
From: "Tim Armstrong (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, dev@impala.incubator.apache.org
CC: Matthew Jacobs <mj@cloudera.com>, Dan Hecht <dhecht@cloudera.com>,
        Michael Ho <kwho@cloudera.com>
Reply-To: tarmstrong@cloudera.com
Subject: 
 =?UTF-8?Q?[Impala-CR](cdh5-trunk)_IMPALA-3317:_fix_crash_in_sorter_when_spilling_zero-length_strings=0A?=
In-Reply-To: 
 <gerrit.1460573003000.I5f64593e94d367d6b6efb61a8b86e35516f18839@gerrit.cloudera.org>
References: 
 <gerrit.1460573003000.I5f64593e94d367d6b6efb61a8b86e35516f18839@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.10-rc0

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-3317: fix crash in sorter when spilling zero-length strings
......................................................................


Patch Set 1:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/2780/1/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:

Line 477:         } else {
> how about: DCHECK_EQ(var_len_blocks_.back(), cur_var_len_block) to support 
Done


Line 564:           cur_sorted_var_len_block->Allocate<uint8_t>(total_var_len);
> same dcheck
Done


Line 732:         // offset in the lower 32 bits.
> Add: see CopyVarLenDataConvertOffset()
Done. I'm going to refactor some of this in a follow-up patch so that the encode/decode methods are adjacent to each other.


Line 827: block_size_
> Let's also dcheck that the calculation to get block_offset fits in 32-bits 
Done


http://gerrit.cloudera.org:8080/#/c/2780/1/testdata/workloads/functional-query/queries/QueryTest/large-sorts.test
File testdata/workloads/functional-query/queries/QueryTest/large-sorts.test:

Line 3: set mem_limit=512m;
> Can you please keep these set statements together with the query ? The quer
I was seeing some weird behaviour. There is actually a bug in the regex that implements the reset that I fixed. For some reason num_nodes doesn't seem to be taking effect when this test file is executed (maybe because exec_single_node_rows_threshold was set) so I changed it to set it in the test vector.


http://gerrit.cloudera.org:8080/#/c/2780/1/tests/query_test/test_queries.py
File tests/query_test/test_queries.py:

Line 86
> why is this deleted?
oops


-- 
To view, visit http://gerrit.cloudera.org:8080/2780
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5f64593e94d367d6b6efb61a8b86e35516f18839
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes