Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201702102318.v1ANIcPi020401@ip-10-146-233-104.ec2.internal>
Date: Fri, 10 Feb 2017 23:18:38 +0000
From: "Sailesh Mukil (Code Review)" <gerrit@cloudera.org>
To: Henry Robinson <henry@cloudera.com>, impala-cr@cloudera.com,
        reviews@impala.incubator.apache.org
Reply-To: sailesh@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4905=3A_Reduce_coordinator_lock_contention_in_RPC_handler=0A?=
In-Reply-To: <gerrit.1486766085000.Id7599780785c4e9306711f535bf4726a247873e2@gerrit.cloudera.org>
References: <gerrit.1486766085000.Id7599780785c4e9306711f535bf4726a247873e2@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.7
archived-at: Fri, 10 Feb 2017 23:18:41 -0000

Sailesh Mukil has posted comments on this change.

Change subject: IMPALA-4905: Reduce coordinator lock contention in RPC handler
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/5971/1/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

PS1, Line 829:   for (const InstanceState* state: fragment_instance_states_) {
             :     files_to_move.insert(
             :         state->insert_status().files_to_move.begin(),
             :         state->insert_status().files_to_move.end());
             : 
             :     for (const PartitionStatusMap::value_type& partition:
             :              state->insert_status().per_partition_status) {
             :       TInsertPartitionStatus* status = &(per_partition_status_[partition.first]);
             :       status->__set_num_modified_rows(
             :           status->num_modified_rows + partition.second.num_modified_rows);
             :       status->__set_kudu_latest_observed_ts(std::max(
             :               partition.second.kudu_latest_observed_ts, status->kudu_latest_observed_ts));
             :       status->__set_id(partition.second.id);
             :       status->__set_partition_base_dir(partition.second.partition_base_dir);
             : 
             :       if (partition.second.__isset.stats) {
             :         if (!status->__isset.stats) status->__set_stats(TInsertStats());
             :         DataSink::MergeDmlStats(partition.second.stats, &status->stats);
             :       }
             :     }
             :   }
I expect that large INSERT queries will take a while longer to run because of this change. Are we okay with that?


PS1, Line 1555: DCHECK_GT(num_remaining_fragment_instances_.Load(), 0);
You can move this DCHECK to L1557 and convert it to:
DCHECK_GE(num_remaining, 0);


PS1, Line 1562: num_remaining_fragment_instances_.Load();
Just print "num_remaining" ?
It's guaranteed that someone will print the most updated value if they race between 1556 and here. I don't see any reason to potentially lock the bus again for this.


-- 
To view, visit http://gerrit.cloudera.org:8080/5971
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id7599780785c4e9306711f535bf4726a247873e2
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-HasComments: Yes