impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4905: Reduce coordinator lock contention in RPC handler
Date Fri, 10 Feb 2017 23:18:38 GMT
Sailesh Mukil has posted comments on this change.

Change subject: IMPALA-4905: Reduce coordinator lock contention in RPC handler
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/5971/1/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

PS1, Line 829:   for (const InstanceState* state: fragment_instance_states_) {
             :     files_to_move.insert(
             :         state->insert_status().files_to_move.begin(),
             :         state->insert_status().files_to_move.end());
             : 
             :     for (const PartitionStatusMap::value_type& partition:
             :              state->insert_status().per_partition_status) {
             :       TInsertPartitionStatus* status = &(per_partition_status_[partition.first]);
             :       status->__set_num_modified_rows(
             :           status->num_modified_rows + partition.second.num_modified_rows);
             :       status->__set_kudu_latest_observed_ts(std::max(
             :               partition.second.kudu_latest_observed_ts, status->kudu_latest_observed_ts));
             :       status->__set_id(partition.second.id);
             :       status->__set_partition_base_dir(partition.second.partition_base_dir);
             : 
             :       if (partition.second.__isset.stats) {
             :         if (!status->__isset.stats) status->__set_stats(TInsertStats());
             :         DataSink::MergeDmlStats(partition.second.stats, &status->stats);
             :       }
             :     }
             :   }
I expect that large INSERT queries will take a while longer to run because of this change.
Are we okay with that?


PS1, Line 1555: DCHECK_GT(num_remaining_fragment_instances_.Load(), 0);
You can move this DCHECK to L1557 and convert it to:
DCHECK_GE(num_remaining, 0);


PS1, Line 1562: num_remaining_fragment_instances_.Load();
Just print "num_remaining" ?
It's guaranteed that someone will print the most updated value if they race between 1556 and
here. I don't see any reason to potentially lock the bus again for this.


-- 
To view, visit http://gerrit.cloudera.org:8080/5971
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id7599780785c4e9306711f535bf4726a247873e2
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message