impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huaisi Xu (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3499: Split catalog update
Date Fri, 17 Jun 2016 18:34:37 GMT
Huaisi Xu has submitted this change and it was merged.

Change subject: IMPALA-3499: Split catalog update
......................................................................


IMPALA-3499: Split catalog update

JNI does not support writing java byte array larger than 2GB.
Instead of passing a single serialized update to frontend,
this patch splits the update into a vector of updates less
than 500MB each. Then they are serialized, sent to frontend,
deserialized and merged before calling
Frontend::updateCatalogCache().

Change-Id: I176db25124a32944f2396ce8aafbed49cac95928
Reviewed-on: http://gerrit.cloudera.org:8080/3067
Reviewed-by: Huaisi Xu <hxu@cloudera.com>
Tested-by: Huaisi Xu <hxu@cloudera.com>
---
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/util/jni-util.h
M fe/src/main/java/com/cloudera/impala/service/JniFrontend.java
5 files changed, 85 insertions(+), 20 deletions(-)

Approvals:
  Huaisi Xu: Looks good to me, approved; Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/3067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I176db25124a32944f2396ce8aafbed49cac95928
Gerrit-PatchSet: 26
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Huaisi Xu <hxu@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Huaisi Xu <hxu@cloudera.com>

Mime
View raw message