impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anujphadke (Code Review)" <>
Subject [Impala-CR](cdh5-2.5.0 5.7.x) IMPALA-3732: handle string length overflow in avro files
Date Wed, 29 Jun 2016 20:55:07 GMT
anujphadke has submitted this change and it was merged.

Change subject: IMPALA-3732: handle string length overflow in avro files

IMPALA-3732: handle string length overflow in avro files

Avro string lengths are encoded as 64-bit integers. Impala can only
handle up to 32-bit integers, so we need to be careful about handling
out-of-range integers. Negative integers were already handled by a
previous patch, but if a positive 64-bit integer is truncated to a
32-bit integer, the result can be a negative length.

This patch fixes CHAR/VARCHAR behaviour, where we can just truncate
the string, and STRING, where we can't truncate the string, so must
return an error.

Added unit tests for STRING, CHAR, and VARCHAR that exercise the string
overflow handling.

Change-Id: If6541e7c68255bf599b26386a55057c93e62af51
Reviewed-by: Tim Armstrong <>
Tested-by: Internal Jenkins
(cherry picked from commit e78b6db7e2b334ff88dd5678290f5b932a6a715f)
Tested-by: Anuj Phadke <>
(cherry picked from commit ea0200e1ea6ff0f6e54b22b00b4e25ce95a7c2d9)
Reviewed-by: anujphadke <>
Tested-by: anujphadke <>
M be/src/exec/
M be/src/exec/
M be/src/exec/
M be/src/exec/hdfs-avro-scanner.h
M common/thrift/
5 files changed, 109 insertions(+), 16 deletions(-)

  anujphadke: Looks good to me, approved; Verified

To view, visit
To unsubscribe, visit

Gerrit-MessageType: merged
Gerrit-Change-Id: If6541e7c68255bf599b26386a55057c93e62af51
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.5.0_5.7.x
Gerrit-Owner: anujphadke <>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-Reviewer: anujphadke <>

View raw message