impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-3452: S3: Disable Impala staging for INSERTs via flag for speedup
Date Tue, 03 May 2016 18:13:19 GMT
Sailesh Mukil has posted comments on this change.

Change subject: IMPALA-3452: S3: Disable Impala staging for INSERTs via flag for speedup

Patch Set 2:

File be/src/exec/

Line 285: and the user has specified
        :   // so via the query option S3_SKIP_INSERT_STAGING, in order to achieve better
        :   // performance.
> Talk less about how the option might have got set, and more about what skip

Line 289: output_partition->final_hdfs_file_name_prefix
        :              << "." << output_partition->num_files
        :              << "." << output_partition->writer->file_extension();
> how about creating a:

Line 394: and if the query is not an INSERT OVERWRITE
> say why we can't skip for overwrites (and mention that overwrites aren't af
File be/src/exec/hdfs-table-sink.h:

Line 57: /// Name of the temporary directory that files for this partition are staged to before
       :   /// the coordinator moves them to their permanent location once the query completes.
> Needs an updated comment.

Line 91:   /// This is set if we want to skip the staging step for this partition.
> Say more about why we would want to do this.
File common/thrift/ImpalaService.thrift:

Line 213:   // If true, skips the staging step for INSERTs to S3.
> Be more clear here: not every reader will know what the 'staging step' is. 

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Iff9620d41ba0d5fb1aa0c9f4abb48866fc2b0698
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Sailesh Mukil <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Mostafa Mokhtar <>
Gerrit-Reviewer: Sailesh Mukil <>
Gerrit-HasComments: Yes

View raw message