hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Re: Review Request 37156: HIVE-7476 : CTAS does not work properly for s3
Date Thu, 06 Aug 2015 19:36:09 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37156/
-----------------------------------------------------------

(Updated Aug. 6, 2015, 7:36 p.m.)


Review request for hive and Lenni Kuff.


Changes
-------

Address review comments from Lenni.


Bugs: HIVE-7476
    https://issues.apache.org/jira/browse/HIVE-7476


Repository: hive-git


Description
-------

Currently, CTAS is broken when target is on S3 and source tables are not, or more generally,
where source and target tables are on different file systems.  

Mainly the issues was that during the Move operation (last stage of CTAS), it was using the
destination FileSystem object to run the operations on both the source/dest files, thus error
when running on a source.  The fix is to use the source FileSystem to run operations on the
source file, and the dest FileSystem to run operations on the dest File.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 0a466e4 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5840802 

Diff: https://reviews.apache.org/r/37156/diff/


Testing
-------

Manually ran CTAS to create a table on S3.


Thanks,

Szehon Ho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message