hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Vijayaraghavan <>
Subject Re: Tez : Anyway to avoid creating subdirectories by "Insert with union all² ?
Date Wed, 19 Aug 2015 23:30:54 GMT
> Is there anyway to avoid creating sub-directories? Or this is by design
>and can not be changed?

This is because of the way file-formats generate hadoop name files without

For instance, any change to that would break Parquet-MR for Tez. That's
why we generate a compatible, but colliding
artificially for Tez jobs.

³Map 1² and ³Map 2² would both have an attempt 0 of task 1, generating
colliding file names (0001_0).

The easy workaround is a ³re-load² of the table.

insert overwrite table h1_passwords_target select * from

The slightly more complex one is to add a DISTRIBUTE BY & trigger a
reducer after the UNION ALL.


View raw message