hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <alanfga...@gmail.com>
Subject Re: writing to partitions with HCatWriter
Date Fri, 13 Feb 2015 22:31:39 GMT
This sounds like a bug in the HCatWriter.  You should file a JIRA so we 
can track it.

Alan.

> Nathan Bamford <mailto:nathan.bamford@redpoint.net>
> February 13, 2015 at 13:50
>
> Hi all,
>
>   I'm using HCatWriter in a java program to write records to a 
> partitioned Hive table. It works great, but I notice it leaves behind 
> the _SCRATCH directories it uses for staging (before HCatWriter.commit 
> is called).
>
>   When it's all said and done, the partitioned records are in the 
> appropriate directory (e.g. state=CO), and the _SCRATCH directories 
> are empty.
>
>   I tried running a load of the same records/partition values via the 
> CLI, and after the mapreduce job has finished, the _SCRATCH 
> directories are cleaned up. Only the finished partition dirs remain.
>
>   Is there something I'm  missing with HCatWriter?
>
>
> Thanks,
>
>
> Nathan
>
>

Mime
View raw message