hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihong Liu <>
Subject A possible big bug: Delta files created by hive hcatalog streaming cannot be compacted!
Date Tue, 25 Nov 2014 18:54:42 GMT
hive hcatalog streaming will also create a file like bucket_n_flush_length
in each delta directory. Where "n" is the bucket number. But
the compactor.CompactorMR think this file also needs to compact. However
this file of course cannot be compacted, so compactor.CompactorMR will not
continue to do the compaction.

Did a test, after removed the bucket_n_flush_length file, then the "alter
table partition compact" finished successfully. If don't delete that file,
nothing will be compacted.
This is probably a very severity bug. Both 0.13 and 0.14 have this issue.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message