hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Xiao <xiaotao.cs....@gmail.com>
Subject Why so many unexpected files like partitions_xxxx are created?
Date Mon, 16 Dec 2013 10:47:58 GMT
I imported data into HBase in the fashion of bulk load,  but after that I
found many unexpected file were created in the HDFS directory of
/user/root/, and they like these:

... ...
... ...

It seems that they are HFiles, but I don't know why the were created here?

I bulk load data into HBase in the following way:

Firstly,   I wrote a MapReduce program which only has map tasks. The map
tasks read some text data and emit them in the form of  RowKey and
KeyValue.The following is my program:

        protected void map(NullWritable NULL, GtpcV1SignalWritable signal,
Context ctx) throws InterruptedException, IOException {
            String strRowkey = xxx;
            byte[] rowkeyBytes = Bytes.toBytes(strRowkey);



            KeyValue kv = new KeyValue(rowkeyBytes, Family_A, Qualifier_Q,
            ctx.write(rowkey, kv);

            kv = new KeyValue(rowkeyBytes, Family_B, Qualifier_Q,
            ctx.write(rowkey, kv);

after the MR programs finished, there were several HFiles generated in the
output directory I specified.

Then I bean to load these HFiles into HBase using the following command:
       hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles
HFiles-Dir  MyTable

Finally , I could see that the data were indeed loaded into the table in

But, I could also see that there were many unexpected files generated in
the HDFS directory of  /user/root/,  just as I have mentioned at the
begining of this mail,  and I did not specify any files to be produced in
this directory.

What happened ? Who can tell me what there files are and who produced them?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message