hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Kelkar <rohitkel...@gmail.com>
Subject Scanner problem after bulk load hfile
Date Fri, 12 Jul 2013 15:50:19 GMT
I am having problems while scanning a table created using HFile.
This is what I am doing -
Once Hfile is created I use following code to bulk load

LoadIncrementalHFiles loadTool = new LoadIncrementalHFiles(conf);
HTable myTable = new HTable(conf, mytablename.getBytes());
loadTool.doBulkLoad(new Path(outputHFileBaseDir + "/" + mytablename),

Then scan the table using-

HTable table = new HTable(conf, mytable);
Scan scan = new Scan();
scan.addColumn("cf".getBytes(), "q".getBytes());
ResultScanner scanner = table.getScanner(scan);
for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
numRowsScanned += 1;

This code crashes with following error - http://pastebin.com/SeKAeAST
If I remove the scan.addColumn from the code then the code works.

Similarly on the hbase shell -
- A simple count 'mytable' in hbase shell gives the correct count.
- A scan 'mytable' gives correct results.
- get 'mytable', 'myrow', 'cf:q' crashes

The hadoop dfs -ls /hbase/mytable shows the .tableinfo, .tmp, the directory
for region etc.

Now if I do a major_compact 'mytable' and then execute my code with the
scan.addColumn statement then it works. Also the get 'mytable', 'myrow',
'cf:q' works.

My question is
What is major_compact doing to enable the scanner that the
LoadIncrementalFiles tool is not? I am sure I am missing a step after the

- R

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message