hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bin Wang <binwang...@gmail.com>
Subject Read HFile from Local file system for studying and testing
Date Fri, 15 Apr 2016 03:03:16 GMT
Hi there,

I have a HFile that I copied from HDFS down to my laptop.

I am trying to use the HFile.createreader to read in that HFile and play
with it. However, it was really hard to figure that out how to read it from
local file system.

public class MyHbase {
        public static void main(String[] args) throws IOException {

            Path p = new Path("<path>/Desktop/testHFile");
            Configuration conf = new Configuration();
    //      FileSystem localfs = FileSystem.newInstanceLocal(conf);
    //      FileSystem localfs = FileSystem.getLocal(conf).getRawFileSystem();
    //      FileSystem localfs = p.getFileSystem(conf);
             RawLocalFileSystem localfs = new RawLocalFileSystem();
            System.out.println(localfs.toString());
            HFile.Reader reader = HFile.createReader(localfs, p, new
CacheConfig(conf), conf);
            reader.loadFileInfo();

            HFileScanner scanner = reader.getScanner(false, false);
            scanner.seekTo();
            int count = 0;
            do {
                count++;
            } while (scanner.next());
            System.out.println(count);
            reader.close();
        }
    }

And it is giving the error message:

    LocalFS
    16/04/13 14:54:25 INFO hfile.CacheConfig: Allocating LruBlockCache
size=1.42 GB, blockSize=64 KB
    16/04/13 14:54:25 INFO hfile.CacheConfig:
blockCache=LruBlockCache{blockCount=0, currentSize=1567264,
freeSize=1525578848, maxSize=1527146112, heapSize=1567264,
minSize=1450788736, minFactor=0.95, multiSize=725394368,
multiFactor=0.5, singleSize=362697184, singleFactor=0.25},
cacheDataOnRead=true, cacheDataOnWrite=false,
cacheIndexesOnWrite=false, cacheBloomsOnWrite=false,
cacheEvictOnClose=false, cacheDataCompressed=false,
prefetchOnOpen=false
    Exception in thread "main" java.lang.NullPointerException
        at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
        at org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:98)
        at org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:79)
        at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:519)
        at myhbase.MyHbase.main(MyHbase.java:23)


I asked this question
<http://stackoverflow.com/questions/36609428/trying-to-read-hfile-using-hfile-createreader-from-local-file-system>on
Stackoverflow and seems like there are not that many hbase people there.

Can anyone on this mailing-list help me on that?

Best regards,

Bin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message