hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Read HFile from Local file system for studying and testing
Date Fri, 15 Apr 2016 03:47:59 GMT
Please take a look at:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java

There're various tests under hbase-server/src/test which demonstrate the
usage.

FYI

On Thu, Apr 14, 2016 at 8:03 PM, Bin Wang <binwang.cu@gmail.com> wrote:

> Hi there,
>
> I have a HFile that I copied from HDFS down to my laptop.
>
> I am trying to use the HFile.createreader to read in that HFile and play
> with it. However, it was really hard to figure that out how to read it from
> local file system.
>
> public class MyHbase {
>         public static void main(String[] args) throws IOException {
>
>             Path p = new Path("<path>/Desktop/testHFile");
>             Configuration conf = new Configuration();
>     //      FileSystem localfs = FileSystem.newInstanceLocal(conf);
>     //      FileSystem localfs =
> FileSystem.getLocal(conf).getRawFileSystem();
>     //      FileSystem localfs = p.getFileSystem(conf);
>              RawLocalFileSystem localfs = new RawLocalFileSystem();
>             System.out.println(localfs.toString());
>             HFile.Reader reader = HFile.createReader(localfs, p, new
> CacheConfig(conf), conf);
>             reader.loadFileInfo();
>
>             HFileScanner scanner = reader.getScanner(false, false);
>             scanner.seekTo();
>             int count = 0;
>             do {
>                 count++;
>             } while (scanner.next());
>             System.out.println(count);
>             reader.close();
>         }
>     }
>
> And it is giving the error message:
>
>     LocalFS
>     16/04/13 14:54:25 INFO hfile.CacheConfig: Allocating LruBlockCache
> size=1.42 GB, blockSize=64 KB
>     16/04/13 14:54:25 INFO hfile.CacheConfig:
> blockCache=LruBlockCache{blockCount=0, currentSize=1567264,
> freeSize=1525578848, maxSize=1527146112, heapSize=1567264,
> minSize=1450788736, minFactor=0.95, multiSize=725394368,
> multiFactor=0.5, singleSize=362697184, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false,
> cacheIndexesOnWrite=false, cacheBloomsOnWrite=false,
> cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
>     Exception in thread "main" java.lang.NullPointerException
>         at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
>         at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:98)
>         at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:79)
>         at
> org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:519)
>         at myhbase.MyHbase.main(MyHbase.java:23)
>
>
> I asked this question
> <
> http://stackoverflow.com/questions/36609428/trying-to-read-hfile-using-hfile-createreader-from-local-file-system
> >on
> Stackoverflow and seems like there are not that many hbase people there.
>
> Can anyone on this mailing-list help me on that?
>
> Best regards,
>
> Bin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message