hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steven zhuang <steven.zhuang.1...@gmail.com>
Subject what does HBase buffer when we do a scan?
Date Tue, 23 Mar 2010 04:53:50 GMT
hi, all,
         I am scanning on a table with 100+ million rows, each has a row key
about 50 bytes, and one column with less than 20 bytes data.
         I have set the scancaching to 100000 rows, and with 15 scanners
working the same time, I got OutOfMemory exception, my PC has 6GB ram,
through top I saw the program used more than 1GB ram sometime.
         In my calculation, each scanner will buffer 100000*50=5MB, and 15
scanners will buffer 75MB ram, I know each scanner will hold some space,
say, 10MB for each scanner.  my question is how come it uses more than 1GB
ram.
         anything I have missed?
         thanks.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message