hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Agarwal <gau...@arkin.net>
Subject Large number of column qualifiers
Date Wed, 23 Sep 2015 17:18:38 GMT
Hi All,

I have Column Family with very large number of column qualifiers (>
50,000). Each column qualifier is 8 bytes long. The problem is the when I
do a scan operation to fetch some rows, the client side Cell object does
not have enough space allocated in it to hold all the columnQaulifiers for
a given row and hence I cannot read all the columns back for a given row.

Please see the code snippet that I am using:

 final ResultScanner rs = htable.getScanner(scan);
 for (Result row = rs.next(); row != null; row = rs.next()) {
    final Cell[] cells = row.rawCells();
    if (cells != null) {
        for (final Cell cell : cells) {
            final long c = Bytes.toLong(
                    *cell.getQualifierArray()*, cell.getQualifierOffset(),
cell.getQualifierLength());
            final long v = Bytes.toLong(cell.getValueArray(),
cell.getValueOffset());
            points.put(c, v);
        }
    }
}

The cell.getQualifierArray() method says that it's 'Max length is
Short.MAX_VALUE which is 32,767 bytes.'. Hence it can only hold around
4,000 columnQualfiers.

Is there an alternate API that I should be using or am I missing some
setting here? Note that in worst case I need to read all the
columnQualifiers in a row and I may or may not know a subset to fetch in
advance.

Even if this is not possible in a single call, is there a way to cursor
through the columnQualifiers?

I am presently using Hbase 0.96 client but can switch to Hbase 1.x if there
is an API in the newer version.

--cheers, gaurav

-- 
--cheers, gaurav

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message