hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1234) Change HBase StoreKey format
Date Sat, 11 Apr 2009 00:48:14 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-1234:
-------------------------

    Attachment: hbase-1234-v7.patch

v7 fixes most tests but also disables filter tests, compaction, thrift, and atomic increment
unit tests because these do not pass yet.

Here are notes on changes.

{code}
Disabled the filter tests.  Filter interface has changed.  Filters need
to be updated.  Will do in separate issue.

Disabled TestAtomicIncrement.  Will fix in separate issue.

Disabled TestCompaction and the thrift unit tests.

M b/src/java/org/apache/hadoop/hbase/HMerge.java
  Update to deal with results that are List<KeyValue>
M b/src/java/org/apache/hadoop/hbase/HRegionInfo.java
  (getComparator) Added.
M b/src/java/org/apache/hadoop/hbase/HStoreKey.java
   Deprecated.
M b/src/java/org/apache/hadoop/hbase/HTableDescriptor.java
  Changed map of families.  No longer do we do encoding of
  family name.  Instead we do a special comparator, one
  that compares only the family portion of the column
  name.
A b/src/java/org/apache/hadoop/hbase/KeyValue.java
  KeyValue does what HStoreKey used to.
M b/src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
  Fixed comments.
M b/src/java/org/apache/hadoop/hbase/client/MetaScanner.java
  Formatting.
M b/src/java/org/apache/hadoop/hbase/filter/ColumnValueFilter.java
M b/src/java/org/apache/hadoop/hbase/filter/PageRowFilter.java
M b/src/java/org/apache/hadoop/hbase/filter/PrefixRowFilter.java
M b/src/java/org/apache/hadoop/hbase/filter/RegExpRowFilter.java
M b/src/java/org/apache/hadoop/hbase/filter/RowFilterInterface.java
M b/src/java/org/apache/hadoop/hbase/filter/RowFilterSet.java
M b/src/java/org/apache/hadoop/hbase/filter/StopRowFilter.java
M b/src/java/org/apache/hadoop/hbase/filter/WhileMatchRowFilter.java
  Filter interface changed; now we take buffer with offset and
  length.  Deprecated the old.  Filters not finished.  Need work.
M b/src/java/org/apache/hadoop/hbase/io/Cell.java
  (createCells): Added. Method that converts List<KeyValue> to Cells.
  This method should be temporary... should go away eventually.
  (createSingleCellArray): Added.  Temporary method.
M b/src/java/org/apache/hadoop/hbase/io/HalfHFileReader.java
  Updated to deal with KeyValue keys.
M b/src/java/org/apache/hadoop/hbase/io/HbaseMapWritable.java
  Formatting.
M b/src/java/org/apache/hadoop/hbase/io/Reference.java
  Deal with KeyValue keys.
M b/src/java/org/apache/hadoop/hbase/io/RowResult.java
  (createRowResultArray, createRowResult): Temporary convertion
  methods to go from List<KeyValue> to RowResult.
M b/src/java/org/apache/hadoop/hbase/io/hfile/HFile.java
  Add method that takes a KeyValue.  Add methods that
  take buffer + offset + length (rather than just buffer).
M b/src/java/org/apache/hadoop/hbase/io/hfile/HFileScanner.java
  Add methods that take buffer + offset + length (rather than just buffer).
M b/src/java/org/apache/hadoop/hbase/master/BaseScanner.java
  Redo.  Deal with KeyValue.
M b/src/java/org/apache/hadoop/hbase/regionserver/HLog.java
  Deal with KeyValue.  Move the row component from Key to edit value.
M b/src/java/org/apache/hadoop/hbase/regionserver/HLogEdit.java
  Removed extaneous isDeleted stuff -- stuff that no longer applies
  now delete is attribute of key rather than special cell value.
  Add handling of KeyValue.
M b/src/java/org/apache/hadoop/hbase/regionserver/HLogKey.java
  HLK no longer has row in it.
M b/src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  Use skip list for Set of stores.  No need of sync when iterating.
  Use new comparators checking row and row+column matches.
  Add in new methods to check version count and to not add values
  Add in new methods to check version count and to not add values
  already present in results. Add handling of KeyValue.
  (getKeys): Removed.  Doesn't make sense now key and value
  bundled up together.
M b/src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  Use new comparators.  Deal with KeyValue.
M b/src/java/org/apache/hadoop/hbase/regionserver/InternalScanner.java
  Change interface to take KeyValue.
M b/src/java/org/apache/hadoop/hbase/regionserver/Memcache.java
  Deal in KeyValue.  Use SkipList so don't have to lock to iterate tailsets.
  Renamed internalGetFull as getFull.  Removed getKeys.  Use getFull
  instead.
M b/src/java/org/apache/hadoop/hbase/regionserver/Store.java
  Deal with KeyValue. Use new comparators.
  Use skip lists for reconstruction cache, etc.
  Make body of getFull and get agree with whats in Memcache -- call same
  methods. Remove getKeys. Do getFull instead (Did same thing).
  Remove methods like rowMatches and matching columns and use
  comparators instead.
M b/src/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
M b/src/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
M b/src/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  Removed HSK.  Use KV instead.
M b/src/java/org/apache/hadoop/hbase/regionserver/tableindexed/IndexedRegion.java
M b/src/java/org/apache/hadoop/hbase/regionserver/transactional/TransactionalHLogManager.java
M b/src/java/org/apache/hadoop/hbase/regionserver/transactional/TransactionalRegion.java
M b/src/java/org/apache/hadoop/hbase/regionserver/transactional/TransactionalRegionServer.java
  Use KeyValue instead.  Changes to agree with base class changes.
M b/src/java/org/apache/hadoop/hbase/util/Bytes.java
  Add license.  Some bug fixes and added methods (By Jon Gray), methods to
  do ByteBuffer stuff.
M b/src/java/org/apache/hadoop/hbase/util/Merge.java
M b/src/java/org/apache/hadoop/hbase/util/MetaUtils.java
  Update to KeyValue.
D b/src/java/org/apache/hadoop/hbase/util/TestBytes.java
  Removed.
M b/src/test/org/apache/hadoop/hbase/HBaseTestCase.java
  Updates to deal with KeyValue changes.
A b/src/test/org/apache/hadoop/hbase/TestKeyValue.java
  New test for KeyValue.

 

> Change HBase StoreKey format
> ----------------------------
>
>                 Key: HBASE-1234
>                 URL: https://issues.apache.org/jira/browse/HBASE-1234
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.20.0
>
>         Attachments: 1234-v2.patch, 1234-v5.patch, 1234.patch, hbase-1234-v6.patch, hbase-1234-v7.patch
>
>
> HBASE-859 cleaned up keys removing the need of HRegionInfo being in the context comparing
keys.  This issue is about changing the format.  Work done in HBASE-859 means changes have
been localized to HStoreKey, in particular to comparators and parse routines.  We should do
this now since 0.20.0 will require rewriting all data.
> Things to consider:
> <row> <columnfamily> <columnqualifier> <timestamp> <keytype>
> Or leave off columnfamily altogether and just write it once into the hfile metadata (All
key compares are done in the Store context so columnfamily can be safely left out of the equation;
its only when the key rises above Store that the columnfamily needs appending).
> keytype is probably a byte. Types are delete cell, delete row, delete family, delete
column?  What else?  Where should we put it?  At the end?  How should type sort?  Or should
it not be part of sort so its just the order at which we encounter the key?
> How are we going to support keys that go in out of chronological order?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message