hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "Hbase/HbaseArchitecture" by JimKellerman
Date Tue, 13 Feb 2007 20:59:58 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by JimKellerman:
http://wiki.apache.org/lucene-hadoop/Hbase/HbaseArchitecture

------------------------------------------------------------------------------
    * [#ipcclientmaster Client to Master Messages]
    * [#ipcclienttablet Client to Tablet Server Messages]
    * [#ipcmastertablet Master to Tablet Server Messages]
+   * [#ipctabletmaster Tablet Server to Master Messages]
   * [#clientlib Client Library]
    * [#api API]
   * [#other Other]
@@ -300, +301 @@

     * When a tablet is "brand new", all reads can be satisfied from the memtable.
     * When the memtable gets too large ''(how large?)'', the memtable is written to a new
SSTable. This is a ''minor'' compaction. 
  
-   ''Since this is not a tablet split, this implies that information about the new SSTable
is not written to the METADATA table. Is this correct? Maybe not because a minor compaction
does write a new redo point into the METADATA table and the redo point could contain a pointer
to the new SSTable.''
+   ''Minor and merging compactions write a new redo point into the METADATA table. The redo
point contains information about the new SSTables.''
  
   * Reads
    * Checks that the request is well-formed
@@ -331, +332 @@

    * Block Cache: caches blocks read from the SSTables
   * Bloom Filter
    * Optional in-memory structure that reduces disk access by
-  * API
- {{{
-    LoadTablet()
- }}}
  
  [[Anchor(sstable)]]
  = SSTable =
@@ -378, +375 @@

    * Root tablet stores the location of all tablets in a METADATA table
    * The "root tablet" is just the first tablet in the METADATA table, it is never split
  
-   ''I really don't like this definition (and I know it is from the Bigtable paper). Aside
from the fact that it is never split, the root tablet is special in another way: it is the
metadata table for the METADATA table.'' ''It '''does''' however have the same format as the
rest of the METADATA table.''
+    ''I really don't like this definition (and I know it is from the Bigtable paper). Aside
from the fact that it is never split, the root tablet is special in another way: it is the
metadata table for the METADATA table.'' ''It '''does''' however have the same format as the
rest of the METADATA table.''
  
   * The row key is a combination of the table name, column or map name and the __'''first
row key'''__ of the tablet.
  
@@ -388, +385 @@

     1. ''If the first key in a tablet is stored instead, the minimum value could be represented
as the empty string "" and a simple string comparison between a key and the first key would
result in key > first key.'' If you represented the maximum row key as the empty string,
that would require a special case instead of just a simple compare.''
  
     1. ''If the keys go from 1 to n, when the tablet is split, the first tablet still has
the row key "" as its first key and the second tablet has a first row key of n/2. So if a
key is presented and its value is < n/2, you know that if it exists, it is in the first
tablet and if the value >= n/2 it is in the second tablet.''
+ 
+   ''If it turns out that there is a good reason that the last row key should be stored instead
of the first row key, then this can be changed in the future.''
  
   * The "location" map has the ''InMemory'' tuning parameter set
   * Each row stores approximately 1KB of data in memory
@@ -456, +455 @@

  [[Anchor(ipcclienttablet)]]
  == Client to TabletServer Messages ==
  
+  * read (row-key, column-name, [column-key (for maps)], [timestamp]) returns value
+   * for maps, if no column-key is specified, returns the entire map for the specified row
key
+   * ''Question: Is there a need to split up large return values into multiple messages?''
+   * The table name is implied since this message is sent directly to the tablet server for
the specified row-key
+ 
+  * scan (starting-row-key, column-name, [starting-timestamp]) returns a tablet block size
of data for the specified column and starting point.
+ 
+  * write (row-key, column-name, [column-key (for maps)], timestamp, value)
+ 
+  * batchUpdate (row-key, List<parameters>, timestamp)
+   * batchUpdates can only be used when updating multiple values in a map for a specific
row key or if the tablets for all the specified columns are all served by the same tablet
server.
+   * the parameter list consists of column name - value pairs (for columns) or column-name
- column-key - value triples (for maps)
+ 
+  * incrementCounter (row-key, column-name, timestamp, increment)
+ 
  [[Anchor(ipcmastertablet)]]
  == Master to Tablet Server Messages ==
+ 
+  * ''Note: the master uses the client API to read METADATA tablets, so no special messages
are required for this activity.''
+ 
+  * "Are you ok?"
+   * response (indicates tablet server alive)
+    * if the tablet server is serving any tablets, the response will also include a list
of tablets being served.
+ 
+  * start serving tablets (list-of-tablets-to-serve)
+   * response indicates tablets being served and any that it failed to start serving
+ 
+  * major compaction (tablet)
+   * response indicates compaction complete
+ 
+  * stop serving tablets (list-of-tablets-to-stop-serving)
+   * response indicates that tablets are no longer being served
+ 
+  * shut down
+   * response indicates that shutdown was successful
+ 
+ [[Anchor(ipctabletmaster)]]
+ == Tablet Server to Master Messages ==
+ 
+  * tablet split (old-tablet, new-tablet-1, new-tablet-2)
  
  [[Anchor(clientlib)]]
  = Client Library =

Mime
View raw message