hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/Shell" by DavidButtler
Date Wed, 06 Jan 2010 01:24:08 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hbase/Shell" page has been changed by DavidButtler.
http://wiki.apache.org/hadoop/Hbase/Shell?action=diff&rev1=10&rev2=11

--------------------------------------------------

- This page describes the JRuby IRB-based HBase Shell.  It replaces the SQL-like [[Hbase/HbaseShell|
HQL]], the Shell found in HBase versions 0.1.x and previous.  Some discussion of new shell
requirements can be found in the [[Hbase/Shell/Replacement| Shell Replacement]] document.
+ This page describes the JRuby IRB-based HBase Shell.  It replaces the SQL-like [[Hbase/HbaseShell|HQL]],
the Shell found in HBase versions 0.1.x and previous.  Some discussion of new shell requirements
can be found in the [[Hbase/Shell/Replacement|Shell Replacement]] document.
  
  To run the shell, do
  
  {{{
  $ ${HBASE_HOME}/bin/hbase shell
  }}}
- 
  You'll be presented with a prompt like the following:
  
  {{{
@@ -15, +14 @@

  Version: 0.2.0-dev, r670701, Mon Jun 23 17:26:36 PDT 2008
  hbase(main):001:0>
  }}}
- 
  Type 'help' followed by a return to get a listing of commands.
  
  == Commands ==
@@ -25, +23 @@

   alter     Alter column family schema;  pass table name and a dictionary
             specifying new column family schema. Dictionaries are described
             below in the GENERAL NOTES section.  Dictionary must include name
-            of column family to alter.  For example, 
+            of column family to alter.  For example,
-            
+ 
             To change or add the 'f1' column family in table 't1' from defaults
             to instead keep a maximum of 5 cell VERSIONS, do:
             hbase> alter 't1', {NAME => 'f1', VERSIONS => 5}
-            
+ 
             To delete the 'f1' column family in table 't1', do:
             hbase> alter 't1', {NAME => 'f1', METHOD => 'delete'}
  
+            You can also change table-scope attributes like MAX_FILESIZE
+            MEMSTORE_FLUSHSIZE and READONLY.
+ 
+            For example, to change the max size of a family to 128MB, do:
+            hbase> alter 't1', {METHOD => 'table_att', MAX_FILESIZE => '134217728'}
+ 
-   count    Count the number of rows in a table. This operation may take a LONG
+  count     Count the number of rows in a table. This operation may take a LONG
             time (Run '$HADOOP_HOME/bin/hadoop jar hbase.jar rowcount' to run a
             counting mapreduce job). Current count is shown every 1000 rows by
             default. Count interval may be optionally specified. Examples:
-            
+ 
             hbase> count 't1'
             hbase> count 't1', 100000
  
@@ -60, +64 @@

             timestamp coordinates.  Deletes must match the deleted cell's
             coordinates exactly.  When scanning, a delete cell suppresses older
             versions. Takes arguments like the 'put' command described below
-  
+ 
-  deleteall Delete all cells in a given row; pass a table name, row, and optionally 
+  deleteall Delete all cells in a given row; pass a table name, row, and optionally
             a column and timestamp
  
   disable   Disable the named table: e.g. "hbase> disable 't1'"
-  
+ 
-  drop      Drop the named table. Table must first be disabled
+  drop      Drop the named table. Table must first be disabled. If table has
+            more than one region, run a major compaction on .META.:
+ 
+            hbase> major_compact ".META."
  
   enable    Enable the named table
  
@@ -92, +99 @@

  
             hbase> put 't1', 'r1', 'c1', 'value', ts1
  
+  tools     Listing of hbase surgery tools
+ 
-  scan      Scan a table; pass table name and optionally an array of column
+  scan      Scan a table; pass table name and optionally a dictionary of scanner
+            specifications.  Scanner specifications may include one or more of
+            the following: LIMIT, STARTROW, STOPROW, TIMESTAMP, or COLUMNS.  If
+            no columns are specified, all columns will be scanned.  To scan all
+            members of a column family, leave the qualifier empty as in
-            names OR an array of column names AND a dictionary of scanner 
-            specifications.  If you wish to include scanner specifications, 
-            you must also include an array of columns.  Scanner specifications 
-            may include one or more of the following: LIMIT, STARTROW, STOPROW,
-            or TIMESTAMP.  To scan all members of a column family, leave the 
-            qualifier empty as in 'col_family:'.  Examples:
+            'col_family:'.  Examples:
-            
+ 
             hbase> scan '.META.'
-            hbase> scan '.META.', ['info:regioninfo']
+            hbase> scan '.META.', {COLUMNS => 'info:regioninfo'}
-            hbase> scan 't1', ['c1', 'c2'], {LIMIT => 10, STARTROW => 'xyz'}
+            hbase> scan 't1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, \
-            
+              STARTROW => 'xyz'}
+ 
+            For experts, there is an additional option -- CACHE_BLOCKS -- which
+            switches block caching for the scanner on (true) or off (false).  By
+            default it is enabled.  Examples:
+ 
+            hbase> scan 't1', {COLUMNS => ['c1', 'c2'], CACHE_BLOCKS => false}
+ 
+  status    Show cluster status. Can be 'summary', 'simple', or 'detailed'. The
+            default is 'summary'. Examples:
+ 
+            hbase> status
+            hbase> status 'simple'
+            hbase> status 'summary'
+            hbase> status 'detailed'
+ 
+  shutdown  Shut down the cluster.
+ 
+  truncate  Disables, drops and recreates the specified table.
+ 
   version   Output this HBase version
  
  GENERAL NOTES:
@@ -119, +146 @@

  NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
  'Object.constants' to see a (messy) list of all constants in the environment.
  
+ In case you are using binary keys or values and need to enter them into the
+ shell then use double-quotes to make use of hexadecimal or octal notations,
+ for example:
+ 
+   hbase> get 't1', "key\x03\x3f\xcd"
+   hbase> get 't1', "key\003\023\011"
+   hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"
+ 
+ Using the double-quote notation you can directly use the values output by the
+ shell for example during a "scan" call.
+ 
  This HBase shell is the JRuby IRB with the above HBase-specific commands added.
  For more on the HBase Shell, see http://wiki.apache.org/hadoop/Hbase/Shell
  }}}
- 
  === General Notes ===
+ GENERAL NOTES: Quote all names in the hbase shell such as table and column names.  Don't
forget commas delimit command parameters.  Type <RETURN> after entering a command to
run it.  Dictionaries of configuration used in the creation and alteration of tables are ruby
Hashes. They look like this:
- GENERAL NOTES:
- Quote all names in the hbase shell such as table and column names.  Don't
- forget commas delimit command parameters.  Type <RETURN> after entering a
- command to run it.  Dictionaries of configuration used in the creation and
- alteration of tables are ruby Hashes. They look like this:
  
-   {'key1' => 'value1', 'key2' => 'value2', ...}
+  . {'key1' => 'value1', 'key2' => 'value2', ...}
  
+ They are opened and closed with curley-braces.  Key/values are delimited by the '=>'
character combination.  Usually keys are predefined constants such as NAME, VERSIONS, COMPRESSION,
etc.  Constants do not need to be quoted.  Type 'Object.constants' to see a (messy) list of
all constants in the environment.
- They are opened and closed with curley-braces.  Key/values are delimited by
- the '=>' character combination.  Usually keys are predefined constants such as
- NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
- 'Object.constants' to see a (messy) list of all constants in the environment.
  
  This HBase shell is the JRuby IRB with the above HBase-specific commands added.
  
  == Command-line Options ==
- 
  A few command-line arguments can be passed the shell.  Pass ''--help'' to see whats available:
e.g. ''durruti:~ stack$ ./bin/hbase shell --help''.  For example, its possible to point the
shell at a cluster other than the one that is set in the ''hbase-site.xml'' using the ''--master=MASTER''
option.
  
  == Anatomy Lesson ==
  The HBase Shell is a ruby script, ''${HBASE_HOME}/bin/hirb.rb'', that is passed to the JRuby
class ''org.jruby.Main'' (See ''${HBASE_HOME}/bin/shell'' and search for org.jruby.Main to
see how its done).
  
- The ''hirb.rb' script defines a set of hbase methods (get, set, scan, etc.), sets some environment
variables, imports a few hbase-particular modules -- a results Formatter and a ruby wrapper
around the ''HBaseAdmin'' and ''HTable'' java classes that can be found at ''${HBASE_HOME}/bin/HBase.rb''
-- and then starts up a subclass of IRB, named HIRB (IRB is the name of the interactive Ruby
interpreter; for an untainted irb experience, type 'irb' on a system that has ruby installed).
 The subclassing is done to alter slightly some of the usual IRB behaviors.  For instance,
every command invocation returns a result and the result is printed to the terminal even if
the result is nil.  The subclass intercepts nil emissions and just suppresses them.
+ The ''hirb.rb' script defines a set of hbase methods (get, set, scan, etc.), sets some environment
variables, imports a few hbase-particular modules -- a results Formatter and a ruby wrapper
around the ''HBaseAdmin'' and ''HTable'' java classes that can be found at ''${HBASE_HOME}/bin/HBase.rb''
-- and then starts up a subclass of IRB, named HIRB (IRB is the name of the interactive Ruby
interpreter; for an untainted irb experience, type 'irb' on a system that has ruby installed).
 The subclassing is done to alter slightly some of the usual IRB behaviors.  For instance,
every command invocation returns a result and the result is printed to the terminal even if
the result is nil.  The subclass intercepts nil emissions and just suppresses them. ''
  
- Anything you can do in irb, or at least in its JRuby equivalent, ''jirb'', you should be
able to do in HBase Shell.
+ ''Anything you can do in irb, or at least in its JRuby equivalent, ''jirb'', you should
be able to do in HBase Shell. ''
  
  == Scripting ==
- 
- You can pass scripts to the HBase Shell by doing the following:
+ ''You can pass scripts to the HBase Shell by doing the following: ''
  
- {{{durruti:~ stack$ ${HBASE_HOME}/bin/hbase shell PATH_TO_SCRIPT}}}
+ ''{{{durruti:~ stack$ ${HBASE_HOME}/bin/hbase shell PATH_TO_SCRIPT}}} ''
  
- Your script can lean on the methods provided by the HBase Shell.
+ ''Your script can lean on the methods provided by the HBase Shell. ''
  
- For more control, you may prefer manipulating HTable and HBaseAdmin methods directly.  For
example, if you would do your own formatting or you are storing structures that are mangled
when manipulated in HBase Shell.  In this case, you might pass your scripts directly to JRuby
by doing:
+ ''For more control, you may prefer manipulating HTable and HBaseAdmin methods directly.
 For example, if you would do your own formatting or you are storing structures that are mangled
when manipulated in HBase Shell.  In this case, you might pass your scripts directly to JRuby
by doing: ''
  
- {{{durruti:~ stack$ ${HBASE_HOME}/bin/hbase org.jruby.Main PATH_TO_SCRIPT}}}
+ ''{{{durruti:~ stack$ ${HBASE_HOME}/bin/hbase org.jruby.Main PATH_TO_SCRIPT}}} ''
  
- For examples writing straight ruby manhandling hbase client instances, see the scripts on
this page: [[Hbase/JRuby]] (Ignore the sections that have you fetching the jruby jar and the
prefacing your scripts with the '#!' bang magic).
+ ''For examples writing straight ruby manhandling hbase client instances, see the scripts
on this page: [[Hbase/JRuby]] (Ignore the sections that have you fetching the jruby jar and
the prefacing your scripts with the '#!' bang magic). ''
  
  == Useful Tricks ==
- 
  === irbrc ===
- Create an ''.irbrc'' file for yourself in your home directory.  Add HBase Shell customizations.
 A useful one is command history:
+ ''Create an ''.irbrc'' file for yourself in your home directory.  Add HBase Shell customizations.
 A useful one is command history: ''
  
+ ''{{{durruti:~ stack$ more .irbrc require 'irb/ext/save-history' IRB.conf[:SAVE_HISTORY]
= 100 IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history"}}} ''
- {{{durruti:~ stack$ more .irbrc
- require 'irb/ext/save-history'
- IRB.conf[:SAVE_HISTORY] = 100
- IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history"}}}
  
  === Log date to timestamp ===
+ ''To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do: {{{hbase(main):021:0>
import java.text.SimpleDateFormat hbase(main):022:0> import java.text.ParsePosition hbase(main):023:0>
SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29", ParsePosition.new(0)).getTime()
=> 1218920189000}}} ''
- To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do:
- {{{hbase(main):021:0> import java.text.SimpleDateFormat
- hbase(main):022:0> import java.text.ParsePosition
- hbase(main):023:0> SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29",
ParsePosition.new(0)).getTime()
- => 1218920189000}}}
  
+ ''To go the other direction, do: {{{hbase(main):021:0> import java.util.Date hbase(main):022:0>
Date.new(1218920189000).toString() => "Sat Aug 16 20:56:29 UTC 2008"}}} ''
- To go the other direction, do:
- {{{hbase(main):021:0> import java.util.Date
- hbase(main):022:0> Date.new(1218920189000).toString()
- => "Sat Aug 16 20:56:29 UTC 2008"}}}
  
- To output in a format that is exactly like hbase log format is a pain messing with SimpleDateFormat.
+ ''To output in a format that is exactly like hbase log format is a pain messing with SimpleDateFormat.
''
  

Mime
View raw message