hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "Hbase/HbaseShell" by udanax
Date Thu, 13 Sep 2007 04:25:56 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/Hbase/HbaseShell

The comment on the change is:
HQL updated

------------------------------------------------------------------------------
  = Hbase Shell Introduction =
  Hbase Shell is a basic, command-line, and interactive 'shell' for manipulating tables in
Hbase. 
  
+ == Evolution ==
-  * Hbase Shell was first added to TRUNK in July, 2007.
+  * Hbase Shell was first added to TRUNK in July, 2007. (eddie)
-   * '''Plans'''
-    * Hbase Query Language ([wiki:Hbase/HbaseShell/HQL HQL]) - SQL-like query language for
Hbase.
+   * ~-Addition of HQL (Hbase Query Language) support in Hbase Shell -- 12/Sep/07 (inchul)-~
+   * ~-[HADOOP-1789] Shell output formatting is currently in process.(izaak)-~
+  * Discussion and description of future operators.
-    * Hbase Shell [wiki:Hbase/ShellPlans Altools] - Algebraic Tools for the Large-scale analysis.
+   * ~-Hbase Shell [:Hbase/HbaseShell/Altools: Altools] - Algebraic Tools for the Large-scale
analysis. (eddie)-~ 
+   * ~-Hbase RDF - [:Hbase/RDF: HbaseRDF] for storing and querying RDF data. (eddie, inchul,
andrew)-~
+   * ~-[:Hbase/HbaseShell/Ideas: Some Ideas and Other Stuff] for Hadoop & Hbase &
Hbase Shell Project-~
-    * Hbase [wiki:Hbase/RDF RDF] - HbaseRDF for storing and querying RDF data.
- 
-  ''The feature of 3 branches appeared(HQL, Altools, RDF) recently in the Hbase Shell.[[BR]]Each
features should be assigned to specifc roles, and these roles should be clearly defined. --
[:udanax:Edward Yoon]''
  
  == People Involved ==
+ 
   * [:udanax:Edward Yoon] (R&D center, NHN corp.) -- Initial contributor
-  * [:InchulSong:Inchul Song] (Database Lab, KAIST)
+  * [:InchulSong:Inchul Song] (Database Lab, KAIST) 
   * Andrew Newman (University of Queensland's eResearch centre)
- 
+  
  ----
  = How to Start a Shell =
  Run the following on the command-line:
@@ -27, +29 @@

  
  You will be presented with the following prompt:
  
- {{{Hbase Shell, 0.0.1 version.
+ {{{Hbase Shell, 0.0.2 version.
  Copyright (c) 2007 by udanax, licensed to Apache Software Foundation.
  Type 'help;' for usage.
  
@@ -35, +37 @@

  
  All commands are terminated with a semi-colon: e.g. Type 'help;' to see list of available
commands.
  
+ == Hbase Shell Configuration ==
+ 
+ ||<bgcolor="#ececec">'''Property''' ||<bgcolor="#ececec">'''Description''' ||
+ ||hbaseshell.jline.bell.enabled ||<99%>If true, enable audible keyboard bells if an
alert is required.||
+ 
  = Hbase Shell Commands =
  '''Note''' that Data should be located by their row, column, and timestamp.
- 
+ == Hbase Shell Basic Commands ==
  ||<bgcolor="#ececec">'''Command''' ||<bgcolor="#ececec">'''Explanation''' ||
  ||Help ||<99%>'''Help''' command provides information about the use of shell script.[[BR]][[BR]]~-''HELP
[function_name];''-~ ||
  ||Show ||<99%>'''Show''' command lists tables.[[BR]][[BR]]~-''SHOW tables;''-~ ||
  ||Describe ||'''Describe''' command provides information about the columnfamilies in a table.[[BR]][[BR]]~-''DESC
table_name;''-~ ||
- ||FS ||<99%>Hadoop FsShell operations.[[BR]][[BR]]~-''FS -copyFromLocal /home/source_files.dat
/dfs/target_folder;''-~ ||
+ ||FS ||<99%>'''FS''' command to run hadoop filesystem commands.[[BR]][[BR]]~-''FS
-copyFromLocal /home/source_file.dat /fs/target_folder;''-~ ||
+ ||JAR ||<99%>'''JAR''' command to run hadoop jar commands.[[BR]][[BR]]~-''JAR ./build/hadoop-0.15.0-dev-examples.jar
pi 10 10;''-~||
  ||Clear ||<99%>'''Clear''' the screen.[[BR]][[BR]]~-''CLEAR;''-~ ||
  ||Exit ||<99%>'''Exit''' from the current shell script.[[BR]][[BR]]~-''EXIT;''-~ ||
  
- And, Commands to manually manipulate data on more detailed parts.
+ == Hbase Query Language (HQL) ==
+ 
+ HQL is an SQL-like query language for Hbase. You can use it to query and modify tables in
Hbase. 
+ [[BR]]'''Note''' that attribute values are quoted with either single or double quotes.
  
  ||<bgcolor="#ececec">'''Command''' ||<bgcolor="#ececec">'''Explanation''' ||
- ||Create ||'''Create''' command creates a new table.[[BR]][[BR]]~-''CREATE table_name[[BR]]COLUMNFAMILIES('columnfamily_name1'[,
'columnfamily_name2', ...])[[BR]][LIMIT=limitNumber_of_Version];''-~ ||
+ ||Create ||'''Create''' command creates a new table.[[BR]][[BR]]~-''CREATE TABLE table_name
([[BR]]{{{    }}}column_family_spec[[BR]]{{{    }}}[, column_family_spec][[BR]]{{{    }}}...[[BR]])[[BR]][[BR]]''-~'''colum_family_spec:'''~-''[[BR]]column_family_name[[BR]]{{{
 }}}[MAX_VERSIONS=n][[BR]]{{{    }}}[MAX_LENGTH=n][[BR]]{{{    }}}[COMPRESSION=NONE|BLOCK|RECORD][[BR]]{{{
   }}}[IN_MEMORY][[BR]]{{{    }}}[BLOOMFILTER=NONE|BLOOM|COUNTING|RETOUCHED VECTOR_SIZE=n
NUM_HASH=n]''-~ ||
+ ||Alter ||'''Alter''' command changes the structure of the specified table.[[BR]][[BR]]~-''ALTER
TABLE table_name[[BR]]{{{    }}}alter_spec [, alter_spec] ...[[BR]][[BR]]''-~'''alter_spec:'''~-''[[BR]]{{{
    }}}ADD column_family_spec[[BR]]{{{    }}}| ADD (column_family_spec, ...)[[BR]]{{{    }}}|
DROP column_family_name[[BR]]{{{    }}}| CHANGE column_family_name column_family_spec''-~||
- ||Drop ||'''Drop''' command drops columnfamilies in a table or tables.[[BR]][[BR]]~-''DROP
table_name1[, table_name2, ...] or columnfamily_name1[, columnfamily_name2, ...];''-~ ||
+ ||Drop ||'''Drop''' command drops columnfamilies in a table or tables.[[BR]][[BR]]~-''DROP
table_name1[, table_name2, ...];''-~ ||
- ||Insert ||<99%>'''Insert''' command inserts one row into the table with a value for
specified column in the table.[[BR]][[BR]]~-''INSERT table_name ('columnfamily_name1:column_key'[,
'columnfamily_name2:column_key', ...])[[BR]] VALUESVALUES ('entry1'[, 'entry2', ...])[[BR]]WHERE
row='row_key';''-~ ||
- ||Delete ||'''Delete''' command deletes specified rows in table. [[BR]][[BR]]~-''DELETE
table_name[[BR]]WHERE row='row_key'[[BR]][AND column='columnfamily_name:column_key'];''-~
||
- ||Select ||<99%>'''Select''' command retrieves rows from a table.[[BR]][[BR]]~-''SELECT
table_name[[BR]][WHERE row='row_key'][[BR]][AND column='columnfamily_name:column_key'];[[BR]][AND
time='Specified_Timestamp'];[[BR]][LIMIT=Number_of_Version];''-~ ||
+ ||Insert ||<99%>'''Insert''' command inserts one row into the table with a value for
specified column in the table.[[BR]][[BR]]~-''INSERT INTO table_name (colmn_name, ...)[[BR]]{{{
   }}}VALUES ('value', ...)[[BR]]{{{    }}}WHERE row = 'row-key'[[BR]]{{{    }}}[TIMESTAMP
'timestamp'];''-~ ||
+ ||Delete ||'''Delete''' command deletes specified rows in table. [[BR]][[BR]]~-''DELETE
{ column_name, [, column_name] ... | *}[[BR]]{{{    }}}FROM table_name[[BR]]{{{    }}}WHERE
row = 'row-key';''-~ ||
+ ||Select ||<99%>'''Select''' command retrieves rows from a table.[[BR]][[BR]]~-''SELECT
{ column_name [, column_name] ... | * }[[BR]]{{{    }}}FROM table_name[[BR]]{{{    }}}[WHERE
row = 'row-key' | STARTING FROM 'row-key'][[BR]]{{{    }}}[NUM_VERSIONS = version_count][[BR]]{{{
   }}}[TIMESTAMP 'timestamp'][[BR]]{{{    }}}[LIMIT = row_count][[BR]]{{{    }}}[INTO FILE
'file_name'][[BR]][[BR]]''-~'''column_name:'''~-''[[BR]]{{{     }}}column_family_name[[BR]]{{{
   }}}| column_family_name:column_label_name''-~ ||
  
  ----
- = Example Of Hbase Shell Use =
+ = Example =
+ == Example Of HQL Use ==
- == Create the table in a HBase ==
+ === Create the table in a Hbase ===
  
  {{{
+ HBase > help create;
+ CREATE    Create tables
- Hbase > CREATE movieLog_table
-     --> COLUMNFAMILIES('year', 'length', 'inColor', 'studioName', 'vote', 'producer')
-     --> LIMIT=1; 
  
- Hbase > CREATE movieStar_table
-     --> COLUMNFAMILIES('biography', 'filmography', 'gender', 'birthDate')
-     --> LIMIT=1;
+ Syntax:
+ CREATE TABLE table_name
+         column_family_spec [, column_family_spec] ...
+ 
+ column_family_spec:
+         column_family_name
+         [MAX_VERSIONS=n]
+         [MAX_LENGTH=n]
+         [COMPRESSION=NONE|RECORD|BLOCK]
+         [IN_MEMORY]
+         [BLOOMFILTER=NONE|BLOOM|COUNTING|RETOUCHED VECTOR_SIZE=n NUM_HASH=n]
  }}}
  
- == Insert data into a table ==
+ '''CREATE TABLE''' enables you to create a new table with various options for each column
family.
+ 
+  * ~-'''MAX_VERSIONS''' makes a table keep only the recent n versions in a cell. Its default
value is 3, i.e., only the most recent 3 versions of values are stored in a cell.-~
+  * ~-'''MAX_LENGTH''' specifies the maximum size of a column value. By default, the maximum
size is unlimited. It is limited only by Integer.MAX_VALUE.-~
+  * ~-'''COMPRESSION''' specifies which compression technique to use for the column family.
By default, Hbase does not compress any data.-~
+  * ~-'''IN_MEMORY''' specifies whether to keep the values in the column family in-memory,
or not. By default, values are not kept in-memory.-~
+  * ~-'''BLOOMFILTER''' specifies which bloom filter to use for the column family. By default,
none of the bloom filters is used. You can specify the options in two ways: with '''VECTOR_SIZE'''
and '''NUM_HASH''', or with '''NUM_ENTRIES'''. '''VECTOR_SIZE''' specifies the number of elements
in the vector, and '''NUM_HASH''' specifies the number of hash functions to use; With '''NUM_ENTRIES''',
you  specify only the approximated number of entries in the column, and '''VECTOR_SIZE'''
and '''NUM_HASH''' are automatically determined.-~
+ 
  {{{
+ HBase > CREATE Movie (
+     --> year, length, inColor, studioName, vote, producer)
+     --> NUM_VERSIONS 10; 
+ 
+ HBase > create table webtable (
+     --> contents in_memory max_versions=10 compression=block,
+     --> anchor max_length=256 bloomfilter=counting                                  
+     --> vector_size=1000000 num_hash=4);
+ }}}
+ 
+ === Select data from a table ===
+ {{{
+ Hbase > help select;
+ SELECT    Select values from tables
+ 
+ Syntax:
+ SELECT { column_name, [, column_name] ... | *} FROM table_name
+         [WHERE row='row_key' | STARTING FROM 'row-key']
+         [NUM_VERSIONS = version_count]
+         [TIMESTAMP 'timestamp']
+         [LIMIT = row_count]
+         [INTO FILE 'file_name']
+ }}}
+ 
+ '''SELECT''' retrieves a subset of data from the specified table.
+ 
+  * ~-A column_name with spaces in it should be single-quoted.-~
+  * ~-'''STARTING FROM''' returns all the rows starting from 'row-key'.-~
+  * ~-'''NUM_VERSIONS''' retrieves only the recent n versions of values in a cell.-~
+  * ~-'''TIMESTAMP''' returns only the values with the specified timestamp.-~
+  * ~-'''LIMIT''' limits the number of rows to be returned.-~ 
+  * ~-'''INTO FILE''' outputs the returned rows into the specified file. For '''SELECT INTO'''
another table, see '''INSERT INTO SELECT''' below.-~
+ 
+ {{{
+ Hbase > select studioName: from movieLog_table where row = 'Star Wars';
+ +------+----------------------+----------------------+----------------------+
+ | No.  | Row                  | Column               | Cell                 |
+ +------+----------------------+----------------------+----------------------+
+ | 1    | Star Wars            | studioName:          | Fox                  |
+ +------+----------------------+----------------------+----------------------+
+ 
+ Successfully print out the selected data.(0.05 sec)
+ 
+ Hbase > select 'studioName:YoungGu Art' from movieLog_table where row = 'D-War';
+ }}}
+ 
+ === Insert data into a table ===
+ {{{
+ Hbase > help insert;
+ INSERT    Insert values into tables
+ 
+ Syntax:
+ INSERT INTO table_name
+         (colmn_name, ...) VALUES ('value', ...)
+         WHERE row='row_key'
+ 
+ column_name:
+           column_family_name
+         | column_family_name:column_label_name
+ }}}
+ 
+ '''INSERT''' inserts a set of values into a table. 
+ 
+  * ~-If the specified column already exists, the new value is stored as a new version.-~
+  * ~-If '''TIMESTAMP''' is not specified for the value, the current time is used as its
timestamp.-~
+ 
+ {{{
- Hbase > INSERT movieLog_table ('year:', 'length:', 'inColor:', 'studioName:', 'vote:user_1',
'producer:')
+ Hbase > INSERT INTO movieLog_table (year:, length:, inColor:, studioName:, 'vote:user
name', producer:)
      --> VALUES ('1977', '124', 'true', 'Fox', '5', 'George Lucas')
      --> WHERE row='Star Wars';
- 
- 
- Hbase > INSERT movieStar_table ('biography:', 'filmography:Star Wars', 'gender:', 'birthDate:')
-     --> VALUES ('blah blah', 'starring', 'male', 'March 31, 1971')
-     --> WHERE row='Ewan Gordon Mc.Gregor'; 
  }}}
  
- == Show all data in a table ==
- {{{
- Hbase > SELECT movieLog_table;
- }}}
- 
- ||Row Key ||<-12>Column Families ||
- ||<rowbgcolor="#ececec">title ||<-2> year ||<-2>length ||<-2>inColor
||<-2> studioName ||<-2> vote ||<-2> producer ||
- ||Star Wars ||year: || 1977 ||length: || 124 ||inColor: || true ||studioName: || Fox ||
vote:''user_1'' || 5 || producer: || George Lucas ||
- || || || || || || || || || || vote:''user_2'' || 2 || || ||
- ||Mighty Ducks ||year: || 1991 ||length: || 104 ||inColor: || true ||studioName: || Disney
|| vote:''user_1'' || 2 || producer: || Blair Peters ||
- || || || || || || || || || || vote:''user_3'' || 4 || || ||
- ||Wayne's World ||year: || 1992 ||length: || 95 ||inColor: || true ||studioName: || Paramount
|| vote:''user_2'' || 3 || producer: || Penelope Spheeris ||
- || || || || || || || || || || vote:''user_3'' || 4 || || ||
- 
- {{{
- Hbase > SELECT movieStar_table;
- }}}
- 
- ||Row Key ||<-8>Column Families ||
- ||<rowbgcolor="#ececec">starName ||<-2> biography ||<-2>filmography ||<-2>gender
||<-2> birthDate ||
- ||Ewan Gordon Mc.Gregor ||biography: ||blah blah ||filmography:Star Wars ||starring ||gender:
||male ||birthDate: ||March 31, 1971 ||
- || || || ||filmography:Emma ||extra || || || || ||
- ||Kenan Thompson ||biography: ||blah blah ||filmography:Mighty Ducks ||starring ||gender:
||male ||birthDate: ||May 10, 1978 ||
- || || || ||filmography:Big Fat Liar  ||cameo || || || || ||
- ||keanu reeves ||biography: ||blah blah ||filmography:Constantine ||starring ||gender: ||male
||birthDate: ||September 2, 1964||
- || || || ||filmography:The Matrix Reloaded ||starring || || || || ||
- 

Mime
View raw message