hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "Hbase/ShellPlans" by udanax
Date Mon, 06 Aug 2007 01:53:19 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/Hbase/ShellPlans

The comment on the change is:
add a HQL wiki page

------------------------------------------------------------------------------
  
    -- Inchul, Feel free to add your opinion.
  
+ HBase Query Language (HQL) discussions and syntax draft page.
+ 
+  * http://www.hadoop.co.kr/wiki/moin.cgi/HBaseShell/HQL
+ 
  ----
  
  = Hbase Shell altools plans =
@@ -48, +52 @@

  Hbase > eixt;
  }}}
  
- Hbase altools is an Hbase Shell sub 'interpreter' (or 'shell)' program to provide scalable
data processing capabilities like 
- aggregation, algebraic calculation(groups and sets, commutative rings, algebraic geometry,
and linear algebra) on Hadoop + Hbase based parallel machines.
+ Hbase altools is an Hbase Shell sub 'interpreter' (or 'shell)' program to provide scalable
data processing capabilities like aggregation, algebraic calculation(groups and sets, commutative
rings, algebraic geometry, and linear algebra) on Hadoop + Hbase based parallel machines.
+ 
+ ''Altools Matrix operations will show how Google search's LSI, Google Earth's algebraic
topology, Google News' recommendation system are related to Bigtable.''
  
  = Hbase altools Goals =
   * A Simplified Import/Export/Migrate Functionality Between different data sources (Hadoop,
HBase)
@@ -98, +103 @@

  
  == Commands ==
  ||<bgcolor="#ececec">'''Command''' ||<bgcolor="#ececec">'''Explanation''' ||
+ ||Table ||'''Table''' command load from specified table. [[BR]][[BR]]~-''A = Table('movieLog_table');''-~
||
+ ||Matrix ||'''Matrix''' command control the configuration of the logic matrix. [[BR]][[BR]]~-''M
= Matrix(table_name, columnfamily_name[, scalar S]);''-~ ||
- ||Substitute || '''Substitute''' expression to [A~Z][[BR]][[BR]]~-''X = Matrix(table_name,
columnfamily_name);''-~||
+ ||Substitute || '''Substitute''' expression to [A~Z][[BR]][[BR]]~-''A = Table('movieLog_table');''-~
||
- ||Store ||'''STORE''' command will store results to specified table. [[BR]][[BR]]~-''A =
Table('movieLog_table'); [[BR]]B = A.Selection(length > 100); [[BR]]STORE B TO X run_style;''-~
||
+ ||Store ||'''Store''' command will store results to specified table. [[BR]][[BR]]~-''A =
Table('movieLog_table'); [[BR]]B = A.Selection(length > 100); [[BR]]Store B TO table('tmp_table')[or
file('backup.dat')];''-~ ||
- 
  == Relational Operators ==
- 
  ||<bgcolor="#ececec">'''Operator''' ||<bgcolor="#ececec">'''Explanation''' ||
- ||Projection ||<99%>'''Projection''' of a relation ~+R+~, It makes a new relation
as the set that is obtained when all tuples(rows) in ~+R+~ are restricted to the set {columnfamily,,1,,,...,columnfamily,,n,,}.[[BR]][[BR]]~-''A
= Table('movieLog_table');[[BR]]B = A.Projection('year','length');''-~||
+ ||Projection ||<99%>'''Projection''' of a relation ~+R+~, It makes a new relation
as the set that is obtained when all tuples(rows) in ~+R+~ are restricted to the set {columnfamily,,1,,,...,columnfamily,,n,,}.[[BR]][[BR]]~-''A
= Table('movieLog_table');[[BR]]B = A.Projection('year','length');''-~ ||
- ||Selection ||<99%>'''Selection''' of a relation ~+R+~, It makes a new relation as
the set of specified tuples(rows) of the relation ~+R+~[[BR]]'''Set Operations''' : ~-''OR,
AND, NOT''-~[[BR]][[BR]]~-''A = Table('movieLog_table');[[BR]]B = A.Selection(length >
100 AND studioName = 'Fox');''-~||
+ ||Selection ||<99%>'''Selection''' of a relation ~+R+~, It makes a new relation as
the set of specified tuples(rows) of the relation ~+R+~[[BR]]'''Set Operations''' : ~-''OR,
AND, NOT''-~[[BR]][[BR]]~-''A = Table('movieLog_table');[[BR]]B = A.Selection(length >
100 AND studioName = 'Fox');''-~ ||
- ||Group ||<99%>'''Group''' tuples by value of an attribute and apply aggregate function
independently to each group of tuples.[[BR]]'''Aggregate Functions''' : ~-''AVG( attribute
), SUM( attribute ), COUNT( attribute ), MIN( attribute ), MAX( attribute )''-~[[BR]][[BR]]~-''A
= Table('movieLog_table);[[BR]]B = A.Group('studioName', MIN('year'));''-~||
+ ||Group ||<99%>'''Group''' tuples by value of an attribute and apply aggregate function
independently to each group of tuples.[[BR]]'''Aggregate Functions''' : ~-''AVG( attribute
), SUM( attribute ), COUNT( attribute ), MIN( attribute ), MAX( attribute )''-~[[BR]][[BR]]~-''A
= Table('movieLog_table);[[BR]]B = A.Group('studioName', MIN('year'));''-~ ||
- ||Sort ||<99%>'''Sort''' of tuples(rows) of R, ordered according to columnfamilies
on columnfamily-list[[BR]][[BR]]~-''A = Table('movieLog_table');[[BR]]B = Sort by ('length');''-~||
+ ||Sort ||<99%>'''Sort''' of tuples(rows) of R, ordered according to columnfamilies
on columnfamily-list[[BR]][[BR]]~-''A = Table('movieLog_table');[[BR]]B = Sort by ('length');''-~
||
- 
- === Relational Operations Examples ===
- 
- ||Row Key ||<-12>Column Families ||
- ||<rowbgcolor="#ececec">title ||<-2> year ||<-2>length ||<-2>inColor
||<-2> studioName ||<-2> vote ||<-2> producer ||
- ||Star Wars ||year: || 1977 ||length: || 124 ||inColor: || true ||studioName: || Fox ||
vote:''user_1'' || 5 || producer: || George Lucas ||
- || || || || || || || || || || vote:''user_2'' || 2 || || ||
- ||Mighty Ducks ||year: || 1991 ||length: || 104 ||inColor: || true ||studioName: || Disney
|| vote:''user_1'' || 2 || producer: || Blair Peters ||
- || || || || || || || || || || vote:''user_3'' || 4 || || ||
- ||Wayne's World ||year: || 1992 ||length: || 95 ||inColor: || true ||studioName: || Paramount
|| vote:''user_2'' || 3 || producer: || Penelope Spheeris ||
- || || || || || || || || || || vote:''user_3'' || 4 || || ||
- 
- '''~+^π^+~'''~-title-~,~-year-~,~-length-~'''~+^(movieLog_table)^+~'''
- 
- A = table('movieLog_table');
- [[BR]]B = A.projection('year','length');
- 
- ||<rowbgcolor="#ececec">title ||year ||length ||
- ||Star Wars ||1977 ||124 ||
- ||Mighty Ducks ||1991 ||104 ||
- ||Wayne's World ||1992 ||95 ||
- 
- '''~+^σ^+~'''~-length>100-~'''~+^(movieLog_table)^+~'''
- 
- A = Table('movieLog_table');
- [[BR]]B = A.Selection(length > 100);
- 
- ||<rowbgcolor="#ececec">title ||year ||length ||inColor ||studioName ||producer ||
- ||Star Wars ||1977 ||124 ||true ||Fox ||12345 ||
- ||Mighty Ducks ||1991 ||104 ||true ||Disney ||67890 ||
- 
- 
- '''~+^π^+~'''~-title-~,~-year-~'''~+^(σ^+~'''~-length>100-~'''~+^(movieLog_table)∩σ^+~'''~-studioName='Fox'-~'''~+^(movieLog_table))^+~'''
- 
- A = Table('movieLog_table');
- [[BR]]B = A.Projection('year');
- [[BR]]C = B.Selection(length > 100 AND studioName = 'Fox');
- 
- ||<rowbgcolor="#ececec">title ||year ||
- ||Star Wars ||1977 ||
  
  ----
  == Matrix Operators ==
- 
- 
  ||<bgcolor="#ececec">'''Operator''' ||<bgcolor="#ececec">'''Explanation''' ||
- ||Addition ||<99%>... ||
- ||subtraction ||<99%>... ||
- ||multiplication ||<99%>... ||
+ ||Addition ||<99%>Adding entries with the same indices [[BR]][[BR]]~-''C = A + B;''-~
||
+ ||subtraction ||<99%>Subtracting entries with the same indices [[BR]][[BR]]~-''C =
A + B;''-~ ||
+ ||multiplication ||<99%>Product C of two matrices A and B [[BR]][[BR]]~-''C = A *
B;''-~ ||
  ||division ||<99%>... ||
  ||transpose ||<99%>... ||
  ||permutation ||<99%>... ||
  ||norms ||<99%>... ||
- 
  === Factorizations and decompositions ===
- 
  ||<bgcolor="#ececec">'''Function''' ||<bgcolor="#ececec">'''Explanation''' ||
  ||LU ||<99%>... ||
  ||QR ||<99%>... ||
@@ -174, +135 @@

  ||Condition ||<99%>... ||
  ||Determinant ||<99%>... ||
  ||Rank ||<99%>... ||
- 
  === Column-Wise Data Analysis ===
- 
  ||<bgcolor="#ececec">'''Function''' ||<bgcolor="#ececec">'''Explanation''' ||
  ||Frequencies ||<99%>... ||
  ||Sorting ||<99%>... ||
  ||Covariance ||<99%>... ||
  
+ 
+ = Examples =
+ 
+ == Relational Operations Examples ==
+ ||Row Key ||||||||||||||||||||||||Column Families ||
+ ||<rowbgcolor="#ececec">title |||| year ||||length ||||inColor |||| studioName ||||
vote |||| producer ||
+ ||Star Wars ||year: || 1977 ||length: || 124 ||inColor: || true ||studioName: || Fox ||
vote:''user_1'' || 5 || producer: || George Lucas ||
+ || || || || || || || || || || vote:''user_2'' || 2 || || ||
+ ||Mighty Ducks ||year: || 1991 ||length: || 104 ||inColor: || true ||studioName: || Disney
|| vote:''user_1'' || 2 || producer: || Blair Peters ||
+ || || || || || || || || || || vote:''user_3'' || 4 || || ||
+ ||Wayne's World ||year: || 1992 ||length: || 95 ||inColor: || true ||studioName: || Paramount
|| vote:''user_2'' || 3 || producer: || Penelope Spheeris ||
+ || || || || || || || || || || vote:''user_3'' || 4 || || ||
+ '''~+^π^+~'''~-title-~,~-year-~,~-length-~'''~+^(movieLog_table)^+~'''
+ 
+ A = table('movieLog_table'); [[BR]]B = A.projection('year','length');
+ ||<rowbgcolor="#ececec">title ||year ||length ||
+ ||Star Wars ||1977 ||124 ||
+ ||Mighty Ducks ||1991 ||104 ||
+ ||Wayne's World ||1992 ||95 ||
+ 
+ 
+ '''~+^σ^+~'''~-length>100-~'''~+^(movieLog_table)^+~'''
+ 
+ A = Table('movieLog_table'); [[BR]]B = A.Selection(length > 100);
+ ||<rowbgcolor="#ececec">title ||year ||length ||inColor ||studioName ||producer ||
+ ||Star Wars ||1977 ||124 ||true ||Fox ||12345 ||
+ ||Mighty Ducks ||1991 ||104 ||true ||Disney ||67890 ||
+ 
+ 
+ '''~+^π^+~'''~-title-~,~-year-~'''~+^(σ^+~'''~-length>100-~'''~+^(movieLog_table)∩σ^+~'''~-studioName='Fox'-~'''~+^(movieLog_table))^+~'''
+ 
+ A = Table('movieLog_table'); [[BR]]B = A.Projection('year'); [[BR]]C = B.Selection(length
> 100 AND studioName = 'Fox');
+ ||<rowbgcolor="#ececec">title ||year ||
+ ||Star Wars ||1977 ||
+ 

Mime
View raw message