hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "Hbase/ShellPlans" by udanax
Date Tue, 14 Aug 2007 03:55:37 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/Hbase/ShellPlans

------------------------------------------------------------------------------
  
  = Hbase Shell Plan Draft =
  Plan is to significantly expand the set of shell operators.  Basic data manipulation and
data definition operators will be extended and evolved to be more SQL-like ([wiki:Hbase/HbaseShell/HQL
HQL]).  More sophisticated manipulations to do relational and linear algebra, matrix additions,
multiplications, etc., will be added to a HBase subshell to keep the two operator types --
SQL-like vs. non-SQL -- distinct.
- 
- This project is currently in the planning stage.  [https://issues.apache.org/jira/browse/HADOOP-1608
HADOOP-1608] to add "Relational Algrebra Operators" is currently in process.
  
  == Suggested Hbase Query Language plans ==
  
@@ -15, +13 @@

  ----
  
  == Suggested Hbase Shell altools plans ==
- I suggest to develop HBase Shell in SQL-style, and develop '''al'''gebraic '''tools''' as
a sub shell as described below. 
+ I suggest to develop HBase Shell in SQL-style, and develop '''al'''gebraic '''tools''' as
a sub shell in Intuitionalized-style as described below. 
  
  {{{
  HBase > altools;
@@ -46, +44 @@

  ==== Commands ====
  ||<bgcolor="#E5E5E5">'''Command''' ||<bgcolor="#E5E5E5">'''Explanation''' ||
  ||Table ||<99%>'''Table''' command loads specified table. [[BR]][[BR]]~-''A = Table('movieLog_table');''-~
||
- ||Matrix ||<99%>'''Matrix''' command constructs the configuration of the logic matrix.
[[BR]][[BR]]~-''M = Matrix(table_name, columnfamily_name[, option]);''-~ ||
+ ||Matrix ||<99%>'''Matrix''' command constructs the configuration of the logic matrix.[[BR]]'''Options'''
: features not yet. [[BR]][[BR]]~-''M = Matrix(table_name, columnfamily_name[, scalar S]);''-~
||
  ||Substitute ||<99%>'''Substitute''' expression to [A~Z][[BR]][[BR]]~-''A = Table('movieLog_table');''-~
||
  ||IF...ELSE ||<99%>'''IF...ELSE''', Imposes conditions on the execution. [[BR]][[BR]]~-''IF
( boolean_expression )[[BR]]B = command_statements;[[BR]]ELSE[[BR]]B = command_statements;''-~||
  ||Store ||<99%>'''Store''' command will store results to specified table. [[BR]][[BR]]~-''A
= Table('movieLog_table'); [[BR]]B = A.Selection(length > 100); [[BR]]Store B TO table('tmp_table')[or
file('backup.dat')];''-~ ||
@@ -103, +101 @@

  ||<bgcolor="#E5E5E5">'''Function''' ||<bgcolor="#E5E5E5">'''Explanation''' ||
  ||LU ||<99%>'''LU Decomposition'''[[BR]]A procedure for decomposing an N by N matrix
A into a product of a lower triangular matrix L and an upper triangular matrix U, LU = A.[[BR]]'''Functions'''
: ~-''getL(), getU(), isSingular(), getPivot()''-~ [[BR]][[BR]]~-''A = Matrix('m_table','cf_1');[[BR]]B
= LUDecomposition(A);[[BR]]C = getU(B);[[BR]]D = getL(A);''-~||
  ||QR ||<99%>'''QR Decomposition'''[[BR]]For an m-by-n matrix A with m >= n, the
QR decomposition is an m-by-n orthogonal matrix Q and an n-by-n upper triangular matrix R
so that A = Q*R.[[BR]]'''Functions''' : ~-''getH(), getQ(), getR()''-~[[BR]][[BR]]~-''A =
Matrix('m_table','cf_1');[[BR]]B = QRDecomposition(A);[[BR]]C = getH(B);''-~||
- ||Cholesky ||<99%>'''Cholesky Decomposition'''[[BR]]It is a special case of LU decomposition
applicable only if matrix to be decomposed is symmetric positive definite.[[BR]]'''Functions'''
: ~-''getL(), isSPD()''-~ [[BR]][[BR]]~-''A = Matrix('m_table','cf_1');[[BR]]B = CholeskyDecomposition(A);[[BR]]C
= getU(B);[[BR]]D = getL(A);''-~||
+ ||Cholesky ||<99%>'''Cholesky Decomposition'''[[BR]]It is a special case of LU decomposition
applicable only if matrix to be decomposed is symmetric positive definite.[[BR]]'''Functions'''
: ~-''getL(), isSPD()''-~ [[BR]][[BR]]~-''A = Matrix('m_table','cf_1');[[BR]]B = CholeskyDecomposition(A);[[BR]]C
= getL(A);''-~||
  ||SVD ||<99%>'''SV(Singular Value) Decomposition'''[[BR]]For an m-by-n matrix A with
m >= n, the singular value decomposition is an m-by-n orthogonal matrix U, an n-by-n diagonal
matrix S, and an n-by-n orthogonal matrix V so that A = U*S*V'.[[BR]]'''Functions''' : ~-''getS(),
getU(), getV(), getSingularValues()''-~ [[BR]][[BR]]~-''A = Matrix('m_table','cf_1');[[BR]]B
= SVDecomposition(A);[[BR]]C = getU(B);''-~||
  
- ''~-Again, to help readers, you might cite pages that explain 'relational algebra' or examples
of its use in databases to help contextualize your plan (Aren't there other relational operators
than these that might be included?   Do you intend to implement those?  If not, you might
say why not of if you intend to do these as 'Matrix Arithmetic Operators, you might say so.
-- St.Ack-~''
+ '''(ex. 1)''' To find the Singular Value decomposition in Altools, do the following:
+ [[BR]]~-''M = UΣV*''-~
+ 
+ {{{
+ Hbase.altools > M = Matrix('m_table','cf_1');
+ Hbase.altools > U = M.getU();
+ Hbase.altools > V = M.getV();
+ }}}
+ 
  ----
  = Implementation =
- 
- ''~-I'd suggest you add a section on the framework you intend to use developing operators
(Will you be using javacc to parse the shell inputs?  Will some operators run mapreduce jobs?
 Will output always be ASCII or will it specifiable?). -- St.Ack-~''
  
  '''Note''' : ''We should first test on local machines. -- udanax''
  [[BR]] ''Code Style Formatter'' [attachment:uncle-jim-code-style.xml]
@@ -132, +136 @@

  St.Ack
  }}}
  
- 
+ This project is currently in the planning stage.  [https://issues.apache.org/jira/browse/HADOOP-1608
HADOOP-1608] to add "Relational Algrebra Operators" is currently in process.
  
  ----
  = Example Of Hbase Shell Use =

Mime
View raw message