Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Lucenehadoop Wiki" for change notification.
The following page has been changed by udanax:
http://wiki.apache.org/lucenehadoop/Hbase/ShellPlans


= Hbase Shell Plan Draft =
 Plan is to significantly expand the set of shell operators. Basic data manipulation and data definition operators will be extended and evolved to be more SQLlike ([:Hbase/HbaseShell/HQL]). More sophisticated manipulations to do relational and linear algebra, matrix additions, multiplications, etc., will be added to a HBase subshell to keep the two operator types  SQLlike vs. nonSQL  distinct.
+ Plan is to significantly expand the set of shell operators. Basic data manipulation and data definition operators will be extended and evolved to be more SQLlike ([wiki:Hbase/HbaseShell/HQL HQL]). More sophisticated manipulations to do relational and linear algebra, matrix additions, multiplications, etc., will be added to a HBase subshell to keep the two operator types  SQLlike vs. nonSQL  distinct.
This project is currently in the planning stage. [https://issues.apache.org/jira/browse/HADOOP1608 HADOOP1608] to add "Relational Algrebra Operators" is currently in process.
@@ 21, +21 @@
I've made some changes to your initial HQL to make it look more like SQL. I borrowed the syntax definition style from MySQL.
 [:Hbase/HbaseShell/HQL] by Inchul Song

 ~''If you're ready to implement them, I suggest you to open a new issue for "HQL"  Edward''~

@@ 55, +53 @@
* A Simplified Parallel Numerical Analysis by abstracting/numericalizing points, lines, or plane data across multiple maps in HBase.
''~Does the import/export above include being able to write HQL/altool scripts feeding them to the interpreter on stdin or passing the interpreter a file of script? It would be sweet too if the interpreter could be invoked with a flag which stated how results were to be output. ACSII tables could be the default as it is now but users will likely want output without formatting or output formatted as XML, etc. Something to think about. Also, Edward, I'd suggest that you would be doing yourself a service if you added citations for concepts like 'Parallel Numerical Analysis'. It will help folks like myself does not know what this means. Thanks.  St.Ack ~''
 ''~One other thing Edward. What about JOINs? How or where do you foresee these being done? Running a mapreduce job that read from two tables and wrote a third might make for a simple start.  St.Ack ~''
+
=== HBase altools Background ===
I expect Hadoop + Hbase to handle sparsity and data explosion very well in near future. Moreover, i believe the design of the multidimensional map structure and the 3d space model of the data are optimized for rapid adhoc information retrieval in any orientation, as well as for fast, flexible calculation and transformation of raw data based on formulaic relationships. It is advantageous with respect to '''Analysis Processing''' as it allows users to easily formulate complex queries, and filter or slice data into meaningful subsets, among other things.
@@ 86, +84 @@
Table <99%>'''Table''' command loads specified table. [[BR]][[BR]]~''A = Table('movieLog_table');''~ 
Matrix <99%>'''Matrix''' command constructs the configuration of the logic matrix. [[BR]][[BR]]~''M = Matrix(table_name, columnfamily_name[, scalar S]);''~ 
Substitute <99%>'''Substitute''' expression to [A~Z][[BR]][[BR]]~''A = Table('movieLog_table');''~ 
+ IF...ELSE <99%>'''IF...ELSE''', Imposes conditions on the execution. [[BR]][[BR]]~''IF ( boolean_expression )[[BR]]B = command_statements;[[BR]]ELSE[[BR]]B = command_statements;''~
Store <99%>'''Store''' command will store results to specified table. [[BR]][[BR]]~''A = Table('movieLog_table'); [[BR]]B = A.Selection(length > 100); [[BR]]Store B TO table('tmp_table')[or file('backup.dat')];''~ 
 GnuPlot <99%> Runs '''GnuPlot''' and creates an .pdf file.[[BR]]This Command for creating evaluation plots with gnuplot, a common Unix/Linux plotting tool. it is not tested under Windows.)[[BR]]'''Option functions''' : ~''setMinMaxX(), setMinMaxY(), setColor(), addSource() ''~[[BR]][[BR]]~''fill plotting example''~

 ''~Do you intend to allow multicharacter variables? I think you should if possible.  St.Ack~''
==== Relational Operators ====
'''Operator''' '''Explanation''' 
 Projection <99%>'''Projection''' of a relation ~+R+~, It makes a new relation as the set that is obtained when all tuples(rows) in ~+R+~ are restricted to the set {columnfamily,,1,,,...,columnfamily,,n,,}.[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = A.Projection('year','length'); //π,,year.length,,(A)''~ 
+ Projection <99%>'''Projection''' of a relation ~+R+~, It makes a new relation as the set that is obtained when all tuples(rows) in ~+R+~ are restricted to the set {columnfamily,,1,,,...,columnfamily,,n,,}.[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = A.Projection('year','length'); '''//π,,year.length,,(A)''' ''~ 
 Selection <99%>'''Selection''' of a relation ~+R+~, It makes a new relation as the set of specified tuples(rows) of the relation ~+R+~[[BR]]'''Set Operations''' : ~''OR, AND, NOT''~[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = A.Selection(length > 100 AND studioName = 'Fox'); //σ,,length > 100.studioName='Fox',,(A)''~ 
+ Selection <99%>'''Selection''' of a relation ~+R+~, It makes a new relation as the set of specified tuples(rows) of the relation ~+R+~[[BR]]'''Set Operations''' : ~''OR, AND, NOT''~[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = A.Selection(length > 100 AND studioName = 'Fox'); '''//σ,,length > 100.studioName='Fox',,(A)''' ''~ 
 JOINs <99%>Table '''JOIN''' operations, linking and extracting data from two different internal source[[BR]]'''Operations''' : ~''naturalJoin(), thetaJoin(), cartesianProduct() ''~ [[BR]][[BR]]~''R = Table('movieLog_table');[[BR]]S = Table('movieStar_table');[[BR]]C = R.naturalJoin(S); //C = R▷◁S''~ 
+ JOINs <99%>Table '''JOIN''' operations, linking and extracting data from two different internal source[[BR]]'''Operations''' : ~''naturalJoin(), thetaJoin(), cartesianProduct() ''~ [[BR]][[BR]]~''R = Table('movieLog_table');[[BR]]S = Table('movieStar_table');[[BR]]C = R.naturalJoin(S); '''//C = R▷◁S''' ''~ 
 Group <99%>'''Group''' tuples by value of an attribute and apply aggregate function independently to each group of tuples.[[BR]]'''Aggregate Functions''' : ~''AVG( attribute ), SUM( attribute ), COUNT( attribute ), MIN( attribute ), MAX( attribute )''~[[BR]][[BR]]~''A = Table('movieLog_table);[[BR]]B = A.Group('studioName', MIN('year')); //γ,,studioName.MIN( arrtibute ),,(A)''~ 
+ Group <99%>'''Group''' tuples by value of an attribute and apply aggregate function independently to each group of tuples.[[BR]]'''Aggregate Functions''' : ~''AVG( attribute ), SUM( attribute ), COUNT( attribute ), MIN( attribute ), MAX( attribute )''~[[BR]][[BR]]~''A = Table('movieLog_table);[[BR]]B = A.Group('studioName', MIN('year')); '''//γ,,studioName.MIN( year ),,(A)''' ''~ 
 Sort <99%>'''Sort''' of tuples(rows) of R, ordered according to columnfamilies on columnfamilylist[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = Sort A by ('length'); //τ,,length,,(A)''~ 
+ Sort <99%>'''Sort''' of tuples(rows) of R, ordered according to columnfamilies on columnfamilylist[[BR]][[BR]]~''A = Table('movieLog_table');[[BR]]B = Sort A by ('length'); '''//τ,,length,,(A)''' ''~ 
''~Again, to help readers, you might cite pages that explain 'relational algebra' or examples of its use in databases to help contextualize your plan (Aren't there other relational operators than these that might be included? Do you intend to implement those? If not, you might say why not of if you intend to do these as 'Matrix Arithmetic Operators, you might say so.  St.Ack~''
==== Matrix Arithmetic Operators ====
'''Operator''' '''Explanation''' 
 Addition <99%>'''Adding''' entries with the same indices [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A + B;''~ 
+ Addition <99%>'''Adding''' entries with the same indices [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A + B; '''// c,,ij,, = a,,ij,, + b,,ij,, (i : row key, j : column key)''' ''~ 
 Subtraction <99%>'''Subtracting''' entries with the same indices [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A + B;''~ 
+ Subtraction <99%>'''Subtracting''' entries with the same indices [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A  B; '''// c,,ij,, = a,,ij,,  b,,ij,, (i : row key, j : column key)''' ''~ 
 Multiplication <99%>'''Multiplication''' of two matrices, Product C of two matrices A and B [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A * B;''~ 
+ Multiplication <99%>'''Multiplication''' of two matrices, Product C of two matrices A and B [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A * B; '''//C = A · B''' ''~ 
 Division <99%>'''Division''' is solving the matrix equation AX = B for X [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A /[or \] B;''~
+ Division <99%>'''Division''' is solving the matrix equation AX = B for X [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Matrix('m_table','cf_2');[[BR]]C = A /[or \] B; '''// C = A / B''' ''~
 Transpose <99%>'''Transpose''' of a Matrix, A matrix which is formed by turning all the rows of a given matrix into columns and viceversa.[[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Transpose(A);''~
+ Transpose <99%>'''Transpose''' of a Matrix, A matrix which is formed by turning all the rows of a given matrix into columns and viceversa.[[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = Transpose(A); '''// B = A'''' ''~
==== Factorizations and Decompositions ====

 * Boolean Condition Command?
'''Function''' '''Explanation''' 
LU <99%>'''LU Decomposition'''[[BR]]A procedure for decomposing an N by N matrix A into a product of a lower triangular matrix L and an upper triangular matrix U, LU = A[[BR]]'''Functions''' : ~''getL(), getU(), isSingular(), getPivot()''~ [[BR]][[BR]]~''A = Matrix('m_table','cf_1');[[BR]]B = LUDecomposition(A);[[BR]]C = getU(B);[[BR]]D = getL(A);''~
@@ 148, +142 @@

= Example Of Hbase Shell Use =
 See the HBase Shell Usage Page [:Hbase/HbaseShell/Examples]
+ See the HBase Shell Usage Page. [:Hbase/HbaseShell/Examples]