hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From edward yoon <webmas...@udanax.org>
Subject RE: Relational Algrebra Operators on hbase shell.
Date Tue, 06 Nov 2007 07:45:01 GMT

Hi, community.
There was a misunderstanding in my last writing, please ignore it. Thanks.
------------------------------
B. Regards,
Edward yoon @ NHN, corp. 
Home : http://www.udanax.org> From: webmaster@udanax.org> To: hadoop-dev@lucene.apache.org>
Subject: Relational Algrebra Operators on hbase shell.> Date: Mon, 5 Nov 2007 23:14:01
+0000> > > > One last thing, hbase shell (relational) operators running mapreduce
jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating
work. Would it make sense doing further operators as PIG user defined functions? (I suppose
we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).> >
I don't think so.> It's just a relational algebra basics.> No these works, no SQL-like
HQL, RDF, Matrix works.> What is an overlapping character?> I need the exact explain.>
> Also, If Pig has time to discuss matrix operations or linear systems, > Can i said
"No, it's a hbase shell territory." ??> > I feel stuffy becuase i'm just going round
in circles discussing the plans of shell operator on wiki. > (almost for the past two months)
> > I bet my name on shell works and Other members,too.> At least, I have rights
to make it.> > Any advice are welcome.> > Edward J. Yoon,> Thanks.> ------------------------------>
B. Regards,> Edward yoon @ NHN, corp.> Home : http://www.udanax.org> > > >
Date: Mon, 5 Nov 2007 14:04:51 -0800> > From: jira@apache.org> > To: hadoop-dev@lucene.apache.org>
> Subject: [jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra Operators>
>> >> > [ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540287
]> >> > stack commented on HADOOP-1608:> > ------------------------------->
>> > Edward:> >> > I do not understand what you mean by the following
in your 02/Nov/07 06:09 PM comment: "I'll implement relational algebra operators to the tentative
language and HQL by Sub-Tasks List sequence."> >> > In org.apache.hadoop.hbase.shell.algebra,
test for output table presence and creation if missing is duplicated code in Selection, Projection,
DuplicateTable, etc. I tried to move this duplicated code back up into the RelationalOperation
class as a utility but noticed then if table already exists, we don't call initJob because
we return early (See end of the getConf in DuplicateTable for example). Is running one of
these operators a second time, after table has been created, a problem? Have you tried it?>
>> > I did not have a mapreduce cluster running and so the last line hung for ever
(You should add to help need for a mapreduce cluster).> > {code}> > Hbase>
aaa = table('y');> > Hbase> aaa;> > Syntax error : Type 'help;' for usage.Message
: Encountered ";" at line 1, column 5.> > Hbase> show aaa;> > Missing parameters.
Please check 'Show' syntax> > Hbase> bbb = group aaa by ('x'); Hbase> save bbb
into table ('a');> > {code}> >> > Note, it would be nice if you entered
a variable name only if you got a description of the variable content or perhaps doing 'show
VARIABLE_NAME' output some kind of description.> >> > I then started up a cluster
and did the simplest of operations:> > {code}> > Hbase> aaa = table('x');>
> Hbase> save aaa into table(aaaaaa);> > 07/11/05 21:26:57 WARN mapred.JobClient:
No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).Job
job_200711052125_0001 is still running........Job failed.> > {code}> >> >
Job failed because of 'Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapred.TableSplit'.
How do you run on a cluster Edward? Do you copy the hbase jar into the hadoop lib dir all
over the cluster? Does HADOOP-1622 help here?> >> > On job failure, should the
table be removed? Currently, I have a new table 'aaaaaa' with nothing in it.> >>
> The output of Group, Selection, etc. is saved to an 'output' table?> >> >
It would be good if we could get into the help listing somewhere the operators and types of
conditions allowed on selection (>, > > One last thing, hbase shell (relational)
operators running mapreduce jobs begins to impinge on PIG territory. We should be careful
and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user
defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store,
is done. Smile).> >> > Otherwise, patch looks good Edward.> >> >>
>> >> >> >> [Hbase Shell] Relational Algrebra Operators> >>
-------------------------------------------> >>> >> Key: HADOOP-1608>
>> URL: https://issues.apache.org/jira/browse/HADOOP-1608> >> Project: Hadoop>
>> Issue Type: Improvement> >> Components: contrib/hbase> >> Affects
Versions: 0.14.1> >> Environment: All environments> >> Reporter: Edward
Yoon> >> Priority: Minor> >> Fix For: 0.16.0> >>> >> Attachments:
patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt,
patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt,
patch_v13.txt, patch_v14.txt, patch_v15.txt> >>> >>> >> Development
of relational algebra operators has begun.> >> * Projection> >> ** selects
a subset of the columnfamilies of a relation> >> ** Result = π ~column_list~ (Relation)>
>> {code}> >> Hbase> Result = Relation.Projection('year','legnth');>
>> Hbase> save Result into table('result');> >> {code}> >> * Selection>
>> ** See : HADOOP-2003 issue's description> >> {code}> >> Hbase>
Result = Relation.Selection(length> 100 and studioName = 'Fox');> >> Hbase>
save Result into table('result');> >> {code}> >> * Group> >> **
more details about 'GROUP' operation will be handled in HADOOP-1658 issue.> >> *
θ Join> >> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B
~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B
~2~,...,B ~m~) that satisfy the join condition> >> {code}> >> Hbase>
R1 = table('movieLog_table');> >> Hbase> R2 = table('personInfo_table');> >>
Hbase> Result = R1.join(R1.producer: = R2.ROW) and R2;> >> or Result = R1.join(R1.actor:hero
= R2.Row) and R2;> >> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName
= 'Fox' and R2.occupation = 'singer') and R2;> >> {code}> >> > -->
> This message is automatically generated by JIRA.> > -> > You can reply to
this email to add a comment to the issue online.> >> > _________________________________________________________________>
Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare!> http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
_________________________________________________________________
Peek-a-boo FREE Tricks & Treats for You!
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message