hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Scan output to file on each regserver node?
Date Wed, 20 Aug 2014 02:39:55 GMT
A coprocessor is certainly possible. You haven't shared your motivation,
only a specific implementation, so I cannot assist further.


On Tue, Aug 19, 2014 at 6:28 PM, Demai Ni <nidmgg@gmail.com> wrote:

> Nick,
>
> Thanks for the quick responds, I will definitely look into the Hadoop
> streaming.
>
> What do you think about AggregationClient? It is carried out at
> region/region server level, maybe instead do a count/min/avg, a method can
> be used to write the data out to local file system?
>
> Demai on the run
>
> On Aug 19, 2014, at 5:04 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > This sounds an awful lot like a map-only MR job... With Hadoop Streaming,
> > you should be able to achieve your goal of piping to an arbitrary
> process.
> >
> >
> > On Tue, Aug 19, 2014 at 4:26 PM, Demai Ni <nidmgg@gmail.com> wrote:
> >
> >> Dear experts ,
> >>
> >> I understand that I can do a simple command like:
> >>
> >> echo "scan 'table1'"| hbase she'll > myoutput
> >>
> >> This scenario i am thinking is to:
> >> 1) output to local file system(like Linux ) instead of hdfs
> >> 2) each regserver only output its only data to it's node's file system
> >>
> >> To elaborate the 2) a bit. Basically, this will be like export Hbase
> data
> >> to local file system without going through network. And on each node,
> one
> >> file will be created.
> >>
> >> Is there a way to achieve it? Actually the receiving side of 1) doesn't
> >> have to be a file system , it can be another process to process the
> data.
> >> But let's use file system to simplify the scenario for now.
> >>
> >> Thanks
> >>
> >> Demai on the run
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message