incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: Blur computation colocation?
Date Fri, 02 Jan 2015 18:13:28 GMT
Ruchir,

Take a look at:

https://incubator.apache.org/blur/docs/0.2.4/getting-started.html

The compile section is likely where you want to start.  Also that page has
broken download links for 0.2.4 because we haven't released it yet.  So you
will need to clone and compile it yourself.

and

http://incubator.apache.org/blur/get_involved.html

Something to note, if you are already running hadoop somewhere and you want
Blur to use that install for storage.  You will need to compile Blur for
the given Hadoop distro.

For Hadoop 1:

mvn install -DskipTests -Dhadoop1

The default hadoop version is 1.2.1, use -Dhadoop.version=<desired version>
to change.

For Hadoop 2 MR1:

mvn install -DskipTests -Dhadoop2-mr1

The default hadoop version is 2.0.0-mr1-cdh4.6.0, use
-Dhadoop.version=<desired version> to change.

For Hadoop 2:

mvn install -DskipTests -Dhadoop2

The default hadoop version is 2.2.0, use -Dhadoop.version=<desired version>
to change.

Let me know if you have any questions.

Aaron



On Thu, Jan 1, 2015 at 8:55 PM, Ruchir Choudhry <ruchirchoudhry23@gmail.com>
wrote:

> Hello  Aaron,
>
>                     Will appreciate if you can pass me a quick dev and
> setup document for Blur, I want to start working on it.
>
> Thanks,
> Ruchir
>
>
>
> On Thu, Jan 1, 2015 at 9:30 AM, Aaron McCurry <amccurry@gmail.com> wrote:
>
> > On Wed, Dec 31, 2014 at 5:27 PM, Mark Kerzner <mark@elephantscale.com>
> > wrote:
> > >
> > > Hi,
> > >
> > > To put my question in a context, here is an article about Hadoop/ES
> > >  integration, http://hortonworks.com/partner/elasticsearch/. It talks
> > > about
> > > the colocation capability
> > > <
> > >
> >
> http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/arch.html
> > > >:
> > > ship my code to the ES node, and do a number of look-up on the ES node
> > > locally. However, that is supposedely done only for mapreduce, and one
> > > would have to use ES API for non-mapreduce tasks, such as Storm nodes.
> > >
> > > Can local computation be solved with Blur? That is, can I discover
> where
> > > the shards are, and send my computation to that server?
> > >
> >
> > The short answer is yes.  Take a look at the documentation.
> >
> > https://incubator.apache.org/blur/docs/0.2.4/platform.html
> >
> > It's a work in progress, but should give you an idea of thoughts behind
> the
> > blur platform feature.
> >
> > Also take a look at
> > https://github.com/apache/incubator-blur/tree/master/blur-command for
> the
> > current commands and how to run them and test them.  Thanks!
> >
> > Aaron
> >
> >
> >
> > >
> > > Thank you,
> > > Mark
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message