incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Femiano <bfemi...@gmail.com>
Subject Re: Giraph as Whirr service, see WHIRR-530
Date Thu, 05 Apr 2012 06:08:15 GMT
I've used it on clusters I started on EC2 launched by Whirr. Simply copy
the fat
jar to your client machine and it will distribute normally as a M/R
dependency.

It works very well.

The only limitation I could potentially find (without much proof) was on VMs
with limited IO the RPC message overhead between workers could be an issue.
I never tried it on VMs with less than 'High' IO, so take that with a grain
of salt.

On Thu, Apr 5, 2012 at 12:51 AM, Jakob Homan <jghoman@gmail.com> wrote:

> This is interesting.  Whirr can already spin up Hadoop MR clusters,
> which can then run the Giraph jobs.  Once Giraph is bootstrapped onto
> YARN, this will make more sense as a Whirr service.
>
> On Wed, Apr 4, 2012 at 9:43 PM, Avery Ching <aching@apache.org> wrote:
> > I don't use Whirr...I haven't heard it mentioned on this forum yet.
>  Anyone?
> >
> > Avery
> >
> >
> > On 4/4/12 9:30 PM, Paolo Castagna wrote:
> >>
> >> Hi,
> >> seen this?
> >>
> >>   WHIRR-530 - Add Giraph as a service
> >>   https://issues.apache.org/jira/browse/WHIRR-530
> >>
> >> This could be quite useful for users who want to give Giraph a spin on
> >> cloud
> >> infrastructure, just for testing or to run a few small experiments.
> >> My experience with Whirr an small 10-20 nodes clusters has be quite
> >> positive.
> >> Less so for larger clusters, but it more a problem/limit with the cloud
> >> provider rather than Whirr itself. I think.
> >>
> >> Whirr makes extremely easy and pleasant deploy stuff on-demand.
> >>
> >> ... and Whirr already supports YARN:
> >> https://issues.apache.org/jira/browse/WHIRR-391
> >>
> >> Is any Giraph developers/users here also a Whirr user?
> >>
> >> Paolo
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message