hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Loddengaard <a...@cloudera.com>
Subject Re: hadoop hardware suggestion
Date Tue, 09 Jun 2009 21:13:24 GMT
100 nodes is certainly overkill for 500MBs of data, but if you have the
resources, you might as well use them I suppose (assuming you're already
paying for power, network, cooling, etc.).

As for your idea of virtualization, it makes sense.  I don't know of anyone
running a Hadoop cluster on Windows.  That said, it might not be impossible,
but it'll be hard to get help from the list, as most people run Hadoop on
Linux.  For this reason, virtualization is probably a good idea, though it
will slow your cluster down.  Running in a virtualized world is generally
slower than a native world.

Hope this helps.


On Tue, Jun 9, 2009 at 9:23 AM, PORTO aLET <portoalet@gmail.com> wrote:

> Hi,
> I am trying to setup a hadoop cluster to process our apache log (about
> 500MB
> a day).
> I am just not sure what kind of pc configuration I should use?
> We have a few windows xp machines (about 100+, too many to process 'just'
> 500MB ?) that I am thinking of using sparingly (during the night) to
> process
> the logs.
> So I am thinking of installing Linux in VirtualBox in them to run hadoop?
> Please share your knowledge and wisdom,
> Regards.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message