hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thiago Vieira <tpbvie...@gmail.com>
Subject Re: building a department GPU cluster
Date Fri, 18 Jan 2013 03:37:52 GMT
I've seen some academic researches on this direction, with good results.
Some computations can be expressed by GPGPU, but it is still a restrict
number of cases. If is not easy to solve problems using MapReduce, solve
some problems with SIMD is harder.

Thiago Vieira

On Thu, Jan 17, 2013 at 9:24 PM, Russell Jurney <russell.jurney@gmail.com>wrote:

> Hadoop streaming can do this, and there's been some discussion in the
> past, but it's not a core use case. Check the list archives.
> Russell Jurney http://datasyndrome.com
> On Jan 17, 2013, at 9:25 AM, Jeremy Lewi <jeremy@lewi.us> wrote:
> I don't think running hadoop on a GPU cluster is a common use case; the
> types of workloads for a hadoop vs. gpu cluster are very different although
> a quick google search did turn up some. So this is probably not the best
> mailing list for your question.
> J
> On Thu, Jan 17, 2013 at 5:18 AM, Roberto Nunnari <roberto.nunnari@supsi.ch
> > wrote:
>> Roberto Nunnari wrote:
>>> Hi all.
>>> I'm writing to you to ask for advice or a hint to the right direction.
>>> In our department, more and more researchers ask us (IT administrators)
>>> to assemble (or to buy) GPGPU powered workstations to do parallel computing.
>>> As I already manage a small CPU cluster (resources managed using SGE),
>>> with my boss we talked about building a new GPU cluster. The problem is
>>> that I have no experience at all with GPU clusters.
>>> Apart from the already running GPU workstations, we already have some
>>> new HW that looks promising to me as a starting point for a GPU cluster.
>>> - 1x Dell PowerEdge R720
>>> - 1x Dell PowerEdge C410x
>>> - 1x NVIDIA M2090 PCIe x16
>>> - 1x NVIDIA iPASS Cable Kit
>>> (Dell forgot to include the iPASS adapter for the R720!! :-D)
>>> I'd be grateful if you could kindly give me some advice and/or hint to
>>> the right direction.
>>> In particular I'm interested on your opinion on:
>>> 1) is the above HW suitable for a small (2 to 4/6 GPUs) GPU cluster?
>>> 2) is apache adhoop suitable (or what could we use?) as a queuing and
>>> resource management system? We would like the cluster to be usable by many
>>> users at once in a way that no user has to worry about resources, just like
>>> we do on the CPU cluster with SGE.
>>> 3) What distribution of linux would be more appropriate?
>>> 4) necessary stack of sw? (cuda, hadoop, other?)
>>> Thank you very much for your valuable insight!
>>> Best regards.
>>> Robi
>> Anybody on this, please?
>> Robi

View raw message