hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Maier <wcma...@hep.wisc.edu>
Subject Re: Any other uses of hdfs?
Date Sun, 17 Jul 2011 05:46:55 GMT
Hi Owen-

On Sat, Jul 16, 2011 at 09:14:59AM -0700, Owen O'Malley wrote:
> The scientists at CERN use HDFS for storing their large data sets and
> don't use MapReduce at all. (I believe most of their applications are
> MPI.) At Yahoo, some users run non-MapReduce in the shared MapReduce
> cluster.

Various groups supporting LHC physics at CERN do have about a dozen
petabyte-sized HDFS clusters. Our workflows are highly parallel, so we mostly
use traditional schedulers like Condor[0], PBS and LSF.


[0] http://www.cs.wisc.edu/condor


Will Maier - UW High Energy Physics
cel: 608.438.6162
tel: 608.263.9692
web: http://www.hep.wisc.edu/~wcmaier/

View raw message