hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: KFS jar
Date Mon, 21 Jun 2010 09:30:56 GMT
Stas Oskin wrote:
> Hi.
> I will point you at this presentation by my colleague Johannes Kirschnick,
>> Making Hadoop HA,
>> http://www.slideshare.net/steve_l/high-availability-hadoop
>> There's a performance graph on that slideset for small, virtualised
>> clusters, results in large physical clusters may vary
> Unfortunately they discuss HP IBRIX and not KFS.

I know, it just discusses why you'd want alternate filesystems. 
Parascale have the patches for their FS to work with Hadoop in Jira too.

You'd need to find someone using KFS in their production environment, 
and find out about things like server setup, disks/server, network 
topology etc to see whether it would work for you, or you could do the 
experiments yourself. The trouble is, experiments show things like 
performance and robustness to managed outages (let's turn this server 
off) and not the things you encounter in the field (a rack overheating, 
switches misbehaving, race conditions surfacing). Everyone is on their 
own there

>> just the snippet of my build which sets up the various classpath options
>> for different hadoop deployments. There's a lot of JARs there, but you can
>> strip the client-side down fairly well, even if Avro pushes out stuff you
>> don't need.
> Now it's clear, thanks.
> Regards.

View raw message