hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bogdan M. Maryniuk" <bogdan.maryn...@gmail.com>
Subject Re: What OS?
Date Fri, 14 Aug 2009 03:58:53 GMT
On Fri, Aug 14, 2009 at 12:27 PM, Jason Venner<jason.hadoop@gmail.com> wrote:
> Anyone have any performance numbers for Solaris or ZFS based datanodes.
>
> The directory and inode cache sizes are a limiting factor for linux for
> large and busy datanodes.

Uhmm... I do run it on zoned OpenSolaris, but I don't have a real
numbers, since you have to measure it yourself on the same hardware.

Actually, Phoronix.com (Warning: Biased Linux fanboys!) has a general
performance tests and they usually claim that Linux is mostly as twice
as faster at everything. However, I never saw such slow ZFS as they
show on their benchmarks as well as other factors are sometimes
ridiculously slow (some of them are true).

That's is really interesting to measure it on a two identical clusters
and see how well it works all together (I/O, memory, Networking etc).
But that's needed to kill lots of time for that, to make such
measurements properly, otherwise you will go definitely wrong
conclusions. However, building two identical clusters just for test —
lilbit boring. :-) And I seriously won't go Linux anyway due to a big
number of other reasons, even if someone proves OpenSolaris bit
slower.

At least what I can tell you right away: ZFS is a killer all aspects
to any FS Linux has at the moment (including bloody alpha BTRFS that
suffers due to weak for higher loads software RAID layer that is in a
Linux kernel) and Java runs faster on Solaris. Also make sure you
tuned TCP/IP stack, which is by default too conservative.

If you could try to measure it — we would really appreciate that!

-- 
Kind regards, BM

Things, that are stupid at the beginning, rarely ends up wisely.

Mime
View raw message