Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of jason.rutherglen@gmail.com
 designates 209.85.216.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=Ki6t2L/i/cbMH6MKy3fz+JMn5wLuhR57C8fxrMYrXxZuVISbaOPaNKAF6E0qDuIrgW
         URRGClZ9f32Q84RlN3X57pG4YBTcA20tHO7cd42h08Cp1rOJgwIhsK2PicNVmAPlnzCZ
         BW9Xt8yZYJIouuGQ7QlkOayEjfgpgvlL8Mk6c=
MIME-Version: 1.0
In-Reply-To: <AANLkTimDGaLwy2cTmcPKftyfjx8arCghP9OeUw3+ArXd@mail.gmail.com>
References: <937582.63759.qm@web130123.mail.mud.yahoo.com>
	<AANLkTikcNpNdYrvqVkR-ZihY_oT2CTM-5p6qjBuUgQQj@mail.gmail.com>
	<AANLkTimMXkPMB=N9wH-JjNCH=w8aZncN9dwWG+j4wW8E@mail.gmail.com>
	<AANLkTimRfVNiiERXS8AR+3=AeBfd7NX7+u1+pEzgFAZs@mail.gmail.com>
	<AANLkTinkfvBcVgd7dd7FN15T39Ahmy4=b0=SvbpmRM2M@mail.gmail.com>
	<AANLkTim788UjjKLGfj6W3-o7SnewQ0RB8SUzV48btq-v@mail.gmail.com>
	<AANLkTiknU_BfzCEgcspYDjHPVcJLDbTAX=-neJFMydoN@mail.gmail.com>
	<AANLkTimDGaLwy2cTmcPKftyfjx8arCghP9OeUw3+ArXd@mail.gmail.com>
Date: Wed, 16 Feb 2011 18:35:43 -0800
Message-ID: <AANLkTimdJUj=NZCxor34QCbVuwh9ADpUZtn6_9y12W-G@mail.gmail.com>
Subject: Re: Major compactions and OS cache
From: Jason Rutherglen <jason.rutherglen@gmail.com>
To: Ryan Rawson <ryanobjc@gmail.com>
Cc: Edward Capriolo <edlinuxguru@gmail.com>, user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

> There is a patch that causes us to evict the block cache on close of
> hfile, and populate the block cache during compaction write out. =A0This
> is included in 0.90.

That's good!

> HDFS-347, which is a huge
> clear win but still no plans to include it in any hadoop version.

Why's that?  It seems to be fairly logical.  Does it affect the
'over-the-wire' protocol?

On Wed, Feb 16, 2011 at 6:23 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> There is a patch that causes us to evict the block cache on close of
> hfile, and populate the block cache during compaction write out. =A0This
> is included in 0.90.
>
> So that helps. =A0Fixing VFS issues is quite a bit longer term, since
> the on-wire format of HDFS rpc is kind of "fixed", petitioning for
> changes will be a little tricky. Again, see HDFS-347, which is a huge
> clear win but still no plans to include it in any hadoop version.
>
> -ryan
>