Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of ddlatham@gmail.com designates
 209.85.215.42 as permitted sender)
MIME-Version: 1.0
Sender: ddlatham@gmail.com
From: Dave Latham <latham@davelink.net>
Date: Mon, 24 Jun 2013 10:32:29 -0700
Message-ID: 
 <CAA4mzo=i_sY3MTXkMKZjqy839=DGV+HnHemaR+ir8GCdX6L79A@mail.gmail.com>
Subject: region opening slow on large tables
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a11c34f702ada1b04dfe9cd42

--001a11c34f702ada1b04dfe9cd42
Content-Type: text/plain; charset=ISO-8859-1

I'd really appreciate some input on HBASE-8778 from the developers.

Short summary:
When opening regions, the region server (via FSTableDescriptors) scans the
entire table directory to find the latest table descriptor file.  On large
tables this can make region opening slow as it invokes many round trips to
the NameNode and a lot of data transferred.  On our production cluster it
really slows down our MTTR as it takes 10 minutes to open 200+ regions
after logs are split.

I put a patch up there that is solving the issue for us, but not sure if
it's the direction the community would want to go.

Thanks for taking a look,
Dave

--001a11c34f702ada1b04dfe9cd42--