Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <973448870.487911267003589507.JavaMail.jira@brutus.apache.org>
Date: Wed, 24 Feb 2010 09:26:29 +0000 (UTC)
From: "ryan rawson (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2165) Improve fragmentation display and
 implementation
In-Reply-To: <647179909.11161264426534559.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837713#action_12837713 ] 

ryan rawson commented on HBASE-2165:
------------------------------------

I'm not sure its perfectly clear that 'more fragmented = slower'.  certainly we should be as fast under 2 store files as 1, since it is nearly impossible on a real cluster to get to 1 store file per region.

Another issue, is this calculation is done inline with the web hit (contrary to what everyone has told me), jstack says thus:

"1674385661@qtp0-5" prio=10 tid=0x0000000041f25800 nid=0x419a in Object.wait() [0x00007f43b0fe8000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.hadoop.ipc.Client.call(Client.java:752)
        - locked <0x00007f43c77725f8> (a org.apache.hadoop.ipc.Client$Call)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:223)
        at $Proxy0.getListing(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
        at $Proxy0.getListing(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:709)
        at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:289)
        at org.apache.hadoop.hbase.util.FSUtils.getTableFragmentation(FSUtils.java:441)
        at org.apache.hadoop.hbase.util.FSUtils.getTableFragmentation(FSUtils.java:395)
        at org.apache.hadoop.hbase.master.HMaster.getTableFragmentation(HMaster.java:1265)
        at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:69)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1124)
        at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:667)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

> Improve fragmentation display and implementation
> ------------------------------------------------
>
>                 Key: HBASE-2165
>                 URL: https://issues.apache.org/jira/browse/HBASE-2165
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.20.4, 0.21.0
>            Reporter: Lars George
>            Priority: Minor
>             Fix For: 0.20.4, 0.21.0
>
>
> Improve by 
> - moving the "blocking" FS scan into a thread so that the UI loads fast and initially displays "n/a" but once it has completed the scan it displays the proper numbers
> - explaining what fragmentation means to the user (better hints or help text in UI)
> - Switch -ROOT- (and maybe even .META.?) to simply say "Yes" or a tick that it is fragmented as it only has 0% or 100% available (since it has only a single region)
> - also computing the space occupied by each table and the total and - if easily done - add a graph to display it (Google Pie Chart would be nice but is an external link)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.