hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1413) fall back to filesystem block size default if hbase.regionserver.hlog.blocksize is not specified
Date Wed, 13 May 2009 01:16:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708712#action_12708712
] 

stack commented on HBASE-1413:
------------------------------

I added logging of blocksize in HLog.  Indeed its 1MB.

I added logging of the size of log files.  I ran a test.  The last log file was as follows:

2009-05-13 01:10:44,538 [HMaster] DEBUG org.apache.hadoop.hbase.regionserver.HLog: Splitting
6 of 6: hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/.logs/aa0-000-13.u.powerset.com_1242176118526_60021/hlog.dat.1242176867603
length=87031808

i.e. 87MBs in size.

We got no edits out of it.  We just skipped over it.

I even tried adding this code to HLog splitLog:

{code}
@@ -730,8 +739,25 @@
       for (int i = 0; i < logfiles.length; i++) {
         if (LOG.isDebugEnabled()) {
           LOG.debug("Splitting " + (i + 1) + " of " + logfiles.length + ": " +
-            logfiles[i].getPath());
+            logfiles[i].getPath() + " length=" + logfiles[i].getLen());
         }
+// Trying recovery
+boolean recovered = false;
+while (!recovered) {
+  try {
+    FSDataOutputStream out = fs.append(logfiles[i].getPath());
+    out.close();
+    recovered = true;
+  } catch (IOException e) {
+    LOG.info("Triggering lease recovery", e);
+    try {
+      Thread.sleep(1000);
+    } catch (InterruptedException ex) {
+      // ignore it and try again
+    }
+  }
+}
+    LOG.info("Past out lease recovery");
{code}

Doesn't work in hadoop 0.20.0.. I get this over and over:

org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException:
failed to create file /hbasetrunk2/.logs/aa0-000-13.u.powerset.com_1242174244206_60021/hlog.dat.1242175401082
for DFSClient_-775762375 on client 208.76.44.139 because current leaseholder is trying to
recreate file.

Andrew, mind if I take this.. let me put up a patch with some better logging added.

> fall back to filesystem block size default if hbase.regionserver.hlog.blocksize is not
specified
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1413
>                 URL: https://issues.apache.org/jira/browse/HBASE-1413
>             Project: Hadoop HBase
>          Issue Type: Task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>             Fix For: 0.19.2, 0.20.0
>
>         Attachments: hbase-1413.patch
>
>
> Fall back to filesystem block size default if hbase.regionserver.hlog.blocksize is not
specified.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message