hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhihong Yu (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4608) HLog Compression
Date Fri, 20 Jan 2012 00:12:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189520#comment-13189520
] 

Zhihong Yu commented on HBASE-4608:
-----------------------------------

I tried to run TestWALReplayCompressed:
{code}
Running org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.008 sec

Results :

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (secondPartTestsExecution) @ hbase ---
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:50.838s
{code}
Looks like the ShutdownHooks took a long time to finish:
{code}
"main" prio=5 tid=104000800 nid=0x100601000 in Object.wait() [100600000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Thread.join(Thread.java:1210)
	- locked <78e887470> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer)
	at java.lang.Thread.join(Thread.java:1263)
	at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
	at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
	at java.lang.Shutdown.runHooks(Shutdown.java:79)
	at java.lang.Shutdown.sequence(Shutdown.java:123)
	at java.lang.Shutdown.exit(Shutdown.java:168)
	- locked <7faf9d288> (a java.lang.Class for java.lang.Shutdown)
	at java.lang.Runtime.exit(Runtime.java:90)
	at java.lang.System.exit(System.java:921)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:73)
{code}
                
> HLog Compression
> ----------------
>
>                 Key: HBASE-4608
>                 URL: https://issues.apache.org/jira/browse/HBASE-4608
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Li Pi
>            Assignee: Li Pi
>         Attachments: 4608v1.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt
>
>
> The current bottleneck to HBase write speed is replicating the WAL appends across different
datanodes. We can speed up this process by compressing the HLog. Current plan involves using
a dictionary to compress table name, region id, cf name, and possibly other bits of repeated
data. Also, HLog format may be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message