hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Fri, 21 Oct 2016 15:55:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595488#comment-15595488

ramkrishna.s.vasudevan commented on HBASE-16890:

Tried the following
-.> Disabled WAL roll writer in both cases. That does not help in performance.
-> Ensured we avoid that one copy from CellEncoder to BAOS and then to AsyncOutput. Ya
we get some improvement here but not much.
-> bq.summer.calculateChunkedSums(buf.nioBuffer(), checksumBuf.nioBuffer(0, checksumLen));
In the AsyncOutput case we create direct bytebuffer. The internal impl of the DataCheckSum
does not handle DBB and does internal copy to onheap byte[].
public void calculateChunkedSums(ByteBuffer data, ByteBuffer checksums) {
    if (type.size == 0) return;
    if (data.hasArray() && checksums.hasArray()) {
      System.out.println("coming here on onheap");
      calculateChunkedSums(data.array(), data.arrayOffset() + data.position(), data.remaining(),
          checksums.array(), checksums.arrayOffset() + checksums.position());

    if (NativeCrc32.isAvailable()) {
      NativeCrc32.calculateChunkedSums(bytesPerChecksum, type.id,
          checksums, data);
    try {
      System.out.println("coming here for dbb");
      byte[] buf = new byte[bytesPerChecksum];
      while (data.remaining() > 0) {
        int n = Math.min(data.remaining(), bytesPerChecksum);
        data.get(buf, 0, n);
        summer.update(buf, 0, n);
    } finally {
So changed to work with onheap BB. Still not yet able to reach the perf close to Default WAL.

Will report back.

> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: contention.png, contention_defaultWAL.png
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.

This message was sent by Atlassian JIRA

View raw message