hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15537) Add multi WAL support for AsyncFSWAL
Date Tue, 05 Apr 2016 02:54:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225562#comment-15225562
] 

Yu Li commented on HBASE-15537:
-------------------------------

AFAIK which DNs to use for each block is decided by {{BlockPlacementPolicy#chooseTarget}}
in HDFS, and I just used the default policy when testing multiwal with FSHLog.

>From the current implement of {{FanOutOneBlockAsyncDFSOutput}} the block placement seems
to follow the same way as pipeline write (#createOutput still depends on NameNodeRpcServer#addBlock
for block locations):
{code}
      locatedBlock = namenode.addBlock(src, client.getClientName(), null, null, stat.getFileId(),
        null);
      List<Channel> datanodeList = new ArrayList<>();
      futureList = connectToDataNodes(conf, clientName, locatedBlock, 0L, 0L, PIPELINE_SETUP_CREATE,
        summer, eventLoop);
      for (Future<Channel> future : futureList) {
        // fail the creation if there are connection failures since we are fail-fast. The
upper
        // layer should retry itself if needed.
        datanodeList.add(future.syncUninterruptibly().getNow());
      }
{code}
So IMHO we could still believe NN for DNs load balance and don't need to worry about this
in our testing. Or could you further clarify your concern if I misunderstood the question
[~stack]? Thanks.

> Add multi WAL support for AsyncFSWAL
> ------------------------------------
>
>                 Key: HBASE-15537
>                 URL: https://issues.apache.org/jira/browse/HBASE-15537
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15537-v3.patch, HBASE-15537-v4.patch, HBASE-15537.patch, HBASE-15537_v2.patch
>
>
> The multi WAL should not be bound with {{FSHLog}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message