hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11620) Add Support for Load Balancing across a group of KMS servers for HA
Date Wed, 25 Feb 2015 22:39:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337359#comment-14337359
] 

Andrew Wang commented on HADOOP-11620:
--------------------------------------

Thanks for revving Arun, I think just a few nits you might have missed:

{code}
  LoadBalancingKMSClientProvider(KMSClientProvider[] providers, long seed,
{code}

This is still named seed.

Thinking about it a bit too, are we worried about convoy effects? With a bunch of clients
doing RR, if one KMS goes down, all the clients are going to pile up until they timeout on
that one KMS. Then the next KMS in the list is going to get hammered when all the clients
try the next one.

The easy fix is to shuffle the list of KMSs initially.

{code}
Actually atomic wont work here.. since I have to do modulo increment. but ive changed it increment
in a synchronized scope
{code}

You could use compareAndSet in a spinloop right? I see it's also still returning the {{currentIdx+1}}
rather than {{currentIdx}}, which as I said above makes using the test constructor a little
weird since it starts one after.

{code}
    while (true) {
{code}

Why not a for loop here?

{code}
        LOG.warn("KMS provider at [{}] threw an IOException !!",
{code}

Let's print the IOE message in this log, not the other one.

> Add Support for Load Balancing across a group of KMS servers for HA
> -------------------------------------------------------------------
>
>                 Key: HADOOP-11620
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11620
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: kms
>    Affects Versions: 2.6.0
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: HADOOP-11620.1.patch, HADOOP-11620.2.patch, HADOOP-11620.3.patch,
HADOOP-11620.4.patch, HADOOP-11620.5.patch
>
>
> This patch needs to add support for :
> * specification of multiple hostnames in the kms key provider uri
> * KMS client to load balance requests across the hosts specified in the kms keyprovider
uri.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message