Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Message-ID: <55FC291A.3060303@gmail.com>
Date: Fri, 18 Sep 2015 11:09:14 -0400
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Postbox 3.0.11 (Macintosh/20140602)
MIME-Version: 1.0
To: user@accumulo.apache.org
Subject: Re: Warning about outstanding migrations
References: <55FBA3AE.60402@orkash.com>
In-Reply-To: <55FBA3AE.60402@orkash.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

A migration is the Master choosing to move a Tablet from TabletServerX 
to TabletServerY.

Balancing is the process of moving a Tablet, again from one TabletServer 
to another, per the configured LoadBalancer implementation to meet some 
Tablet distribution.

In this case, there is a migration for one Tablet. If the Balancer were 
to run now, the current state of Tablets it would operate on would be 
invalidate because the current distribution of Tablets doesn't match the 
desired distribution.

A good example would be a system with 10 tablets on one tabletserver. A 
second tabletserver is started, but has no tablets yet. The balancer 
would not want to rebalance [10, 0], so it would wait until the 
migrations have been processed.

As long as balancing is happening (you don't constantly see such a 
warning message), you can safely ignore it. Sometimes a server will take 
a little time to process the migration due to load. However, we have had 
bugs in the past where the Master gets stuck thinking there are 
outstanding migrations and never runs the balancer. This causes lots of 
problems after some time. If you are running into this, just restart the 
master when you see this happen and consider updating to a newer version 
of Accumulo (I'm not aware of any current bugs with this issue).

- Josh

mohit.kaushik wrote:
> I am getting the following error usually in my master log file..
>
> 2015-09-17 10:22:55,798 [balancer.DefaultLoadBalancer] WARN : Not
> balancing due to 1 outstanding migrations.
> 2015-09-17 10:22:55,798 [balancer.DefaultLoadBalancer] WARN : Not
> balancing due to 1 outstanding migrations.
> 2015-09-17 10:22:55,798 [balancer.DefaultLoadBalancer] WARN : Not
> balancing due to 1 outstanding migrations.
> 2015-09-17 10:23:00,823 [master.EventCoordinator] INFO : Migrating 1
> more tablets, 1 total
> 2015-09-17 10:23:01,034 [master.EventCoordinator] INFO : [Normal
> Tablets]: 1 tablets unloaded
> 2015-09-17 10:23:01,071 [master.EventCoordinator] INFO : tablet
> 7n;EsBihar_1234|modi_1234< was unloaded from orkash1:9997
> 2015-09-17 10:23:01,272 [master.Master] INFO : Assigning 1 tablets
> 2015-09-17 10:23:01,352 [master.EventCoordinator] INFO : [Normal
> Tablets]: 1 tablets are UNASSIGNED
> 2015-09-17 10:23:01,352 [master.EventCoordinator] INFO : [Normal
> Tablets]: 474 tablets are HOSTED
> 2015-09-17 10:23:01,381 [master.EventCoordinator] INFO : tablet
> 7n;EsBihar_1234|modi_1234< was loaded on orkash2:9997
> 2015-09-17 10:23:01,606 [master.EventCoordinator] INFO : [Normal
> Tablets]: 475 tablets are HOSTED
>
>
> Can anybody explain, what are the outstanding migrations in Accumulo and
> how do they effect balancing?
>
> Thanks
> Mohit Kaushik//