Return-Path: X-Original-To: apmail-stratos-dev-archive@minotaur.apache.org Delivered-To: apmail-stratos-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 659A69526 for ; Tue, 9 Dec 2014 09:06:46 +0000 (UTC) Received: (qmail 71490 invoked by uid 500); 9 Dec 2014 09:06:46 -0000 Delivered-To: apmail-stratos-dev-archive@stratos.apache.org Received: (qmail 71435 invoked by uid 500); 9 Dec 2014 09:06:46 -0000 Mailing-List: contact dev-help@stratos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@stratos.apache.org Delivered-To: mailing list dev@stratos.apache.org Received: (qmail 71423 invoked by uid 99); 9 Dec 2014 09:06:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Dec 2014 09:06:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lahirus@wso2.com designates 209.85.216.170 as permitted sender) Received: from [209.85.216.170] (HELO mail-qc0-f170.google.com) (209.85.216.170) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Dec 2014 09:06:18 +0000 Received: by mail-qc0-f170.google.com with SMTP id x3so87140qcv.15 for ; Tue, 09 Dec 2014 01:06:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wso2.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=HRame7+a7lrkySXGa7Z0KU5fAWttUPsBmKVtMsdl/NY=; b=Y0CjUZQHnb2li+H0XEDO4HTn0wJPXlyItQ8bvoc50s99mbwjoW9xuNKnwpQfuSlhfH rnxlRAEI+VitKJ0kBAYtCZR1peI2gQkkJN4ldWTSkL/aToixmQ5bwiZJ83JPu6WxEw9A MkaIJtsG+eXse8bdpl1rRTbS/53VcGRlfUcOI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=HRame7+a7lrkySXGa7Z0KU5fAWttUPsBmKVtMsdl/NY=; b=VuJy+jaHTZOzwU+0N8Ha19WLuWMZJ/U5z5TjQelPKrXXCoIqVlrdzPWyR2RNiu9BdL /L7N316He8orpSH5UbAlsQ2PCZFTSSqCuoHAff8yCtNjbZzmndeipLPznKKdCxeQobkB FOz9acNpvWGwU1h6s8sf3uRxHZqpEBxSmYPrXuhlfxUgXUmGyGLIdz0AjmTJD0Hssz5c s/g5+XSX39MfA1Mkb3YEnQt5BzcXLYYIB7IMjFipN3n+IdOxR/isumyhXYmfEwzatFMU UYE9T506eSNyQFveqQdKaYe3eBJwlbNaF2m1UQWqbsmYJJOa7XlYj1nu47AydyS9EmOV zKxg== X-Gm-Message-State: ALoCoQmJJFWBtvLLSiRmlG4Pd4e7aNWv2qJ4TNf0JVCMYyrFDD2IY6j29Z8gqtrhlOO/Kjp363E7 X-Received: by 10.140.84.197 with SMTP id l63mr3115537qgd.100.1418115976776; Tue, 09 Dec 2014 01:06:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.50.170 with HTTP; Tue, 9 Dec 2014 01:05:46 -0800 (PST) In-Reply-To: References: From: Lahiru Sandaruwan Date: Tue, 9 Dec 2014 14:35:46 +0530 Message-ID: Subject: Re: Member ID is not found in Member Fault Event To: dev Content-Type: multipart/alternative; boundary=001a11c1550c0be5520509c4dbcb X-Virus-Checked: Checked by ClamAV on apache.org --001a11c1550c0be5520509c4dbcb Content-Type: text/plain; charset=UTF-8 I will fix it to use this method when scaling down. Earlier we used SM service call. Thanks. On Tue, Dec 9, 2014 at 2:12 PM, Imesh Gunaratne wrote: > I fixed the above issue and pushed it to 4.1.0-test branch. > > Now I see another problem: Instance clean up event is not published to > member. Seems > like InstanceNotificationPublisher.sendInstanceCleanupEventForMember() > method in Autoscaler is not used. > > Thanks > > > > On Tue, Dec 9, 2014 at 1:34 PM, Imesh Gunaratne wrote: > >> Found the problem, InstanceSpawnedMessageProcessor has not added instance >> id to the member object. >> >> On Tue, Dec 9, 2014 at 1:07 PM, Imesh Gunaratne wrote: >> >>> Now I can see another problem in autoscaler in member fault handling >>> scenario: >>> >>> [2014-12-09 13:04:03,795] ERROR >>> {org.apache.stratos.messaging.message.receiver.health.stat.HealthStatEventMessageDelegator} >>> - Failed to retrieve instance notifier event message >>> java.lang.NullPointerException >>> at >>> org.apache.stratos.autoscaler.monitor.cluster.VMClusterMonitor.handleMemberFaultEvent(VMClusterMonitor.java:785) >>> at >>> org.apache.stratos.autoscaler.event.receiver.health.AutoscalerHealthStatEventReceiver$10.onEvent(AutoscalerHealthStatEventReceiver.java:330) >>> at >>> org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42) >>> at java.util.Observable.notifyObservers(Observable.java:159) >>> at >>> org.apache.stratos.messaging.event.EventObservable.notifyEventListeners(EventObservable.java:51) >>> >>> On Tue, Dec 9, 2014 at 1:06 PM, Imesh Gunaratne >>> wrote: >>> >>>> This is now fixed in 4.1.0-test branch. >>>> >>>> On Tue, Dec 9, 2014 at 12:54 PM, Imesh Gunaratne >>>> wrote: >>>> >>>>> Found the problem, FaultHandlingWindowProcessor has invoked the member >>>>> fault event constructor with a wrong parameter order: >>>>> >>>>> public MemberFaultEvent(String clusterId, String memberId, String >>>>> partitionId, >>>>> String instanceId, String >>>>> networkPartitionId, float value) { >>>>> >>>>> MemberFaultEvent memberFaultEvent = new >>>>> MemberFaultEvent(member.getClusterId(), member.getInstanceId(), >>>>> member.getMemberId(), >>>>> member.getPartitionId(), >>>>> member.getNetworkPartitionId(), 0) >>>>> >>>>> On Tue, Dec 9, 2014 at 12:41 PM, Imesh Gunaratne >>>>> wrote: >>>>> >>>>>> Hi Devs, >>>>>> >>>>>> I see the below error in autoscaler: >>>>>> >>>>>> [2014-12-09 12:40:02,716] INFO >>>>>> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} - Faulty >>>>>> member detected [member-id] >>>>>> tom1group6.tomcat1.domain587441ed-2b02-42a9-aec7-77ba0b193d37 with [last >>>>>> time-stamp] 1418108426872 [time-out] 60000 milliseconds >>>>>> [2014-12-09 12:40:02,716] INFO >>>>>> {org.apache.stratos.cep.extension.FaultHandlingWindowProcessor} - >>>>>> Publishing member fault event for [member-id] >>>>>> tom1group6.tomcat1.domain587441ed-2b02-42a9-aec7-77ba0b193d37 >>>>>> [2014-12-09 12:40:02,724] ERROR >>>>>> {org.apache.stratos.autoscaler.event.receiver.health.AutoscalerHealthStatEventReceiver} >>>>>> - Member id not found in received message >>>>>> >>>>>> May be CEP is not sending the member id in this event. >>>>>> >>>>>> Thanks >>>>>> >>>>>> >>>>>> -- >>>>>> Imesh Gunaratne >>>>>> >>>>>> Technical Lead, WSO2 >>>>>> Committer & PMC Member, Apache Stratos >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Imesh Gunaratne >>>>> >>>>> Technical Lead, WSO2 >>>>> Committer & PMC Member, Apache Stratos >>>>> >>>> >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>> >>> >>> >>> -- >>> Imesh Gunaratne >>> >>> Technical Lead, WSO2 >>> Committer & PMC Member, Apache Stratos >>> >> >> >> >> -- >> Imesh Gunaratne >> >> Technical Lead, WSO2 >> Committer & PMC Member, Apache Stratos >> > > > > -- > Imesh Gunaratne > > Technical Lead, WSO2 > Committer & PMC Member, Apache Stratos > -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 --001a11c1550c0be5520509c4dbcb Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I will fix it to use this method when scaling down. Earlie= r we used SM service call.

Thanks.

On Tue, Dec 9, 2014 at 2:12= PM, Imesh Gunaratne <imesh@apache.org> wrote:
I fixed the above issue and pushed it = to 4.1.0-test branch.=C2=A0

Now I see another problem: I= nstance clean up event is not published to member. Seems like=C2=A0Instance= NotificationPublisher.sendInstanceCleanupEventForMember() method in Autosca= ler is not used.

Thanks



On Tue, Dec 9, 2014 at 1:34 PM, I= mesh Gunaratne <imesh@apache.org> wrote:
Found the problem,=C2=A0InstanceSpawnedMessa= geProcessor has not added instance id to the member object.
=

On Tue, Dec 9, 20= 14 at 1:07 PM, Imesh Gunaratne <imesh@apache.org> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Now I can see another proble= m in autoscaler in member fault handling scenario:

= [2014-12-09 13:04:03,795] ERROR {org.apache.stratos.messaging.message.recei= ver.health.stat.HealthStatEventMessageDelegator} - =C2=A0Failed to retrieve= instance notifier event message
java.lang.NullPointerException
at org.apache.stratos= .autoscaler.monitor.cluster.VMClusterMonitor.handleMemberFaultEvent(VMClust= erMonitor.java:785)
= at org.apache.stratos.autoscaler.event.receiver.health.AutoscalerHealthStat= EventReceiver$10.onEvent(AutoscalerHealthStatEventReceiver.java:330)
<= div> at org.apache.stratos.mess= aging.listener.EventListener.update(EventListener.java:42)
at java.util.Observable.notifyObserv= ers(Observable.java:159)
at org.apache.stratos.messaging.event.EventObservable.notifyEventListe= ners(EventObservable.java:51)

On Tue, Dec 9, 2014 at 1:06 PM, Ime= sh Gunaratne <imesh@apache.org> wrote:
This is now fixed in 4.1.0-test branch.

On Tue= , Dec 9, 2014 at 12:54 PM, Imesh Gunaratne <imesh@apache.org>= wrote:
Found the proble= m, FaultHandlingWindowProcessor has invoked the member fault event construc= tor with a wrong parameter order:

public MemberFaultEven= t(String clusterId, String memberId, String partitionId,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 String instanceId, String networkPartitionId, floa= t value) {

MemberFaultEvent memberFaultEvent = =3D new MemberFaultEvent(member.getClusterId(), member.getInstanceId(), mem= ber.getMemberId(),
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 member.getPartitionId(), member.getNetworkPartitionId(), 0)

On Tue, Dec 9, 2014 at 12:41 PM, Imesh Gunaratne <imesh@apache.= org> wrote:
Hi Devs,

I see the below error in autoscaler:

[2014-12-09 12:40:02,716] =C2=A0INFO {org.apache.stra= tos.cep.extension.FaultHandlingWindowProcessor} - =C2=A0Faulty member detec= ted [member-id] tom1group6.tomcat1.domain587441ed-2b02-42a9-aec7-77ba0b193d= 37 with [last time-stamp] 1418108426872 [time-out] 60000 milliseconds
=
[2014-12-09 12:40:02,716] =C2=A0INFO {org.apache.stratos.cep.extension= .FaultHandlingWindowProcessor} - =C2=A0Publishing member fault event for [m= ember-id] tom1group6.tomcat1.domain587441ed-2b02-42a9-aec7-77ba0b193d37
[2014-12-09 12:40:02,724] ERROR {org.apache.stratos.autoscaler.event= .receiver.health.AutoscalerHealthStatEventReceiver} - =C2=A0Member id not f= ound in received message

May be CEP is not s= ending the member id in this event.

Thanks


--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
<= /div>



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
=



--
=
Imesh Gunaratne

Technical Lead, WS= O2
Committer & PMC Member, Apache Stratos=



--
=
Imesh Gunaratne

Technical Lead, WS= O2
Committer & PMC Member, Apache Stratos=



--
=
Imesh Gunaratne

Technical Lead, WS= O2
Committer & PMC Member, Apache Stratos=



--
=
Imesh Gunaratne

Technical Lead, WS= O2
Committer & PMC Member, Apache Stratos=



--
=
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior=C2=A0Software = Engineer,
WSO2 Inc., http:= //wso2.com
--001a11c1550c0be5520509c4dbcb--