stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashan Dhananjaya <dhananjay...@gmail.com>
Subject Re: [Dev] [GSoC-2015] Showing health statistics in GUI
Date Thu, 30 Jul 2015 17:30:14 GMT
Hi All,

I have implemented formatters and did some changes in the cep event flow to
change the RDBMS 3 tables in to one table as we discussed last week. And
committed it. And id did few changes to the UI according to the feedback i
had last week. I finished writing the REST API documentation and sent it to
everyone.

I have started to write a blog for the marketing aspect for stratos how
Health Stat UI works and a documentation for the health statistics UI.

[1]=
https://github.com/apache/stratos/commit/1490310cae3ba085fbb77d84346dd66fd2b12132

Thank You!
Best Regards,
Ashan

On Fri, Jul 24, 2015 at 11:17 PM, Imesh Gunaratne <imesh@apache.org> wrote:

> Great work Ashan!
>
> Actually you don't need to replace MySQL database with H2 rather both
> options might be needed. The reason for having H2 is to allow users to see
> the statistics when running on Single-JVM mode with embedded H2 database.
>
> On Thu, Jul 23, 2015 at 12:35 AM, Ashan Dhananjaya <dhananjaya92@gmail.com
> > wrote:
>
>> Hi All,
>>
>> I have implemented the UI for the Health Statistics UI. The git hub
>> commits can be find from here [1].
>>
>> And im moving forward to replace the MySql database from the embedded H2
>> RDBMS as we discussed offline on 22nd of  July.
>>
>> Thank You!
>> Best Regards,
>> Ashan
>>
>>
>> [1]=
>> https://github.com/ashandk/stratos/commit/310ea343d80d009778f572da12a4098c7d2177ec
>>
>>
>> On Thu, Jul 16, 2015 at 7:29 PM, Ashan Dhananjaya <dhananjaya92@gmail.com
>> > wrote:
>>
>>> Hi Imesh,
>>>
>>> Noted. I will create a documentation for the REST API.
>>>
>>> Thank You!
>>> Best Regards,
>>> Ashan
>>>
>>> On Thu, Jul 16, 2015 at 10:51 AM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>>
>>>> On Tue, Jul 14, 2015 at 9:44 PM, Reka Thirunavukkarasu <reka@wso2.com>
>>>>  wrote:
>>>>>
>>>>>
>>>>>> 1.getAverageClusterMemoryByClusterId
>>>>>>
>>>>>
>>>>> /cluster/averageMemory/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 2.getAverageClusterLoadByClusterId
>>>>>>
>>>>>
>>>>> /cluster/averageLoad/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 3.getAverageMemberMemoryByMemberId
>>>>>>
>>>>>
>>>>> /member/averageMemory/{Id}/{Interval}}
>>>>>
>>>>> 4.getAverageMemberLoadByMemberId
>>>>>>
>>>>>
>>>>> /member/averageLoad/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 5.getAverageClusterFlightRequestCountByClusterId
>>>>>>
>>>>>
>>>>>  /cluster/flightRequestCount/{Id}/{Interval}}
>>>>>
>>>>
>>>> +1 A great thought! @Ashan: It would be better if you can document the
>>>> API methods (may be in a Google doc).
>>>>
>>>> Thanks
>>>>
>>>> On Tue, Jul 14, 2015 at 9:44 PM, Reka Thirunavukkarasu <reka@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Ashan,
>>>>>
>>>>> Wouldn't it be better to stick with the REST API paths currently used
>>>>> by stratos? You can inherit and add more stuffs as you want. Please see
>>>>> below for some sample paths:
>>>>>
>>>>> On Tue, Jul 14, 2015 at 1:55 AM, Ashan Dhananjaya <
>>>>> dhananjaya92@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I have implemented the REST api methods for the health statistics
UI
>>>>>> in the stratos existing REST API.
>>>>>>
>>>>>> 1.getAverageClusterMemoryByClusterId
>>>>>>
>>>>>
>>>>> /cluster/averageMemory/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 2.getAverageClusterLoadByClusterId
>>>>>>
>>>>>
>>>>> /cluster/averageLoad/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 3.getAverageMemberMemoryByMemberId
>>>>>>
>>>>>
>>>>> /member/averageMemory/{Id}/{Interval}}
>>>>>
>>>>> 4.getAverageMemberLoadByMemberId
>>>>>>
>>>>>
>>>>> /member/averageLoad/{Id}/{Interval}}
>>>>>
>>>>>
>>>>>> 5.getAverageClusterFlightRequestCountByClusterId
>>>>>>
>>>>>
>>>>>  /cluster/flightRequestCount/{Id}/{Interval}}
>>>>>
>>>>> Thanks,
>>>>> Reka
>>>>>
>>>>>>
>>>>>> Please find the Github commit here[1].
>>>>>>
>>>>>> [1]=
>>>>>> https://github.com/apache/stratos/commit/5ab9159744f136e649174f7293adabe5c77fb4bd
>>>>>>
>>>>>>
>>>>>> Thank You!
>>>>>> Best Regards,
>>>>>> Ashan
>>>>>>
>>>>>> On Thu, Jul 9, 2015 at 8:30 PM, Ashan Dhananjaya <
>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>
>>>>>>> Noted Lahiru.
>>>>>>>
>>>>>>> On Thu, Jul 9, 2015 at 7:48 PM, Lahiru Sandaruwan <lahirus@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ashan,
>>>>>>>>
>>>>>>>> Good stuff. It is fine to set the time stamp from CEP for
now. But
>>>>>>>> ultimately we need the timestamp to be sent from Python cartridge
agent.
>>>>>>>> Let's fix that part we proceed.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> On Thu, Jul 9, 2015 at 7:38 PM, Ashan Dhananjaya <
>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I was able to get the TimeaStamp to the RDBMS without
any conflict
>>>>>>>>> AFAIK. I am testing the flow more. Please find the commits
[1], [2] for the
>>>>>>>>> Formatters.
>>>>>>>>>
>>>>>>>>> [1]=
>>>>>>>>> https://github.com/ashandk/stratos/commit/efa25eb255cadbf2843e5f0d0e13b34c09535f91
>>>>>>>>> [2]=
>>>>>>>>> https://github.com/ashandk/stratos/commit/9c0fce9081c299c34031d602c94ea712602c7a6c
>>>>>>>>>
>>>>>>>>> Thank You!
>>>>>>>>> Best Regards,
>>>>>>>>> Ashan
>>>>>>>>>
>>>>>>>>> On Thu, Jul 9, 2015 at 2:41 PM, Lahiru Sandaruwan <
>>>>>>>>> lahirus@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Great work Ashan.
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 9, 2015 at 2:39 PM, Ashan Dhananjaya
<
>>>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> I was able to take the timestamp from the cartridge
agent
>>>>>>>>>>> (WSO2CEPHealthStatisticsPublisher and WSO2CEPInFlightRequestPublisher)
to
>>>>>>>>>>> CEP PayLoad definition.Git Hub commits can be
found here[1], [2]. Currently
>>>>>>>>>>> working on sending the timestamp to the RDBMS
without conflicting execution
>>>>>>>>>>> plans.
>>>>>>>>>>>
>>>>>>>>>>> I have included the read-me [2] file to the cep
extensions how
>>>>>>>>>>> to configure should happen for the Health Stat
publishing to the RDBMS.
>>>>>>>>>>>
>>>>>>>>>>> [1]=
>>>>>>>>>>> https://github.com/apache/stratos/commit/4c65e705be822fd420b4e48ebafef789c8686b76
>>>>>>>>>>> [2]=
>>>>>>>>>>> https://github.com/apache/stratos/commit/e09334cd98f245a4633c992f02db2878ac24d3d0
>>>>>>>>>>> [3]=
>>>>>>>>>>> https://github.com/apache/stratos/commit/5ca895ef030f6aaeba56183d982aa5ae95db9337
>>>>>>>>>>>
>>>>>>>>>>> Thank You!
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ashan
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jul 7, 2015 at 1:13 PM, Lasantha Fernando
<
>>>>>>>>>>> lasantha.fdo@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Ashan,
>>>>>>>>>>>>
>>>>>>>>>>>> Currently, you would not be able to get the
timestamp out of
>>>>>>>>>>>> the event
>>>>>>>>>>>> unless you use a databridge agent callback.
In that case, you
>>>>>>>>>>>> can use
>>>>>>>>>>>> Java code to simply get by calling event.getTimestamp().
>>>>>>>>>>>>
>>>>>>>>>>>> However, if you want to use the timestamp
as an attribute while
>>>>>>>>>>>> doing
>>>>>>>>>>>> processing within CEP, you would have to
pass the timestamp as
>>>>>>>>>>>> an
>>>>>>>>>>>> attribute of the stream. i.e. add it as an
attribute of either
>>>>>>>>>>>> meta,
>>>>>>>>>>>> correlation or payload data. Then you can
use that timestamp and
>>>>>>>>>>>> format the output event as necessary.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Lasantha
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jul 7, 2015 at 1:04 PM, Ashan Dhananjaya
<
>>>>>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>>>>> > Hi All,
>>>>>>>>>>>> >
>>>>>>>>>>>> > I applied the above patch and get the
timestamp in the Input
>>>>>>>>>>>> Adapter. The
>>>>>>>>>>>> > formatter i used was
>>>>>>>>>>>> "member_average_memory_consumption_stats:1.0.0".
I
>>>>>>>>>>>> > tried several time to get the timeStamp
attribute but failed.
>>>>>>>>>>>> Is there any
>>>>>>>>>>>> > way to get a data which is outside in
the payload? The Input
>>>>>>>>>>>> stream is as
>>>>>>>>>>>> > below,
>>>>>>>>>>>> >
>>>>>>>>>>>> > 10:40:27,757 [-] [DataBridge-Core-pool-2-thread-1]
 INFO
>>>>>>>>>>>> TenantId=-1234 :
>>>>>>>>>>>> > Input Event Adaptor : DefaultWSO2EventInputAdaptor,
received
>>>>>>>>>>>> >
>>>>>>>>>>>> > Event{
>>>>>>>>>>>> >   streamId='cartridge_agent_health_stats:1.0.0',
>>>>>>>>>>>> >   timeStamp=1436245827731,
>>>>>>>>>>>> >   metaData=null,
>>>>>>>>>>>> >   correlationData=null,
>>>>>>>>>>>> >   payloadData=[single-cartridge-app.my-php.php.domain,
>>>>>>>>>>>> > single-cartridge-app-1, network-partition-1,
>>>>>>>>>>>> >
>>>>>>>>>>>> single-cartridge-app.my-php.php.domaina5598e2b-995f-4620-b652-e66341e24c99,
>>>>>>>>>>>> > partition-1, memory_consumption, 20.0],
>>>>>>>>>>>> >   arbitraryDataMap={},
>>>>>>>>>>>> > }
>>>>>>>>>>>> >
>>>>>>>>>>>> > Any help will be really appreciated.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thank You!
>>>>>>>>>>>> > Best Regards,
>>>>>>>>>>>> > Ashan
>>>>>>>>>>>> >
>>>>>>>>>>>> > On Tue, Jul 7, 2015 at 9:32 AM, Chamila
De Alwis <
>>>>>>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Hi Ashan,
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> As I've seen from the PCA testing
recently, the timestamp is
>>>>>>>>>>>> added to
>>>>>>>>>>>> >> every ThriftEvent before publishing.
Are you working on PCA
>>>>>>>>>>>> or the Mock
>>>>>>>>>>>> >> IaaS? I took a look at the Mock
IaaS and the
>>>>>>>>>>>> WSO2CEPStatisticsPublisher as
>>>>>>>>>>>> >> well, and found out that the latter
indeed doesn't add the
>>>>>>>>>>>> timestamp to the
>>>>>>>>>>>> >> ThriftEvents. This means that the
JCA also publishes Thrift
>>>>>>>>>>>> Events with
>>>>>>>>>>>> >> empty timestamp fields.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Following is the simple fix for
that. I've attached the
>>>>>>>>>>>> patch herewith.
>>>>>>>>>>>> >> Could you quickly verify with the
patch applied build also?
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> chamilad@chamilad-ThinkPad-T530:~/dev/stratos[master
*]$
>>>>>>>>>>>> git diff
>>>>>>>>>>>> >> diff --git
>>>>>>>>>>>> >>
>>>>>>>>>>>> a/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>> >> b/c
>>>>>>>>>>>> >> index 653288d..f1fe426 100644
>>>>>>>>>>>> >> ---
>>>>>>>>>>>> >>
>>>>>>>>>>>> a/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>> >> +++
>>>>>>>>>>>> >>
>>>>>>>>>>>> b/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>> >> @@ -29,6 +29,7 @@ import
>>>>>>>>>>>> >>
>>>>>>>>>>>> org.wso2.carbon.databridge.agent.thrift.exception.AgentException;
>>>>>>>>>>>> >>  import org.wso2.carbon.databridge.commons.Event;
>>>>>>>>>>>> >>  import org.wso2.carbon.databridge.commons.StreamDefinition;
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> +import java.util.Date;
>>>>>>>>>>>> >>  import java.util.HashMap;
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>  /**
>>>>>>>>>>>> >> @@ -98,6 +99,7 @@ public class WSO2CEPStatisticsPublisher
>>>>>>>>>>>> implements
>>>>>>>>>>>> >> StatisticsPublisher {
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>          Event event = new Event();
>>>>>>>>>>>> >>          event.setPayloadData(payload);
>>>>>>>>>>>> >> +        event.setTimeStamp(new
Date().getTime());
>>>>>>>>>>>> >>          event.setArbitraryDataMap(new
HashMap<String,
>>>>>>>>>>>> String>());
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>          try {
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Regards,
>>>>>>>>>>>> >> Chamila de Alwis
>>>>>>>>>>>> >> Committer and PMC Member - Apache
Stratos
>>>>>>>>>>>> >> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>> >> Blog: code.chamiladealwis.com
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> --
>>>>>>>>>> Lahiru Sandaruwan
>>>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>>>> Senior Software Engineer,
>>>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>
>>>>>>>>>> phone: +94773325954
>>>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>>>> linked-in:
>>>>>>>>>> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Dev mailing list
>>>>>>>>>> Dev@wso2.org
>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> --
>>>>>>>> Lahiru Sandaruwan
>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>> lean.enterprise.middleware
>>>>>>>>
>>>>>>>> phone: +94773325954
>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Reka Thirunavukkarasu
>>>>> Senior Software Engineer,
>>>>> WSO2, Inc.:http://wso2.com,
>>>>> Mobile: +94776442007
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>
>
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>

Mime
View raw message