stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Sandaruwan <lahi...@wso2.com>
Subject Re: [GSOC - 2015] Introducing “curve fitting” for statistics prediction algorithm of Autoscaler
Date Wed, 25 Mar 2015 05:00:28 GMT
On Wed, Mar 25, 2015 at 10:10 AM, Pranavan Theivendiram <
pranavan.11@cse.mrt.ac.lk> wrote:

> Hi Lahiru,
>
> Thanks for the prompt reply.
>
> *+1 for*
> *we can find the polynomial in each minute, considering 10 minutes of past
> data, we can change this if required.*
>
> If we are getting the health stats events uniformly for every 10 seconds.
> my question is resolved. (I thought that we are getting in a random manner).
>
> Can I add the things in my proposal regarding "Curve Fitting"?
>

Yes, Please.

>
> Do you think that any other important things I need to add in my proposal?
>

It looks good. Hope you referred [1] on this.

Thanks.

[1]  http://community.apache.org/gsoc.html#applying-for-gsoc

Thanks.

>
> *T. Pranavan*
> *BSc Eng Undergraduate| Department of Computer Science & Engineering
> ,University of Moratuwa*
> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
> *Mobile| *0775136836
>
> On 25 March 2015 at 10:09, Pranavan Theivendiram <
> pranavan.11@cse.mrt.ac.lk> wrote:
>
>> Hi Lahiru,
>>
>> Thanks for the prompt reply.
>>
>> *+1 for*
>> *we can find the polynomial in each minute, considering 10 minutes of
>> past data, we can change this if required.*
>>
>> If we are getting the health stats events uniformly for every 10 seconds.
>> my question is resolved. (I thought that we are getting in a random manner).
>>
>> Can I add the things in my proposal regarding "Curve Fitting"?
>>
>> Do you think that any other important things I need to add in my proposal?
>>
>> *T. Pranavan*
>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>> ,University of Moratuwa*
>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>> *Mobile| *0775136836
>>
>> On 25 March 2015 at 09:59, Lahiru Sandaruwan <lahirus@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> +1 for using Common math library, as we already using it.
>>>
>>>    - Currently we have calculating numbers per minute, so it is not per
>>>    specified number of events, but for a specified time.
>>>    - IMO we can find the polynomial in each minute, considering 10
>>>    minutes of past data, we can change this if required.
>>>    - Normally, health stat events are received in each 10 seconds.
>>>    - Therefore we will have 60 ( = 6 * 10) events per 10 minutes.
>>>
>>> On Wed, Mar 25, 2015 at 9:11 AM, Pranavan Theivendiram <
>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>
>>>> Hi Lahiru and Raj,
>>>>
>>>> I have gone through the curve fitting materials that you have provided.
>>>> I have understood the regression part and prediction of degree polynomials.
>>>> We can use apache common math libraries to do this. I have a question below.
>>>>
>>>> How often we are going to find the polynomials?
>>>>
>>>> For example,
>>>>
>>>> If we have *3 samples* events during *10 minutes interval*, then we
>>>> can produce a second degree polynomial. (We will get constants *a*,*b*,
>>>> and *c*). If we are going to use this second degree polynomial for
>>>> next *60 minutes *(Suppose in this 60 minutes we are getting *200
>>>> samples *events).  As we can see our *extrapolation
>>>> <http://en.wikipedia.org/wiki/Extrapolation>* becomes a severe problem
>>>> in this particular example.
>>>>
>>>
>>> What do you mean by " our *extrapolation
>>> <http://en.wikipedia.org/wiki/Extrapolation>* becomes a severe
>>> problem"? Complexity of the calculation of predicted value?
>>>
>>> Thanks.
>>>
>>>> I do not know how earlier versions handle this scenario.
>>>>
>>>> Can you tell that are we going produce new polynomials for *a
>>>> specified time* or *a specified number of events*?
>>>>
>>>> Please clear this issue.
>>>>
>>>> *T. Pranavan*
>>>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>>>> ,University of Moratuwa*
>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>> *Mobile| *0775136836
>>>>
>>>> On 24 March 2015 at 09:02, Lahiru Sandaruwan <lahirus@wso2.com> wrote:
>>>>
>>>>> Thanks Pranavan,
>>>>>
>>>>> I'll have a look.
>>>>>
>>>>>
>>>>> On Tue, Mar 24, 2015 at 8:52 AM, Pranavan Theivendiram <
>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>
>>>>>> Hi Lahiru and Raj,
>>>>>>
>>>>>> This is my draft proposal which has been done up to 50%. I am going
>>>>>> to add things about curve fitting as well. Still many things need
to be
>>>>>> added. Please comment on my initial draft.
>>>>>> I am expecting pros and cons regarding my proposal. Please find the
>>>>>> link for the initial draft below.
>>>>>>
>>>>>>
>>>>>> https://docs.google.com/a/cse.mrt.ac.lk/document/d/1TzHYI2o9bIdZWj4qAffNmH5FTgeTOV4TuSsxlISlU8c/edit?usp=sharing
>>>>>>
>>>>>> *T. Pranavan*
>>>>>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>>>>>> ,University of Moratuwa*
>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>> *Mobile| *0775136836
>>>>>>
>>>>>> On 22 March 2015 at 23:36, Pranavan Theivendiram <
>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>
>>>>>>> Thanks Lahiru for the prompt reply. I have already started working
>>>>>>> on the proposal. I will send it before tomorrow.
>>>>>>>
>>>>>>> *T. Pranavan*
>>>>>>> *BSc Eng Undergraduate| Department of Computer Science &
Engineering
>>>>>>> ,University of Moratuwa*
>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>>> *Mobile| *0775136836
>>>>>>>
>>>>>>> On 22 March 2015 at 23:31, Lahiru Sandaruwan <lahirus@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Yes, You have found the information correctly and on right
track.
>>>>>>>> It would be great if we can incorporate CEP 4.0.0 changes
as it would
>>>>>>>> minimize our customized function.
>>>>>>>>
>>>>>>>> I would like to suggest you starting the drafting of the
proposal
>>>>>>>> as you have some background now.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> On Sun, Mar 22, 2015 at 9:48 PM, Pranavan Theivendiram <
>>>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> With the few days of research on the project "Introducing
“curve
>>>>>>>>> fitting” for statistics prediction algorithm of Autoscaler".
I have found
>>>>>>>>> followings regarding the project
>>>>>>>>>
>>>>>>>>>    1. ClusterMonitor class passes the calculated values
from the
>>>>>>>>>    CEP. This value passing happens after org.apache.stratos.messaging.event
>>>>>>>>>             happens.
>>>>>>>>>    2. RuleTasksDelegator class calculates the prediction
values
>>>>>>>>>    using the values(Ex gradient) from the CEP
>>>>>>>>>    3. The classes in the package org.apache.stratos.cep.extension
>>>>>>>>>    deals with the analysis of data in the real time and
publish the relevant
>>>>>>>>>    summarized health stats, which are needed for the
autoscaling.
>>>>>>>>>
>>>>>>>>> So according to my understanding, I need to modify above
mentioned
>>>>>>>>> classes, in order to implement the curve fitting for
statistics prediction.
>>>>>>>>>
>>>>>>>>> Furthermore,for this project, we can use the regression
>>>>>>>>> implementation of CEP 4.0, which is expected to be released
on May 15th
>>>>>>>>> according to WSO2 public jira. I am planning to do a
feasibility study for
>>>>>>>>> the first two weeks of the project, where we can explore
other better
>>>>>>>>> options as well.
>>>>>>>>>
>>>>>>>>> Am I on a right track? Please share your thoughts on
this.
>>>>>>>>>
>>>>>>>>> [1] https://wso2.org/jira/browse/CEP
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> *T. Pranavan*
>>>>>>>>> *BSc Eng Undergraduate| Department of Computer Science
&
>>>>>>>>> Engineering ,University of Moratuwa*
>>>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>>>>> *Mobile| *0775136836
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> --
>>>>>>>> Lahiru Sandaruwan
>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>> lean.enterprise.middleware
>>>>>>>>
>>>>>>>> phone: +94773325954
>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Lahiru Sandaruwan
>>>>> Committer and PMC member, Apache Stratos,
>>>>> Senior Software Engineer,
>>>>> WSO2 Inc., http://wso2.com
>>>>> lean.enterprise.middleware
>>>>>
>>>>> phone: +94773325954
>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Committer and PMC member, Apache Stratos,
>>> Senior Software Engineer,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> phone: +94773325954
>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>


-- 
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

phone: +94773325954
email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Mime
View raw message