stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Sandaruwan <lahi...@wso2.com>
Subject Re: [GSOC - 2015] Introducing “curve fitting” for statistics prediction algorithm of Autoscaler
Date Wed, 25 Mar 2015 13:15:45 GMT
Sure. Let me check on possible time.

Thanks/

On Wed, Mar 25, 2015 at 5:04 PM, Pranavan Theivendiram <
pranavan.11@cse.mrt.ac.lk> wrote:

> Hi Lahiru,
>
> Shall we have hangout on saturday?
>
> *T. Pranavan*
> *BSc Eng Undergraduate| Department of Computer Science & Engineering
> ,University of Moratuwa*
> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
> *Mobile| *0775136836
>
> On 25 March 2015 at 10:39, Pranavan Theivendiram <
> pranavan.11@cse.mrt.ac.lk> wrote:
>
>> Ok Lahiru.
>>
>> I have noted down some problems in understanding the code base. We will
>> have hangout in near future and let us resolve it out.
>>
>> *T. Pranavan*
>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>> ,University of Moratuwa*
>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>> *Mobile| *0775136836
>>
>> On 25 March 2015 at 10:36, Lahiru Sandaruwan <lahirus@wso2.com> wrote:
>>
>>> Yes, Please arrange a Google hangout if needed.
>>>
>>> Thanks.
>>>
>>> On Wed, Mar 25, 2015 at 10:35 AM, Pranavan Theivendiram <
>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>
>>>> Thanks Lahiru,
>>>>
>>>> I will contact you if I need any assistance.
>>>>
>>>> *T. Pranavan*
>>>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>>>> ,University of Moratuwa*
>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>> *Mobile| *0775136836
>>>>
>>>> On 25 March 2015 at 10:30, Lahiru Sandaruwan <lahirus@wso2.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Mar 25, 2015 at 10:10 AM, Pranavan Theivendiram <
>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>
>>>>>> Hi Lahiru,
>>>>>>
>>>>>> Thanks for the prompt reply.
>>>>>>
>>>>>> *+1 for*
>>>>>> *we can find the polynomial in each minute, considering 10 minutes
of
>>>>>> past data, we can change this if required.*
>>>>>>
>>>>>> If we are getting the health stats events uniformly for every 10
>>>>>> seconds. my question is resolved. (I thought that we are getting
in a
>>>>>> random manner).
>>>>>>
>>>>>> Can I add the things in my proposal regarding "Curve Fitting"?
>>>>>>
>>>>>
>>>>> Yes, Please.
>>>>>
>>>>>>
>>>>>> Do you think that any other important things I need to add in my
>>>>>> proposal?
>>>>>>
>>>>>
>>>>> It looks good. Hope you referred [1] on this.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> [1]  http://community.apache.org/gsoc.html#applying-for-gsoc
>>>>>
>>>>> Thanks.
>>>>>
>>>>>>
>>>>>> *T. Pranavan*
>>>>>> *BSc Eng Undergraduate| Department of Computer Science & Engineering
>>>>>> ,University of Moratuwa*
>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>> *Mobile| *0775136836
>>>>>>
>>>>>> On 25 March 2015 at 10:09, Pranavan Theivendiram <
>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>
>>>>>>> Hi Lahiru,
>>>>>>>
>>>>>>> Thanks for the prompt reply.
>>>>>>>
>>>>>>> *+1 for*
>>>>>>> *we can find the polynomial in each minute, considering 10 minutes
>>>>>>> of past data, we can change this if required.*
>>>>>>>
>>>>>>> If we are getting the health stats events uniformly for every
10
>>>>>>> seconds. my question is resolved. (I thought that we are getting
in a
>>>>>>> random manner).
>>>>>>>
>>>>>>> Can I add the things in my proposal regarding "Curve Fitting"?
>>>>>>>
>>>>>>> Do you think that any other important things I need to add in
my
>>>>>>> proposal?
>>>>>>>
>>>>>>> *T. Pranavan*
>>>>>>> *BSc Eng Undergraduate| Department of Computer Science &
Engineering
>>>>>>> ,University of Moratuwa*
>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>>> *Mobile| *0775136836
>>>>>>>
>>>>>>> On 25 March 2015 at 09:59, Lahiru Sandaruwan <lahirus@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> +1 for using Common math library, as we already using it.
>>>>>>>>
>>>>>>>>    - Currently we have calculating numbers per minute, so
it is
>>>>>>>>    not per specified number of events, but for a specified
time.
>>>>>>>>    - IMO we can find the polynomial in each minute, considering
10
>>>>>>>>    minutes of past data, we can change this if required.
>>>>>>>>    - Normally, health stat events are received in each 10
seconds.
>>>>>>>>    - Therefore we will have 60 ( = 6 * 10) events per 10
minutes.
>>>>>>>>
>>>>>>>> On Wed, Mar 25, 2015 at 9:11 AM, Pranavan Theivendiram <
>>>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>>>
>>>>>>>>> Hi Lahiru and Raj,
>>>>>>>>>
>>>>>>>>> I have gone through the curve fitting materials that
you have
>>>>>>>>> provided. I have understood the regression part and prediction
of degree
>>>>>>>>> polynomials. We can use apache common math libraries
to do this. I have a
>>>>>>>>> question below.
>>>>>>>>>
>>>>>>>>> How often we are going to find the polynomials?
>>>>>>>>>
>>>>>>>>> For example,
>>>>>>>>>
>>>>>>>>> If we have *3 samples* events during *10 minutes interval*,
then
>>>>>>>>> we can produce a second degree polynomial. (We will get
constants
>>>>>>>>> *a*,*b*, and *c*). If we are going to use this second
degree
>>>>>>>>> polynomial for next *60 minutes *(Suppose in this 60
minutes we
>>>>>>>>> are getting *200 samples *events).  As we can see our
*extrapolation
>>>>>>>>> <http://en.wikipedia.org/wiki/Extrapolation>* becomes
a severe
>>>>>>>>> problem in this particular example.
>>>>>>>>>
>>>>>>>>
>>>>>>>> What do you mean by " our *extrapolation
>>>>>>>> <http://en.wikipedia.org/wiki/Extrapolation>* becomes
a severe
>>>>>>>> problem"? Complexity of the calculation of predicted value?
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>>> I do not know how earlier versions handle this scenario.
>>>>>>>>>
>>>>>>>>> Can you tell that are we going produce new polynomials
for *a
>>>>>>>>> specified time* or *a specified number of events*?
>>>>>>>>>
>>>>>>>>> Please clear this issue.
>>>>>>>>>
>>>>>>>>> *T. Pranavan*
>>>>>>>>> *BSc Eng Undergraduate| Department of Computer Science
&
>>>>>>>>> Engineering ,University of Moratuwa*
>>>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt) Ltd.*
>>>>>>>>> *Mobile| *0775136836
>>>>>>>>>
>>>>>>>>> On 24 March 2015 at 09:02, Lahiru Sandaruwan <lahirus@wso2.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Pranavan,
>>>>>>>>>>
>>>>>>>>>> I'll have a look.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 24, 2015 at 8:52 AM, Pranavan Theivendiram
<
>>>>>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Lahiru and Raj,
>>>>>>>>>>>
>>>>>>>>>>> This is my draft proposal which has been done
up to 50%. I am
>>>>>>>>>>> going to add things about curve fitting as well.
Still many things need to
>>>>>>>>>>> be added. Please comment on my initial draft.
>>>>>>>>>>> I am expecting pros and cons regarding my proposal.
Please find
>>>>>>>>>>> the link for the initial draft below.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://docs.google.com/a/cse.mrt.ac.lk/document/d/1TzHYI2o9bIdZWj4qAffNmH5FTgeTOV4TuSsxlISlU8c/edit?usp=sharing
>>>>>>>>>>>
>>>>>>>>>>> *T. Pranavan*
>>>>>>>>>>> *BSc Eng Undergraduate| Department of Computer
Science &
>>>>>>>>>>> Engineering ,University of Moratuwa*
>>>>>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt)
Ltd.*
>>>>>>>>>>> *Mobile| *0775136836
>>>>>>>>>>>
>>>>>>>>>>> On 22 March 2015 at 23:36, Pranavan Theivendiram
<
>>>>>>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Lahiru for the prompt reply. I have
already started
>>>>>>>>>>>> working on the proposal. I will send it before
tomorrow.
>>>>>>>>>>>>
>>>>>>>>>>>> *T. Pranavan*
>>>>>>>>>>>> *BSc Eng Undergraduate| Department of Computer
Science &
>>>>>>>>>>>> Engineering ,University of Moratuwa*
>>>>>>>>>>>> *Intern Software Engineer**| WSO2 Lanka (Pvt)
Ltd.*
>>>>>>>>>>>> *Mobile| *0775136836
>>>>>>>>>>>>
>>>>>>>>>>>> On 22 March 2015 at 23:31, Lahiru Sandaruwan
<lahirus@wso2.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, You have found the information correctly
and on right
>>>>>>>>>>>>> track. It would be great if we can incorporate
CEP 4.0.0 changes as it
>>>>>>>>>>>>> would minimize our customized function.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would like to suggest you starting
the drafting of the
>>>>>>>>>>>>> proposal as you have some background
now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Mar 22, 2015 at 9:48 PM, Pranavan
Theivendiram <
>>>>>>>>>>>>> pranavan.11@cse.mrt.ac.lk> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> With the few days of research on
the project "Introducing
>>>>>>>>>>>>>> “curve fitting” for statistics
prediction algorithm of Autoscaler". I have
>>>>>>>>>>>>>> found
>>>>>>>>>>>>>> followings regarding the project
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    1. ClusterMonitor class passes
the calculated values from
>>>>>>>>>>>>>>    the CEP. This value passing happens
after
>>>>>>>>>>>>>>    org.apache.stratos.messaging.event
         happens.
>>>>>>>>>>>>>>    2. RuleTasksDelegator class calculates
the prediction
>>>>>>>>>>>>>>    values using the values(Ex gradient)
from the CEP
>>>>>>>>>>>>>>    3. The classes in the package
>>>>>>>>>>>>>>    org.apache.stratos.cep.extension
deals with the analysis of data in the
>>>>>>>>>>>>>>    real time and publish the relevant
summarized health stats, which are
>>>>>>>>>>>>>>    needed for the autoscaling.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So according to my understanding,
I need to modify above
>>>>>>>>>>>>>> mentioned classes, in order to implement
the curve fitting for statistics
>>>>>>>>>>>>>> prediction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Furthermore,for this project, we
can use the regression
>>>>>>>>>>>>>> implementation of CEP 4.0, which
is expected to be released on May 15th
>>>>>>>>>>>>>> according to WSO2 public jira. I
am planning to do a feasibility study for
>>>>>>>>>>>>>> the first two weeks of the project,
where we can explore other better
>>>>>>>>>>>>>> options as well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Am I on a right track? Please share
your thoughts on this.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://wso2.org/jira/browse/CEP
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> *T. Pranavan*
>>>>>>>>>>>>>> *BSc Eng Undergraduate| Department
of Computer Science &
>>>>>>>>>>>>>> Engineering ,University of Moratuwa*
>>>>>>>>>>>>>> *Intern Software Engineer**| WSO2
Lanka (Pvt) Ltd.*
>>>>>>>>>>>>>> *Mobile| *0775136836
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Lahiru Sandaruwan
>>>>>>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>>>>>>> Senior Software Engineer,
>>>>>>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>>>>
>>>>>>>>>>>>> phone: +94773325954
>>>>>>>>>>>>> email: lahirus@wso2.com blog:
>>>>>>>>>>>>> http://lahiruwrites.blogspot.com/
>>>>>>>>>>>>> linked-in:
>>>>>>>>>>>>> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> --
>>>>>>>>>> Lahiru Sandaruwan
>>>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>>>> Senior Software Engineer,
>>>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>
>>>>>>>>>> phone: +94773325954
>>>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>>>> linked-in:
>>>>>>>>>> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> --
>>>>>>>> Lahiru Sandaruwan
>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>> lean.enterprise.middleware
>>>>>>>>
>>>>>>>> phone: +94773325954
>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Lahiru Sandaruwan
>>>>> Committer and PMC member, Apache Stratos,
>>>>> Senior Software Engineer,
>>>>> WSO2 Inc., http://wso2.com
>>>>> lean.enterprise.middleware
>>>>>
>>>>> phone: +94773325954
>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Committer and PMC member, Apache Stratos,
>>> Senior Software Engineer,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> phone: +94773325954
>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>


-- 
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

phone: +94773325954
email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Mime
View raw message