Mailing-List: contact dev-help@kylin.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kylin.apache.org
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii
From: Santosh Akhilesh <santoshakhilesh@gmail.com>
Mime-Version: 1.0 (1.0)
Subject: Few Questions about Kylin Ability 
Message-Id: <BC10524F-9229-47F7-BB13-0A01AADF7668@gmail.com>
Date: Sat, 2 Jul 2016 17:55:20 +0800
To: dev@kylin.apache.org
Cc: Santoshakhilesh <santosh.akhilesh@huawei.com>
archived-at: Sat, 02 Jul 2016 09:55:28 -0000

Hi All ,
Last year I had done a PoC for one of our products using Kylin. Our distribu=
ted architecture journey was on hold for some time but now we are back again=
 to rearchitect our system to distributed. I am writing this mail to underst=
and how and whether Kylin can fit in to our requirements.
Let me give background of our requirement.
Ours is a network performance management solution which needs to handle foll=
owing scenes.

1. Collect data from network elements in granularity between 30 sec to 5 min=
ute period. Every period we collect around 150Million KPIs Which are distrib=
uted across different service type. The service types are model driven and c=
an change over period of time.
2. Data which we collect needs to available for Adhoc and OLAP type query AS=
AP. For example data collected between 10:00 and 10:05 for 5 mins period sho=
uld be available for reports to fire query by 10:06. Query will involve join=
ing performance data with inventory data and also have filters like query da=
ta for Area =3D Area1 and we also need sort by KPI or property of inventory w=
ith order by Clause
3. We also need OLAP type query like group by area , province , country etc.=
.. and needs to apply sum , max , min , avg aggregator. We also need to gene=
rate Top talkers report which means we need Top N function.
4. There will be background machine learning jobs which need to scan raw and=
 aggregated data.
5. We would be generating around 5-10 TB of data every day and In future may=
 be more.
Now my questions are these. We need to retain data for several days and mont=
hs based on aggregation period.
6. Adhoc and OLAP query from report should take < 2 seconds.
So my questions are;

1. Which of the use cases Kylin can support?
2. How long cube building takes and how does it handle the data which will b=
e appended every 30 sec or 5 minutes.
3. Can Kylin support both Adhoc query and OLAP query ?

I have several other questions but I would like to initiate the discussion w=
ith these.=20
We plan to start a test next week with Kylin I am just setting up a cluster n=
ow. We don't plan to use cloud era or Horton work sandbox as our company has=
 its own sandbox.

Appreciate response from Kylin experts.

Regards
Santosh=20


Sent from my iPhone=