kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: S3 as kylin storage.
Date Thu, 21 Nov 2019 17:47:21 GMT
Agree with Yaqian; S3 is object storage, while HBase is a fast K-V storage.
S3 is good for big objects like files, video, images, etc, but not suitable
for small objects like a number, a string, etc. You can use S3 as the DFS
for Kylin (like HDFS), and also can use S3 as HBase's underlying storage.
These are all supported, many AWS users are running Kylin on HBase+S3.

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org




Yaqian Zhang <Yaqian_Zhang@126.com> 于2019年11月21日周四 上午10:43写道:

> Hi:
>
> I think your understanding is correct,S3 can’t instead of Hbase.
>
> You should have read this article.
>
> http://kylin.apache.org/docs/install/kylin_aws_emr.html <
> http://kylin.apache.org/docs/install/kylin_aws_emr.html>.
>
> This introduce how to use S3 as kylin.env.hdfs-working-dir, andassume
> HBase is also on S3.
>
> > 在 2019年11月19日,20:11,Tanmay Movva <tanmay.movva@razorpay.com.INVALID>
写道:
> >
> > Hello,
> >
> > We have decided to use kylin to build our data warehouse and are going to
> > setup kylin on a cluster soon. We would like to know if S3 can be used
> as a
> > storage for apache kylin instead of Hbase as both are key-value stores
> and
> > query directly on S3.
> >
> > From what I understand I can only use hbase.storage_mode = s3 but not S3
> as
> > a direct storage for kylin cubes. Please help me out here, Thanks.
> >
> > --
> > Regards,
> > Tanmay Krishna Movva
> > Razorpay
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message