kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Ramón <a.ramonporto...@gmail.com>
Subject Re: Kylin Dependencies
Date Thu, 03 Nov 2016 08:25:38 GMT
yes,  I saw 1.200 process
(this can be OK for All in One Docker , for developing or training process)

I showed this to a "Docker Captain" .... and (was funny)

(
Docker is some radical , also:
Understand that there is only 1 user process by container , with PID = 1
And only monitorice/check PID = 1
If PID 1 is ok  -->  container is OK
if PID 1 is no OK, --> auto restart container

Which process will be PID 1?  The process that you put CMD in dockerfile
(obviously only can have one CMD per dockerfile)
)

I will try to build a "MIni Kylin", ... I will info you about my progress ¡¡

2016-11-03 6:49 GMT+01:00 ShaoFeng Shi <shaofengshi@apache.org>:

> are there so many running processes in the kylin docker image? Although
> Kylin relies on Hive/YARN/HDFS/HBase, they are all client jars instead of
> running services; To minimal the docker image, some components can be
> removed like openssh-server openssh-clients snappy snappy-devel
> hadoop-native (from 1.5.4 kylin doesn't use compression by default)
>
> 2016-11-02 23:15 GMT+08:00 Alberto Ramón <a.ramonportoles@gmail.com>:
>
>> yes, I tested (and use) this and other previous version
>>
>> BUT the image :
>>   -  more than > 1000 process
>>   -  more than > 3GB
>>
>> This is OK (very OK) for testing / develop / PoC
>>
>> But for production (docker recomendations):
>>   -  Ideally 1 process (5-10 can be acceptable)
>>   -  < 100 MB (200 - 300 MB can be acceptable)
>>
>> The target is: Create Kylin docker (minimal) *with out install *Hive,
>> YARN, HDFS, or HBase ....
>>
>> 2016-11-02 15:52 GMT+01:00 Billy(Yiming) Liu <liuyiming.vip@gmail.com>:
>>
>>> Here is a quick start for running Kylin on docker,
>>> https://github.com/kyligence/kylin-docker
>>>
>>> From the docker file, you could find the kylin dependencies.
>>>
>>> 2016-11-02 22:46 GMT+08:00 Alberto Ramón <a.ramonportoles@gmail.com>:
>>>
>>>> With configs ....... I can try it (Will be an interesting exercise for
>>>> me)
>>>> But libraries, ...
>>>>    These libraries can be static compiled on Kylin?
>>>>     Any Idea / solution about how to solve all dependecies with out
>>>> install HDFS, Yarn, Hive, HBase in this minimal Linux... ?
>>>>
>>>> the idea is make "minimal linux + Kylin" "to docker it"
>>>> (The result must be few MB, < 150 MB)
>>>>
>>>>
>>>> 2016-11-02 14:20 GMT+01:00 Li Yang <liyang@apache.org>:
>>>>
>>>>> Kylin needs Hadoop client library and configs, including hdfs, yarn,
>>>>> hive, hbase.
>>>>>
>>>>> On Sun, Oct 30, 2016 at 1:42 AM, Alberto Ramón <
>>>>> a.ramonportoles@gmail.com> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Target:
>>>>>>   All Kylin docker are VERY heavy !! (GB and hundred of process)
--->
>>>>>> That Is Good for Develop / testing , but BAD Idea for production
>>>>>> I'm trying to install Kylin on minimal linux, ideally Alpine or
>>>>>> similar
>>>>>>
>>>>>> I have:
>>>>>> -  a clean install of linux (minimal Centos for example) , without
>>>>>> Hadoop, and and install Kylin from binary
>>>>>>  - use remote HBase & Hive
>>>>>>
>>>>>>
>>>>>> Which dependencies of Kylin I Will need on my Centos / Alpine?
>>>>>>
>>>>>> BR, Alb
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> With Warm regards
>>>
>>> Yiming Liu (刘一鸣)
>>>
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>

Mime
View raw message