ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evgeniy Stanilovskiy" <estanilovs...@gridgain.com>
Subject Re: HDP, Hive + Ignite
Date Mon, 15 May 2017 13:10:37 GMT
Ivan, what kind of tests did u run ? plz show SQL requests ?
1-node tests for distributed computing looks like wierd

> My observations (on very simplified 1-node environment) show that  
> Ignite-MR ~10% faster than TEZ under equal >conditions.
>
> On Mon, May 15, 2017 at 1:52 PM, Ivan V. <iveselovskiy@gridgain.com>  
> wrote:
>> Hi, Alena, regarding "1) Ignite MR works slower than Hive on TEZ, but  
>> faster than Hive on MR." -- as far as >>I remember, you have observed  
>> incorrect results with Ignite MR, and we didn't find the reason, just  
>> abandoned that. >>Performance measurements don't have much sense until  
>> we have correct query results. So, I would say that there we >>just  
>> don't have results we can trust.
>>
>> Regarding ""Out of memory: Kill process" -- this means that Ignite node  
>> process requested so much memory that OS >>failed to give. This may be  
>> investigated further -- all the memory limits set for Ignite node  
>> should be checked and >>compared to the real memory physically  
>> available on the host. Do I correctly understand that Ignite MR was not  
>> used >>in that experiment?
>>
>> On Mon, May 15, 2017 at 9:32 AM, Alena Melnikova <aloha@74.ru> wrote:
>>> Hi Ivan,
>>>
>>> You're right. In kernel log there is message: "Out of memory: Kill  
>>> process
>>> 19988 (java)"
>>>
>>> Let me sum up, please, correct me if I'm wrong.
>>> If we use Hive + Tez we don't need Hadoop Accelerator because:
>>> 1) Ignite MR works slower than Hive on TEZ, but faster than Hive on MR.
>>> 2) TEZ+HDFS and TEZ+IGFS work at the same speed. Although TEZ+IGFS can  
>>> be
>>> faster in queries with intensive I/O (need to test).
>>>
>>> Many thanks for your patience and prompt help.
>>> I'm going to try Ignite + Spark, I'll open new topic)
>>>
>>>
>>>
>>> --
>>> View this message in context:  
>>> http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12838.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
    • Unnamed multipart/related (inline, None, 0 bytes)
View raw message