ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan V." <iveselovs...@gridgain.com>
Subject Re: HDP, Hive + Ignite
Date Mon, 15 May 2017 13:07:26 GMT
My observations (on very simplified 1-node environment) show that Ignite-MR
~10% faster than TEZ under equal conditions.

On Mon, May 15, 2017 at 1:52 PM, Ivan V. <iveselovskiy@gridgain.com> wrote:

> Hi, Alena, regarding "1) Ignite MR works slower than Hive on TEZ, but
> faster than Hive on MR." -- as far as I remember, you have observed
> incorrect results with Ignite MR, and we didn't find the reason, just
> abandoned that. Performance measurements don't have much sense until we
> have correct query results. So, I would say that there we just don't have
> results we can trust.
>
> Regarding ""Out of memory: Kill process" -- this means that Ignite node
> process requested so much memory that OS failed to give. This may be
> investigated further -- all the memory limits set for Ignite node should be
> checked and compared to the real memory physically available on the host.
> Do I correctly understand that Ignite MR was not used in that experiment?
>
> On Mon, May 15, 2017 at 9:32 AM, Alena Melnikova <aloha@74.ru> wrote:
>
>> Hi Ivan,
>>
>> You're right. In kernel log there is message: "Out of memory: Kill process
>> 19988 (java)"
>>
>> Let me sum up, please, correct me if I'm wrong.
>> If we use Hive + Tez we don't need Hadoop Accelerator because:
>> 1) Ignite MR works slower than Hive on TEZ, but faster than Hive on MR.
>> 2) TEZ+HDFS and TEZ+IGFS work at the same speed. Although TEZ+IGFS can be
>> faster in queries with intensive I/O (need to test).
>>
>> Many thanks for your patience and prompt help.
>> I'm going to try Ignite + Spark, I'll open new topic)
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/HDP-Hive-Ignite-tp12195p12838.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Mime
View raw message