ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alena Melnikova <al...@74.ru>
Subject Re: HDP, Hive + Ignite
Date Wed, 26 Apr 2017 09:11:44 GMT
Hi Ivan,

> Run Ignite nodes with -v option and see console logs of the nodes

Unfortunately, I can’t run again, it fails. I tried to run on one node with
–v option. Logs in attachment.

> use IGFS w/o Ignite Map-Reduce, Ignite Map-Reduce w/o IGFS

Sorry, I don’t understand, I'm newbie. Could you explain it more detailed?

> set  -Dignite.job.shared.classloader=false

Where I can set it? I don’t know and don't use Java.

> Simplest solution is to give Ignite more memory

How can I know how much memory Ignite uses?
Currently every cluster node has 8 CPU and 16Gb RAM.
I tried to add these parameters to default-config.xml:
/<property name="memoryMode" value="OFFHEAP_TIERED"/>
<property name="offHeapMaxMemory" value="#{4 * 1024L * 1024L * 1024L}"/>/
But then OOM errors arose, so I removed it from config.

> In code you can create bean objects  (like new FileSystemConfiguration() )
> , then take the values  via getters

To my shame I don’t know Java. Although I guess that I'm not the only one
who does not know Java, but is attracted by Ignite. So I think
default-config.xml with all possible parameters would be really useful.

> what do you mean "cluster is HA"?

High Available cluster. I think if you explain “use IGFS w/o Ignite
Map-Reduce, Ignite Map-Reduce w/o IGFS” a little bit more I stop generating
silly questions.

> This question does not have simple answer: what is done with these data? 

Suppose this file is Hive table (40 mln records, 30 columns, ORC 500MB, csv
9Gb), and we want to calculate some aggregation (SUM, AVG) group by 3-5
columns. How Ignite will utilize memory for that?

> 1 Ignite node per host is okay, if the node can utilize all the host
> resources (memory & cpu) 

How can I find out that ignite node utilizes all the host resources?

Thanks for your help!

View this message in context: http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12257.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message