ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taras Ledkov <tled...@gridgain.com>
Subject Re: Hadoop + IGFS + Hive+Tez
Date Fri, 23 Jun 2017 09:38:18 GMT
Hi,

I don't catch how you configured hadoop Job and where Ignite instances 
are started (on all hadoop hosts?).

Shared memory is default and preferred type of connection to Ignite.
In this case Ignite nodes must be started on hadoop nodes.

If you started  Ignite on either hosts IGFS URL must be specified with 
the host/port.

Please take a look:
https://apacheignite.readme.io/v1.2/docs/file-system#section-file-system-uri


On 21.06.2017 18:28, Mimmo Celano wrote:
> Hi, thanks for reply.
>
> Ignite configuration is a default configuration in 
> config/hadoop/default-configuration.xml i don't change anything. 
> Hadoop configuration is a cluster o 3 node, it work properly withouth 
> ignite. The problem is in a connection between the client and the 2 
> server. I don't change anything in ignite file configuration, the only 
> things i do is the code who i posted.
> I agree with you that is a configuration problem but i don't know how 
> to configure it. The 2 server node are slave in hadoop configuration, 
> but i don't think it's a problem. I don't find another ignite logs 
> other that posted... I don't use tez. You think that i use another 
> configuration for ignite instead the default for hadoop? I want just 
> save all word of wordcount processing in a grid cache between 2 node 
> server. There's no log of the task because the application crash 
> before task......
>
> The node are connected via lan. Thanks
>
> 2017-06-21 16:59 GMT+02:00 Taras Ledkov <tledkov@gridgain.com 
> <mailto:tledkov@gridgain.com>>:
>
>     Hi,
>
>     Please provide:
>     - ignite configuration;
>     - ignite logs;
>     - hadoop configurations;
>     - logs of the task.
>
>     I don't have tez experience but the cause of the problem looks
>     like Ignite is not configured properly.
>
>
>     On 21.06.2017 8:39, ishan-jain wrote:
>>     Hi all, I have configured hive to run over IGFS for hadoop file system. I am
>>     trying to install tez on it but am not able to configure tez-site.xml and
>>     bashrc file properly.
>>     I have downloaded a binary tez bin to my system (0.8.4)
>>     I Put the tez tar ball in hdfs dire /user/tez
>>     I pointed my tez.lib.uri to this path
>>     I have configured mapred-site.xml to use yarn-tez
>>     I have hive already configured to run over igfs.
>>     But when i execute the mapred task it says
>>     Failed to connect to shared memory endpoint .
>>     I am using TCP as ipc endpoint configuration and have enabled ipc endpoint
>>     property.
>>
>>
>>
>>     --
>>     View this message in context:http://apache-ignite-users.70518.x6.nabble.com/Hadoop-IGFS-Hive-Tez-tp14012.html
>>     <http://apache-ignite-users.70518.x6.nabble.com/Hadoop-IGFS-Hive-Tez-tp14012.html>Sent
>>     from the Apache Ignite Users mailing list archive at Nabble.com.
>
>     -- 
>     Taras Ledkov
>     Mail-To:tledkov@gridgain.com <mailto:tledkov@gridgain.com>
>
>

-- 
Taras Ledkov
Mail-To: tledkov@gridgain.com


Mime
View raw message