falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mpopa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1503) cluster mirroring
Date Thu, 01 Oct 2015 13:54:27 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939853#comment-14939853
] 

mpopa commented on FALCON-1503:
-------------------------------

seems the hive2 and thrift are only available starting at version 4.0:

https://www-01.ibm.com/support/knowledgecenter/SSPT3X_4.1.0/com.ibm.swg.im.infosphere.biginsights.install.doc/doc/inst_otherports.html

I've tried however the only two combinations available to me:

1.
sourceMetastoreUri=thrift://bivm.ibm.com:9999
sourceHiveServer2Uri=hive2://bivm.ibm.com:10000

yields an OutOfMemory exception

2.
sourceMetastoreUri=thrift://bivm.ibm.com:10000
sourceHiveServer2Uri=hive2://bivm.ibm.com:9999


after submit nothing happens, it just hangs there. I can't see anything in the logs, the tables
don't get copied, the control never returns...

So it looks have to go the manual way, export the hive tables, distcp them to hortonworks
and re-import them.

thank you


> cluster mirroring 
> ------------------
>
>                 Key: FALCON-1503
>                 URL: https://issues.apache.org/jira/browse/FALCON-1503
>             Project: Falcon
>          Issue Type: Bug
>    Affects Versions: 0.6.1
>         Environment: Hortonworks 2.3 sandbox 
> BigInsights 3.0.2 sandbox
>            Reporter: mpopa
>
> Hi,
> I've been trying to setup mirroring between two clusters.
> One runs BigInsights 3.0.2, the other Hortonworks 2.3.
> I'm running Falcon on the Hortonworks cluster (which is actually just a sandbox).
> I have no trouble defining the target cluster (on the localhost), but cannot define the
source
> cluster (on the remote BigInsights install).
> The problem is with the "execute" interface; I've tried a series of ports starting with
9000
> which is the default namenode, 9001, 50030.
> <interface type='execute' endpoint='bivm.ibm.com:50030' version='2.2.0'/>
> (I do have a DNS entry for bivm.ibm.com)
> The error that I get is:
> org.apache.falcon.entity.parser.ValidationException: Invalid Execute server or port:
bivm.ibm.com:50030
> It seems like this version of BIgInsights does not come with Yarn, and from what I see,
the execute interface tries to connect to yarn's port.
> Any insight (pun intended) is greatly appreciated.
> thank you in advance
> mihai popa



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message