falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mpopa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1503) cluster mirroring
Date Wed, 30 Sep 2015 19:23:05 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938328#comment-14938328
] 

mpopa commented on FALCON-1503:
-------------------------------

so I found the proper cli, but, as expected, the unavailable ports on the BigInsights side
are spoiling the party:

[falcon@sandbox ~]$ falcon recipe -name hive-disaster-recovery -operation HIVE_DISASTER_RECOVERY
Recipe processing failed com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException:
Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException:
Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2263)
	at com.google.common.cache.LocalCache.get(LocalCache.java:4000)
	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4789)
	at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:227)
	at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:202)
	at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
	at org.apache.hive.hcatalog.api.HCatClientHMSImpl.initialize(HCatClientHMSImpl.java:823)
	at org.apache.hive.hcatalog.api.HCatClient.create(HCatClient.java:71)
	at org.apache.falcon.recipe.HiveReplicationRecipeTool.getHiveMetaStoreClient(HiveReplicationRecipeTool.java:140)
	at org.apache.falcon.recipe.HiveReplicationRecipeTool.validate(HiveReplicationRecipeTool.java:52)
	at org.apache.falcon.recipe.RecipeTool.run(RecipeTool.java:89)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.falcon.recipe.RecipeTool.main(RecipeTool.java:64)
	at org.apache.falcon.client.FalconClient.submitRecipe(FalconClient.java:976)
	at org.apache.falcon.cli.FalconCLI.recipeCommand(FalconCLI.java:988)
	at org.apache.falcon.cli.FalconCLI.run(FalconCLI.java:204)
	at org.apache.falcon.cli.FalconCLI.main(FalconCLI.java:148)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:118)
	at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:230)
	at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
	at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4792)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)
	... 16 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
	... 26 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided.
Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused


Looking at the list of ports available on BIgInsights here:
https://www-01.ibm.com/support/knowledgecenter/SSPT3X_3.0.0/com.ibm.swg.im.infosphere.biginsights.install.doc/doc/c0057855.html

it does not seem they expose the hive meta store....
any idea?

thank you

> cluster mirroring 
> ------------------
>
>                 Key: FALCON-1503
>                 URL: https://issues.apache.org/jira/browse/FALCON-1503
>             Project: Falcon
>          Issue Type: Bug
>    Affects Versions: 0.6.1
>         Environment: Hortonworks 2.3 sandbox 
> BigInsights 3.0.2 sandbox
>            Reporter: mpopa
>
> Hi,
> I've been trying to setup mirroring between two clusters.
> One runs BigInsights 3.0.2, the other Hortonworks 2.3.
> I'm running Falcon on the Hortonworks cluster (which is actually just a sandbox).
> I have no trouble defining the target cluster (on the localhost), but cannot define the
source
> cluster (on the remote BigInsights install).
> The problem is with the "execute" interface; I've tried a series of ports starting with
9000
> which is the default namenode, 9001, 50030.
> <interface type='execute' endpoint='bivm.ibm.com:50030' version='2.2.0'/>
> (I do have a DNS entry for bivm.ibm.com)
> The error that I get is:
> org.apache.falcon.entity.parser.ValidationException: Invalid Execute server or port:
bivm.ibm.com:50030
> It seems like this version of BIgInsights does not come with Yarn, and from what I see,
the execute interface tries to connect to yarn's port.
> Any insight (pun intended) is greatly appreciated.
> thank you in advance
> mihai popa



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message