apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akshay S Harale <akshay.har...@synerzip.com>
Subject Regarding issue APEXMALHAR-2122
Date Tue, 05 Jul 2016 11:10:48 GMT
Hi,

I have created the pull request for issue APEXMALHAR-2122.

The reason behind this is to support the latest version of elastic search.
Also updated the AbstractElasticSearchOutputOperator.processBatch() api to
process the batch only if its not empty.
I have created the pull request for the same : apex-malhar 325
<https://github.com/apache/apex-malhar/pull/325>

I created the pull request after running the example in local mode(junit
test case).

Now I am trying to run application using APEX cli. First got the exception
about the googles guava library:




*java.lang.NoSuchMethodError:
com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor;
      at
org.elasticsearch.threadpool.ThreadPool.<clinit>(ThreadPool.java:190)
  at
org.elasticsearch.client.transport.TransportClient$Builder.build(TransportClient.java:131)
      at
com.datatorrent.contrib.elasticsearch.ElasticSearchConnectable.connect(ElasticSearchConnectable.java:123)*

The reason behind above exception is : elastic search java client requires
guava 18.0 and the Hadoop 2.2 comes with guava 11.0.2.
In our application package (apa file) we have guava 18.0 jar. But i think
while launching application apex gives preference to the guava 11.0.2.

So we updated the apex-engine.jar and changed guava version to 18.0
manually which is not good practice :)
After this we are able to see that application is running in hadoop cluster
but in log there is an exception :







*2016-06-28 15:54:28,721 ERROR com.datatorrent.netlet.AbstractClient:
Exception in event loop {id=ProcessWideEventLoop, head=1, tail=1,
capacity=1024} java.net.ConnectException: Connection refused         at
sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)         at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
    at
com.datatorrent.netlet.DefaultEventLoop.handleSelectedKey(DefaultEventLoop.java:371)
      at
com.datatorrent.netlet.OptimizedEventLoop$SelectedSelectionKeySet.forEach(OptimizedEventLoop.java:59)
      at
com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:192)*

Also tried using maven shade plugin to include guava classes in output jar
but that also not working.

Regards,
Akshay

-- 
This e-mail, including any attached files, may contain confidential and 
privileged information for the sole use of the intended recipient. Any 
review, use, distribution, or disclosure by others is strictly prohibited. 
If you are not the intended recipient (or authorized to receive information 
for the intended recipient), please contact the sender by reply e-mail and 
delete all copies of this message.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message