Return-Path: X-Original-To: apmail-camel-dev-archive@www.apache.org Delivered-To: apmail-camel-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1D5D717BF3 for ; Sat, 4 Apr 2015 05:59:41 +0000 (UTC) Received: (qmail 16136 invoked by uid 500); 4 Apr 2015 05:59:40 -0000 Delivered-To: apmail-camel-dev-archive@camel.apache.org Received: (qmail 16091 invoked by uid 500); 4 Apr 2015 05:59:40 -0000 Mailing-List: contact dev-help@camel.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@camel.apache.org Delivered-To: mailing list dev@camel.apache.org Received: (qmail 16079 invoked by uid 99); 4 Apr 2015 05:59:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Apr 2015 05:59:40 +0000 X-ASF-Spam-Status: No, hits=0.6 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of claus.ibsen@gmail.com designates 209.85.213.173 as permitted sender) Received: from [209.85.213.173] (HELO mail-ig0-f173.google.com) (209.85.213.173) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Apr 2015 05:59:36 +0000 Received: by igblo3 with SMTP id lo3so19911321igb.0 for ; Fri, 03 Apr 2015 22:59:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=KMphJL+IJ9c12M5GgVKh9RYfDzAmj1SJHECsDQ0caa4=; b=ChRY2bbR9rWIF+NB3z2yG2pVXgisdSvH3Vs/cvfBwI+2Jbs5ZEV+JTP6MfCHbHduiH zJC8vkm75yOx6yPfU4n/D9xZkYopPXMendylhBp86aKCJVNIz9MH6uspz3Fwh9GhJcnk A5zZIiKdmliE30wWU/pg9a/21SOyZ3LU8LwHl+bME9S4N6Aepjli23fkylSmGp7YbDW2 5z3XbvEBUXdRgjKMPdrw8yCS9W7oNE1PjCFuSMcXbJjtdpLQdABwRkZRjcn+z1TzjB2A 7WMlDnQ7tUUVbYQgxXsEiRZIYXLS2AZZQXImfYGiMX1gOAwV3mpknkb1IvQWV30c34HQ qLpw== X-Received: by 10.50.111.10 with SMTP id ie10mr32452089igb.15.1428127155999; Fri, 03 Apr 2015 22:59:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.149.2 with HTTP; Fri, 3 Apr 2015 22:58:55 -0700 (PDT) In-Reply-To: <1428085983695-5765293.post@n5.nabble.com> References: <1428085983695-5765293.post@n5.nabble.com> From: Claus Ibsen Date: Sat, 4 Apr 2015 07:58:55 +0200 Message-ID: Subject: Re: camel git commit: Upgrade to Elasticsearch 1.4.4 To: dev Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org Hi I noticed that if you are low on free disk space, then you get WARNs - well I got 20gb free. 2015-04-04 07:59:06,391 [pdateTask][T#1]] WARN local - [Enigma] timed out waiting for all nodes to process published state [15] (timeout [30s], pending nodes: [[Night Rider][P7IzfV6oQkWsmz5JJzM-Ug][davsclaus.air][local[5]]{local=true}]) 2015-04-04 07:59:16,621 [anagement][T#2]] WARN decider - [Tempo] high disk watermark [10%] exceeded on [g56rnNQ8Qt-ckLr3_uCg_g][Tempo] free: 18.5gb[7.9%], shards will be relocated away from this node So I think the unit test needs to be adjusted so it wont require a lot of free GB to run. I haven't seen this on the older version, the test passed in reasonable time. Now it runs forever and I have to kill it. On Fri, Apr 3, 2015 at 8:33 PM, Babak Vahdat wrote: > Hi > > Yes I noticed that as well however the test will succeed & pass if you would > wait a bit longer, however with some thrown exceptions like: > > Running org.apache.camel.component.elasticsearch.ElasticsearchComponentTest > Exception in thread > "elasticsearch[Goldeneye][clusterService#updateTask][T#1]" > org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: > rejected execution (shutting down) on > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@25b99556 > at > org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62) > at > java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) > at > java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325) > at > java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530) > at org.elasticsearch.threadpool.ThreadPool.schedule(ThreadPool.java:245) > at > org.elasticsearch.cluster.service.InternalClusterService$1.run(InternalClusterService.java:230) > at > org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:153) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Exception in thread > "elasticsearch[Goldeneye][clusterService#updateTask][T#2]" > org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: > rejected execution (shutting down) on > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@47a0b02b > at > org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62) > at > java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) > at > java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325) > at > java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530) > at org.elasticsearch.threadpool.ThreadPool.schedule(ThreadPool.java:245) > at > org.elasticsearch.cluster.service.InternalClusterService$1.run(InternalClusterService.java:230) > at > org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:153) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Tests run: 16, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 80.196 sec > - in org.apache.camel.component.elastic > > On the other hand this has nothing to do with the ES upgrade and if you > would revert to the commit BEFORE the Lucene and the Elasticsearch upgrade > of yesterday and today then the behaviour would be exactly the same. Again, > it's not nice, but the test passes! > > That all said, maybe something else very important I've noticed which we > should be aware of: in general the upgrade of Lucene and ES inside the > parent POM should be always in sync, as ES internally depends on Lucene, if > you once do only upgrade Lucene version, then for camel-elasticsearch Maven > pulls in the Lucene version we've from the parent POM: > > https://github.com/apache/camel/blob/master/parent/pom.xml#L2161 > > As an example see how we've got a mix of Lucene versions 4.10.3 and 4.10.2 > below. This was the case yesterday as I only upgraded Lucene but not ES. > > ~/dev/workspace/camel/components/camel-elasticsearch>git reset --hard > d91b0c708e6dce761aef12868802e0d57d45d256 > HEAD is now at d91b0c7 Removed unnecessary method calls > ~/dev/workspace/camel/components/camel-elasticsearch>mvn dependency:tree > -Dverbose=true > [INFO] Scanning for projects... > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] Building Camel :: ElasticSearch 2.16-SNAPSHOT > [INFO] > ------------------------------------------------------------------------ > [INFO] > [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ > camel-elasticsearch --- > [INFO] org.apache.camel:camel-elasticsearch:bundle:2.16-SNAPSHOT > [INFO] +- org.apache.camel:camel-core:jar:2.16-SNAPSHOT:compile > [INFO] | +- org.slf4j:slf4j-api:jar:1.6.6:compile > [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:compile - omitted for > duplicate) > [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:compile - omitted for > duplicate) > [INFO] +- org.elasticsearch:elasticsearch:jar:1.4.2:compile > [INFO] | +- org.apache.lucene:lucene-core:jar:4.10.3:compile (version > managed from 4.10.2) > [INFO] | +- org.apache.lucene:lucene-analyzers-common:jar:4.10.3:compile > (version managed from 4.10.2) > [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-queries:jar:4.10.2:compile > [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-memory:jar:4.10.2:compile > [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-highlighter:jar:4.10.2:compile > [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | +- (org.apache.lucene:lucene-memory:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-queryparser:jar:4.10.3:compile > (version managed from 4.10.2) > [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | +- (org.apache.lucene:lucene-queries:jar:4.10.3:compile - > omitted for conflict with 4.10.2) > [INFO] | | \- (org.apache.lucene:lucene-sandbox:jar:4.10.3:compile - > omitted for conflict with 4.10.2) > [INFO] | +- org.apache.lucene:lucene-sandbox:jar:4.10.2:compile > [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-suggest:jar:4.10.2:compile > [INFO] | | +- > (org.apache.lucene:lucene-analyzers-common:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | +- (org.apache.lucene:lucene-misc:jar:4.10.2:compile - omitted > for duplicate) > [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-misc:jar:4.10.2:compile > [INFO] | | \- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-join:jar:4.10.2:compile > [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | \- (org.apache.lucene:lucene-grouping:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | +- org.apache.lucene:lucene-grouping:jar:4.10.2:compile > [INFO] | | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | | \- (org.apache.lucene:lucene-queries:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | \- org.apache.lucene:lucene-spatial:jar:4.10.2:compile > [INFO] | +- (org.apache.lucene:lucene-core:jar:4.10.3:compile - version > managed from 4.10.2; omitted for duplicate) > [INFO] | +- (org.apache.lucene:lucene-queries:jar:4.10.2:compile - > omitted for duplicate) > [INFO] | \- com.spatial4j:spatial4j:jar:0.4.1:compile > [INFO] +- org.apache.camel:camel-test:jar:2.16-SNAPSHOT:test > [INFO] | +- (org.apache.camel:camel-core:jar:2.16-SNAPSHOT:test - omitted > for duplicate) > [INFO] | +- junit:junit:jar:4.11:test > [INFO] | | \- org.hamcrest:hamcrest-core:jar:1.3:test > [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted for > duplicate) > [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted for > duplicate) > [INFO] +- org.apache.camel:camel-test-spring:jar:2.16-SNAPSHOT:test > [INFO] | +- (org.apache.camel:camel-test:jar:2.16-SNAPSHOT:test - omitted > for duplicate) > [INFO] | +- org.apache.camel:camel-spring:jar:2.16-SNAPSHOT:test > [INFO] | | +- (org.apache.camel:camel-core:jar:2.16-SNAPSHOT:test - > omitted for duplicate) > [INFO] | | +- (org.springframework:spring-context:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | +- (org.springframework:spring-aop:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | +- (org.springframework:spring-tx:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted for > duplicate) > [INFO] | | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted for > duplicate) > [INFO] | +- (org.springframework:spring-test:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | +- org.springframework:spring-context:jar:4.1.6.RELEASE:test > [INFO] | | +- (org.springframework:spring-aop:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | +- (org.springframework:spring-beans:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | +- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | \- > (org.springframework:spring-expression:jar:4.1.6.RELEASE:test - omitted for > duplicate) > [INFO] | +- org.springframework:spring-beans:jar:4.1.6.RELEASE:test > [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | +- org.springframework:spring-expression:jar:4.1.6.RELEASE:test > [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | +- org.springframework:spring-aop:jar:4.1.6.RELEASE:test > [INFO] | | +- aopalliance:aopalliance:jar:1.0:test > [INFO] | | +- (org.springframework:spring-beans:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | +- org.springframework:spring-tx:jar:4.1.6.RELEASE:test > [INFO] | | +- (org.springframework:spring-beans:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] | +- org.springframework:spring-core:jar:4.1.6.RELEASE:test > [INFO] | | \- commons-logging:commons-logging:jar:1.1.3:test (version > managed from 1.2) > [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:test - omitted for > duplicate) > [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:test - omitted for > duplicate) > [INFO] +- org.springframework:spring-test:jar:4.1.6.RELEASE:test > [INFO] | \- (org.springframework:spring-core:jar:4.1.6.RELEASE:test - > omitted for duplicate) > [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.10:test > [INFO] | +- (org.slf4j:slf4j-api:jar:1.6.6:test - version managed from > 1.7.10; omitted for duplicate) > [INFO] | \- log4j:log4j:jar:1.2.17:test > [INFO] +- org.apache.camel:apt:jar:2.16-SNAPSHOT:provided > [INFO] | +- (org.slf4j:slf4j-api:jar:1.6.6:provided - version managed from > 1.7.10; omitted for duplicate) > [INFO] | +- org.apache.camel:spi-annotations:jar:2.16-SNAPSHOT:provided > [INFO] | | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:provided - omitted > for duplicate) > [INFO] | | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:provided - omitted > for duplicate) > [INFO] | +- (com.sun.xml.bind:jaxb-core:jar:2.2.11:provided - omitted for > duplicate) > [INFO] | \- (com.sun.xml.bind:jaxb-impl:jar:2.2.11:provided - omitted for > duplicate) > [INFO] +- com.sun.xml.bind:jaxb-core:jar:2.2.11:compile > [INFO] \- com.sun.xml.bind:jaxb-impl:jar:2.2.11:compile > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 4.112 s > [INFO] Finished at: 2015-04-03T20:15:12+02:00 > [INFO] Final Memory: 29M/1963M > [INFO] > ------------------------------------------------------------------------ > > Then as a result of this the camel-elasticsearch tests failed today on CI: > > https://builds.apache.org/view/All/job/Camel.trunk.fulltest/2249/ > > Babak > > > Claus Ibsen-2 wrote >> Hi >> >> Does the unit test of camel-elastichsearch hang for you? I tried >> merging in a PR about another ES fix (a small one) and then it hangs >> for me. >> >> I didnt look into it further, but just wander of the 1.4.2 -> 1.4.4 >> worked fine for you / others who have build latest code? >> >> On Fri, Apr 3, 2015 at 3:29 PM, < > >> bvahdat@ > >> > wrote: >>> Repository: camel >>> Updated Branches: >>> refs/heads/master d91b0c708 -> 9ff05dc66 >>> >>> >>> Upgrade to Elasticsearch 1.4.4 >>> >>> Project: http://git-wip-us.apache.org/repos/asf/camel/repo >>> Commit: http://git-wip-us.apache.org/repos/asf/camel/commit/9ff05dc6 >>> Tree: http://git-wip-us.apache.org/repos/asf/camel/tree/9ff05dc6 >>> Diff: http://git-wip-us.apache.org/repos/asf/camel/diff/9ff05dc6 >>> >>> Branch: refs/heads/master >>> Commit: 9ff05dc6619335bec4c0ea6972a9d3da6f1b04c8 >>> Parents: d91b0c7 >>> Author: Babak Vahdat < > >> bvahdat@ > >> > >>> Authored: Fri Apr 3 15:29:00 2015 +0200 >>> Committer: Babak Vahdat < > >> bvahdat@ > >> > >>> Committed: Fri Apr 3 15:29:00 2015 +0200 >>> >>> ---------------------------------------------------------------------- >>> .../component/elasticsearch/ElasticsearchComponentTest.java | 2 +- >>> parent/pom.xml | 4 >>> ++-- >>> 2 files changed, 3 insertions(+), 3 deletions(-) >>> ---------------------------------------------------------------------- >>> >>> >>> http://git-wip-us.apache.org/repos/asf/camel/blob/9ff05dc6/components/camel-elasticsearch/src/test/java/org/apache/camel/component/elasticsearch/ElasticsearchComponentTest.java >>> ---------------------------------------------------------------------- >>> diff --git >>> a/components/camel-elasticsearch/src/test/java/org/apache/camel/component/elasticsearch/ElasticsearchComponentTest.java >>> b/components/camel-elasticsearch/src/test/java/org/apache/camel/component/elasticsearch/ElasticsearchComponentTest.java >>> index 55d8998..acfa21e 100644 >>> --- >>> a/components/camel-elasticsearch/src/test/java/org/apache/camel/component/elasticsearch/ElasticsearchComponentTest.java >>> +++ >>> b/components/camel-elasticsearch/src/test/java/org/apache/camel/component/elasticsearch/ElasticsearchComponentTest.java >>> @@ -68,7 +68,7 @@ public class ElasticsearchComponentTest extends >>> CamelTestSupport { >>> documents.add(document1); >>> documents.add(document2); >>> >>> - List indexIds = template.requestBody("direct:bulk_index", >>> documents, List.class); >>> + List indexIds = template.requestBody("direct:bulk_index", >>> documents, List.class); >>> assertNotNull("indexIds should be set", indexIds); >>> assertCollectionSize("Indexed documents should match the size of >>> documents", indexIds, documents.size()); >>> } >>> >>> http://git-wip-us.apache.org/repos/asf/camel/blob/9ff05dc6/parent/pom.xml >>> ---------------------------------------------------------------------- >>> diff --git a/parent/pom.xml b/parent/pom.xml >>> index 8956c67..5bc200b2 100644 >>> --- a/parent/pom.xml >>> +++ b/parent/pom.xml >>> @@ -142,8 +142,8 @@ >>> >> >> 2.9.1 >> >>> >> >> 2.1.5 >> >>> >> >> 2.1.5_1 >> >>> - >> >> 1.4.2_1 >> >>> - >> >> 1.4.2 >> >>> + >> >> 1.4.4_1 >> >>> + >> >> 1.4.4 >> >>> >> >> 1.0.1 >> >>> >> >> 1.47.0 >> >>> >> >> 1.2.1 >> >>> >> >> >> >> -- >> Claus Ibsen >> ----------------- >> Red Hat, Inc. >> Email: > >> cibsen@ > >> Twitter: davsclaus >> Blog: http://davsclaus.com >> Author of Camel in Action: http://www.manning.com/ibsen >> hawtio: http://hawt.io/ >> fabric8: http://fabric8.io/ > > > > > > -- > View this message in context: http://camel.465427.n5.nabble.com/Re-camel-git-commit-Upgrade-to-Elasticsearch-1-4-4-tp5765284p5765293.html > Sent from the Camel Development mailing list archive at Nabble.com. -- Claus Ibsen ----------------- Red Hat, Inc. Email: cibsen@redhat.com Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen hawtio: http://hawt.io/ fabric8: http://fabric8.io/