flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wahrmann, Helmut" <helmut.wahrm...@rsa.com>
Subject RE: Merge of patch in Flume-3021?
Date Wed, 29 Aug 2018 09:30:22 GMT
Hi Mike,

Did you have a chance to look at that?


best regards,
Helmut

-----Original Message-----
From: Wahrmann, Helmut 
Sent: Dienstag, 17. April 2018 18:53
To: 'dev@flume.apache.org'
Subject: RE: Merge of patch in Flume-3021?

Hi Mike,

Did you have a chance to look into this?

thanks,

Helmut
 

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org]
Sent: Montag, 26. März 2018 21:46
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

I haven't figured this out yet but I'll look into it this week.

Mike

On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com>
wrote:

> Hi Mike,
>
> With the help of Ferenc, I got rid of the initial errors.
>
> Only one is remaining now:
>
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 7.724 s - in
> org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineSolrSink
> [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> Time
> elapsed: 1.797 s  <<< ERROR!
> java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
>
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> ╗ IllformedLocale
> [INFO]
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] 
> [INFO]
> ----------------------------------------------------------------------
> --
> [INFO] BUILD FAILURE
>
>
> The LuceneTestCase is extended by SolrTestCaseJ4.
> No idea, what I could do against this.
>
> best regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 19. März 2018 05:12
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Nice! Thanks Ferenc. Answered better than I could have done. :)
>
> Sorry, I meant to send this last week but I just found it in my drafts.
>
> Helmut, please let us know if you need more help with this.
>
> Mike
>
> On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fszabo@cloudera.com> wrote:
>
> > the createJetty method of a Test class became final in the new 
> > versions of solr.
> > the kite sdk test-jar has to be removed because it depends on a 
> > different incompatible version of solr
> >
> >     <dependency>
> >       <groupId>org.kitesdk</groupId>
> >       <artifactId>kite-morphlines-solr-core</artifactId>
> >       <version>${kite.version}</version>
> >       <type>test-jar</type>
> >       <scope>test</scope>
> >     </dependency>
> >
> > TestEnvironment.java has to be removed as well because it depends on 
> > the incompatible dependency
> >
> > then we need this class:
> > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > morphline/solr/TestEmbeddedSolrServer.java
> > I believe it is ok to have a copy of this because it is part of the 
> > incompatible test dependency we just removed
> >
> > then we need a newer version of commons-compress:
> >     <commons-compress.version>1.10</commons-compress.version>
> >
> > from here you can fix the actual solr related test errors :)
> >
> >
> >
> > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Ferenc,
> > >
> > > Thanks for offering help.
> > > In agreement with Mike I want to have support for Solr 7.2.1 in 
> > > the morphline solr sink, so that we can easily upgrade the 
> > > Elasticsearch
> > sink.
> > >
> > > My updates are here: https://github.com/hwahrmann/ 
> > > flume/tree/Upgrade_Morphline_Sink
> > >
> > > I changed the solr version to 7.2.1 and was able to compile the 
> > > sink withput any problems.
> > > I can also compile the tests, but when running, I get multiple 
> > > errors
> > like
> > > this:
> > >
> > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> elapsed:
> > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java
> > > :1
> > > 39)
> > >
> > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83
> > > )
> > >
> > >
> > > The all have problems with createJetty. So it seems that I maybe 
> > > need a different version of jetty or something like that.
> > > And for that I have too less knowledge about maven.
> > >
> > > thx,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > Sent: Dienstag, 6. März 2018 11:04
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > >
> > > let me know what can I help You with.
> > > share your current code on a github fork and describe the issue. I 
> > > will see what can we do to solve it.
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I am stuck with Solr.
> > > > The morphline-solr sink compiles without any problems, but I am 
> > > > struggling with the tests.
> > > > Seems I need to exclude some stuff from maven, but my knowledge 
> > > > about maven is not good enough to figure out what to do.
> > > > Anyone able to help?
> > > >
> > > > regards,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > As long as the integration tests still pass and the packaging 
> > > > issues are not exacerbated, I don't see why we couldn't merge an 
> > > > upgrade patch, barring any serious concerns with the patch.
> > > >
> > > > Mike
> > > >
> > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I won't have a problem upgrading the Solr sink to the latest
> version.
> > > > > I am missing test environment however.
> > > > > So while it may build correctly and all integration tests 
> > > > > work, I have no real environment to test with.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > OK. In the pull request, it would be nice if whoever submits 
> > > > > or merges it mentions all of the contributors to the patch in 
> > > > > the commit
> > > message.
> > > > >
> > > > > I asked Wolfgang H. about the SolrServer thing and this is 
> > > > > what he told
> > > > me:
> > > > >
> > > > > Hi Mike, the class has been renamed to "SolrClient" (which 
> > > > > unfortunately
> > > > > > breaks compat). It's just a class rename. The functionality

> > > > > > is the same as before. It was called SolrServer in Solr4 
> > > > > > because it was a client proxy that sends RPCs to a Solr 
> > > > > > server, but calling it SolrClient is more straightforward to

> > > > > > understand, hence the community decided to rename the class.
> > > > > > It's possible to spawn an embedded Solr server, for example

> > > > > > for testing purposes, via class EmbeddedSolrServer (a class

> > > > > > that retains the same name in Solr7 and Solr4), which 
> > > > > > extends the
> > > SolrClient class.
> > > > >
> > > > >
> > > > > Hope this helps,
> > > > > Mike
> > > > >
> > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > Thanks for the response. Would be cool if we get that sorted
out.
> > > > > >
> > > > > > I've asked Yonghao Zou to submit the Pull Request, since he

> > > > > > did most of the work and should get the credit.
> > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > >
> > > > > > I will then issue a Pull request for the new ES Rest client,

> > > > > > which is dependent on the above work.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > Hi Helmut,
> > > > > > I see that I neglected to follow up on the other thread on 
> > > > > > this topic after your reply about SolrServer missing from 
> > > > > > the
> solrj jar.
> > > > > > Let me ask around w/ some folks I know that work on Solr and

> > > > > > see if there is any way to retain the SolrServer for our 
> > > > > > tests after upgrading to the
> > > > > new version.
> > > > > >
> > > > > > Thank you very much for working on upgrading Solr. Would you

> > > > > > mind submitting a pull request with your (apparently
> > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > >
> > > > > > To reply to your email in this thread, the JAR packaging 
> > > > > > situation is largely the same after merging FLUME-2957 so 
> > > > > > unfortunately most of what I noted in my reply in the other

> > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > >
> > > > > > I hope that we can upgrade the Solr dependencies as part of

> > > > > > the same commit as the ES dependencies to avoid worrying 
> > > > > > about which lucene jar is first in the classpath, and ensure

> > > > > > we are not adding any additional dependency conflicts to mvn
> dependency:tree.
> > > > > >
> > > > > > Regards,
> > > > > > Mike
> > > > > >
> > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > > > > helmut.wahrmann@rsa.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > now that the blocker for FLUME-3021 is removed by 
> > > > > > > committing FLUME-2957, can we get the patch from 3021 merged
to trunk?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Helmut
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Mime
View raw message