oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Upgrading Lucene and Solr
Date Mon, 11 Jul 2016 15:19:14 GMT
I have a beautiful use case for BigTranslate that I can test this
in pretty immediately - so commit to 1.x soon and I’ll be your 
guinea pig.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Director, Information Retrieval and Data Science Group (IRDS)
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
WWW: http://irds.usc.edu/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++









On 7/11/16, 11:17 AM, "Tom Barber" <tom.barber@meteorite.bi> wrote:

>INFO] Reactor Summary:
>[INFO]
>[INFO] OODT Core .......................................... SUCCESS [
>10.222 s]
>[INFO] Common Utilities ................................... SUCCESS [
>17.715 s]
>[INFO] CAS Command Line Interface ......................... SUCCESS [
> 9.428 s]
>[INFO] Process Control System Input Data Package .......... SUCCESS [
> 1.564 s]
>[INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
>SUCCESS [  3.094 s]
>[INFO] CAS Protocol ....................................... SUCCESS [
> 9.456 s]
>[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> 4.550 s]
>[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>16.927 s]
>[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> 4.857 s]
>[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> 4.020 s]
>[INFO] Query Expression ................................... SUCCESS [
> 3.140 s]
>[INFO] OODT Single Sign On Security Package ............... SUCCESS [
> 2.458 s]
>[INFO] Catalog and Archive File Management Component ...... SUCCESS [04:32
>min]
>[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>15.820 s]
>[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>23.145 s]
>[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>59.957 s]
>[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>22.978 s]
>[INFO] CAS Curation Web Services .......................... SUCCESS [
> 9.949 s]
>[INFO] Process Control System Core Package ................ SUCCESS [
>13.890 s]
>[INFO] OODT Wicket Web Components ......................... SUCCESS [
> 4.355 s]
>[INFO] CAS Curation Interface ............................. SUCCESS [
> 9.994 s]
>[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>18.437 s]
>[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> 6.322 s]
>[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> 9.322 s]
>[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> 1.049 s]
>[INFO] OODT :: Archetypes ................................. SUCCESS [
> 0.014 s]
>[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>15.797 s]
>[INFO] Product Service .................................... SUCCESS [
> 1.704 s]
>[INFO] Profile Service .................................... SUCCESS [
> 7.741 s]
>[INFO] OODT Web Grid ...................................... SUCCESS [
>19.080 s]
>[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
>12.202 s]
>[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>21.353 s]
>[INFO] CAS File Manager Browser Web App ................... SUCCESS [
> 9.061 s]
>[INFO] CAS Product Server Web Application ................. SUCCESS [
>12.943 s]
>[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> 3.712 s]
>[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> 8.218 s]
>[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>10.785 s]
>[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>13.757 s]
>[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> 7.015 s]
>[INFO] Apache OODT ........................................ SUCCESS [
> 0.020 s]
>[INFO]
>------------------------------------------------------------------------
>[INFO] BUILD SUCCESS
>[INFO]
>------------------------------------------------------------------------
>[INFO] Total time: 11:40 min
>[INFO] Finished at: 2016-07-11T15:11:00+00:00
>[INFO] Final Memory: 167M/2913M
>[INFO]
>------------------------------------------------------------------------
>root@ubuntu:~/oodt#
>
>
>Okay I finished up the workflow changes last night and all the tests pass
>on 2.x so I'll lift those commits tonight and patch the 1.x branch.
>
>Just because the tests pass, don't expect it to be perfect, I don't know
>lucene I just manipulated what existed to work with the new API, someone
>who knows what they are doing needs to take a look, specifically at the
>todoc() methods in Filemanager and Workflow.
>
>Tom
>
>On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
>chris.a.mattmann@jpl.nasa.gov> wrote:
>
>> Yeah we can back port the Lucene part to 1.x
>>
>> Sent from my iPhone
>>
>> > On Jul 8, 2016, at 11:15 PM, Tom Barber <tom.barber@meteorite.bi> wrote:
>> >
>> > yeah there's no reason we can't diff the branch and merge it into 1.0 as
>> > well
>> >> On 9 Jul 2016 00:26, "Chris Mattmann" <mattmann@apache.org> wrote:
>> >>
>> >> awesome!!!
>> >>
>> >> Please backport to 1.x :)
>> >>
>> >>
>> >>
>> >>
>> >>> On 7/8/16, 3:01 PM, "Tom Barber" <tom.barber@meteorite.bi> wrote:
>> >>>
>> >>> There you go, running commentary over. The filemanager test suite now
>> >>> passes on Lucene 6, I'll fix up the rest next week.
>> >>>
>> >>> Tom
>> >>>
>> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <tom.barber@meteorite.bi>
>> >>> wrote:
>> >>>
>> >>>> Better than I thought actually:
>> >>>>
>> >>>> Tests in error:
>> >>>>  TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>> >>>> CmdLineAction ...
>> >>>>  TestFileManagerCli.testLuceneQuery:350 » Runtime
>> >>>> org.apache.oodt.cas.cli.excep...
>> >>>>
>> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>> >>>>
>> >>>> The failure is actually one Avro. So only 2 failing on lucene 6
in the
>> >> FM.
>> >>>>
>> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <tom.barber@meteorite.bi>
>> >>>> wrote:
>> >>>>
>> >>>>> Just FYI:
>> >>>>>
>> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based
on
>> 2.0
>> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has
been
>> 75%
>> >>>>> (guess) converted and tests are half passing.
>> >>>>>
>> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>> >>>>> lewis.mcgibbney@gmail.com> wrote:
>> >>>>>
>> >>>>>> ACK
>> >>>>>>
>> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <tom.barber@meteorite.bi
>> >
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch
2.0-dev or
>> >>>>>> similar
>> >>>>>>> then branch the branch to uprade solr and lucene outside
of that
>> >> scope
>> >>>>>> so
>> >>>>>>> we don't destroy the semi working avro upgrades ;)
>> >>>>>>>
>> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
>> tom.barber@meteorite.bi
>> >>>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Coolio!
>> >>>>>>>>
>> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
>> >> mattmann@apache.org>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Yes I was just telling Paul R. the other day
we need to do
>> >>>>>>>>> this, or we’re DOA. I’ve first hand seen
the scalability
>> >>>>>>>>> issues with Big Translate and DRAT using the
old Lucene
>> >>>>>>>>> library. Let’s get it fixed in 2.x.
>> >>>>>>>>>
>> >>>>>>>>> Cheers,
>> >>>>>>>>> Chris
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <tom.barber@meteorite.bi>
>> >> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Just throwing this one out there as we're
making a bunch of
>> >>>>>> breaking
>> >>>>>>>>>> changes in 2.0.
>> >>>>>>>>>>
>> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions
within OODT?
>> >>>>>>>>>>
>> >>>>>>>>>> My feeling is that they are both pretty
old and whilst we're
>> >> making
>> >>>>>>> life
>> >>>>>>>>>> hard for everyone we might as well go the
whole hog and drag
>> >> them
>> >>>>>> into
>> >>>>>>>>> the
>> >>>>>>>>>> 21st century, but I'm interested in others
opinions.
>> >>>>>>>>>>
>> >>>>>>>>>> Chris you have a much better understand
about those things than
>> >>>>>> myself.
>> >>>>>>>>>>
>> >>>>>>>>>> Tom
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Lewis*
>> >>
>> >>
>>
Mime
View raw message