oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mattmann <mattm...@apache.org>
Subject Re: [VOTE] Apache OODT 1.1 RC #1
Date Sat, 22 Jul 2017 15:16:38 GMT
Yep there is a new VOTE (RC #2) my friend (

Please look at that one (




On 7/22/17, 7:51 AM, "Tom Barber" <tom.barber@meteorite.bi> wrote:

    Build looks good
    Tests pass (except its usual hated of the BST timezone)
    
    I assume the repo is actually:
    https://repository.apache.org/content/repositories/orgapacheoodt-1013 ?
    
    Artifacts look good, notice and license in place.
    
    I shall say +1 on the assumption of the above repo mislinking.
    
    Thanks for cobbling it together Chris.
    
    Tom
    
    
    
    
    On Sat, Jul 22, 2017 at 9:26 AM, Tom Barber <tom.barber@meteorite.bi> wrote:
    
    > I have a kid free afternoon I'll test it later.
    >
    > Tom
    >
    > On 21 Jul 2017 9:20 pm, "Chris Mattmann" <mattmann@apache.org> wrote:
    >
    >> Thanks no worries. I just need 1 more +1 and I can release tomorrow…
    >>
    >>
    >>
    >> On 7/21/17, 1:15 PM, "BW" <webb@apache.org> wrote:
    >>
    >>     Hi Chris. Haven't yet; deferred I guess. Sorry, on vacay..
    >>
    >>     On Fri, Jul 21, 2017 at 1:11 PM Chris Mattmann <mattmann@apache.org>
    >> wrote:
    >>
    >>     > Hi BW, have you tried RC #2? +1 still?
    >>     >
    >>     > Cheers,
    >>     > Chris
    >>     >
    >>     >
    >>     >
    >>     >
    >>     > On 7/19/17, 5:32 PM, "BW" <webb@apache.org> wrote:
    >>     >
    >>     >     +1
    >>     >
    >>     >     On Wed, Jul 19, 2017 at 8:51 AM Chris Mattmann <
    >> mattmann@apache.org>
    >>     > wrote:
    >>     >
    >>     >     > Absolutely, here is a simple use case:
    >>     >     >
    >>     >     > If you look at CAS-PGE files, they typically include setup
to
    >> “stage
    >>     > files”
    >>     >     > (already archived in File Manager) to the local job working
    >>     > directly. Many
    >>     >     > times
    >>     >     > this is just handled in the bash script (or Python or
    >> whatever) that
    >>     >     > CAS-PGE generates
    >>     >     > itself. However it’s such a repetitive task and *part of*
the
    >> actual
    >>     >     > workflow for running
    >>     >     > a job, we added a simple interface to do so. See here:
    >>     >     >
    >>     >     >
    >>     >     >
    >>     > https://raw.githubusercontent.com/apache/oodt/master/pge/src
    >> /test/resources/pge-config.xml
    >>     >     >
    >>     >     > And notice this block:
    >>     >     >
    >>     >     >   <fileStaging dir="[StagingDir]" force="[ForceStaging]">
    >>     >     >     <stageFiles metadataKey="InputFiles" />
    >>     >     >   </fileStaging>
    >>     >     >
    >>     >     > What the above does is inspect the value of the InputFiles
    >> metadata
    >>     > as
    >>     >     > provided
    >>     >     > to CAS-PGE (even if it’s a list) and for each of those files,
    >> it
    >>     > “stages”
    >>     >     > them (aka uses
    >>     >     > the associated and declared DataTransferer for this CAS-PGE
    >>     > instance) to
    >>     >     > the directory
    >>     >     > defined by [StagingDir].
    >>     >     >
    >>     >     > Cool huh?
    >>     >     >
    >>     >     > Cheers,
    >>     >     > Chris
    >>     >     >
    >>     >     >
    >>     >     > P.S. It doesn’t work now, but will shortly ;)
    >>     >     >
    >>     >     >
    >>     >     > On 7/19/17, 8:43 AM, "Mallder, Valerie" <
    >> Valerie.Mallder@jhuapl.edu>
    >>     >     > wrote:
    >>     >     >
    >>     >     >     Yes, adding a note to CHANGES.txt is fine.  The fix to
the
    >>     > fileStaging
    >>     >     > feature of CAS-PGE sounds good, although I've never
    >> understood what
    >>     > that
    >>     >     > feature is supposed to do :)  When you have some free time
I
    >> would
    >>     > love to
    >>     >     > hear about what you are using it for.
    >>     >     >
    >>     >     >     Thanks!
    >>     >     >
    >>     >     >
    >>     >     >     -----Original Message-----
    >>     >     >     From: Chris Mattmann [mailto:mattmann@apache.org]
    >>     >     >     Sent: Wednesday, July 19, 2017 11:37 AM
    >>     >     >     To: dev@oodt.apache.org
    >>     >     >     Subject: Re: [VOTE] Apache OODT 1.1 RC #1
    >>     >     >
    >>     >     >     Thank you Val!
    >>     >     >
    >>     >     >     I may spin another RC not only because of the below
    >> (which I
    >>     > think we
    >>     >     > should just note in CHANGES.txt (I am happy to add a note on
    >> it),
    >>     > but also
    >>     >     > b/c I found an interesting bug while testing OODT 1.1 RC #1
    >> with
    >>     > DRAT – the
    >>     >     > bug relates to using fileStaging – it doesn’t explicitly
set a
    >>     >     > Product.PRODUCT_STRUCTURE* and as so gets an NPE if you try
    >> and use
    >>     > it. I
    >>     >     > never encountered this before b/c I never used OODT’s native
    >> file
    >>     > staging
    >>     >     > support in CAS-PGE. It is a simple fix, and I think will add
    >> a lot
    >>     > of value
    >>     >     > (fileStaging is inoperable without it in CAS-PGE).
    >>     >     >
    >>     >     >     Due to that, I will spin another RC shortly.
    >>     >     >
    >>     >     >     Thank you!
    >>     >     >
    >>     >     >     Cheers,
    >>     >     >     Chris
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >     On 7/19/17, 7:55 AM, "Mallder, Valerie" <
    >>     > Valerie.Mallder@jhuapl.edu>
    >>     >     > wrote:
    >>     >     >
    >>     >     >         +1
    >>     >     >
    >>     >     >         I downloaded the source and built it with "mvn package
    >>     > -skipTests"
    >>     >     > and it built ok. I did a system test by running some data
    >> through my
    >>     >     > pipeline that uses cas-workflow-1.1, cas-filemgr-1.1 and
    >> cas-pge-1.1
    >>     > and
    >>     >     > everything worked as I expected.  And it even helped find some
    >>     > things that
    >>     >     > I needed to tweak in my test environment :)
    >>     >     >
    >>     >     >         The only problem I should note is that had to delete
    >> the
    >>     > workflow
    >>     >     > repository directory and allow cas-workflow-1.1 to create a
    >>     > repository from
    >>     >     > scratch. cas-workflow-1.1 reported an incompatibility with
    >> workflow
    >>     >     > repository that was created with cas-workflow-0.10 and has
    >> been used
    >>     > with
    >>     >     > cas-workflow-1.0 for a while now. (Also note that, if the
    >> repository
    >>     >     > directory exists but it is empty, workflow manager will
    >> crash. So, I
    >>     > had to
    >>     >     > delete the repository directory itself rather than just
    >> delete the
    >>     > contents
    >>     >     > within it.)
    >>     >     >
    >>     >     >          The error I received is below. It looks like the
    >> change from
    >>     >     > lucene-core-2.0.0.jar to the lucene-core-6.1.0.jar caused this
    >>     >     > incompatibility between cas-workflow-1.0 and cas-workflow-1.1.
    >>     > Deleting my
    >>     >     > pipeline's workflow respository and allowing
    >> cas-workflow-1.1 to
    >>     > create a
    >>     >     > new one was not a problem for me, but I can't speak for
    >> everyone.
    >>     >     >
    >>     >     >         Jul 19, 2017 10:45:02 AM
    >>     >     >
    >>     > org.apache.oodt.cas.workflow.instrepo.LuceneWorkflowInstance
    >> Repository
    >>     >     > addWorkflowInstanceToCatalog
    >>     >     >         WARNING: Unable to index workflow instance:
    >>     >     > [d8dad567-6c90-11e7-9e4e-7574e3a12106]: Message: Format
    >> version is
    >>     > not
    >>     >     > supported (resource
    >>     >     >
    >>     > BufferedChecksumIndexInput(MMapIndexInput(path="/project/ood
    >> t/dev/jedi/data/workflow/slothrop/segments"))):
    >>     >     > -1 (needs to be between 1071082519 and 1071082519). This
    >> version of
    >>     > Lucene
    >>     >     > only supports indexes created with release 5.0 and later.
    >>     >     >         java.lang.NullPointerException
    >>     >     >
    >>     >     >         Val
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >         -----Original Message-----
    >>     >     >         From: Chris Mattmann [mailto:mattmann@apache.org]
    >>     >     >         Sent: Tuesday, July 18, 2017 6:54 PM
    >>     >     >         To: dev@oodt.apache.org
    >>     >     >         Subject: [VOTE] Apache OODT 1.1 RC #1
    >>     >     >
    >>     >     >         Hi Folks,
    >>     >     >
    >>     >     >         I have posted a 1st release candidate for the Apache
    >> OODT 1.1
    >>     >     > release. The source code is at:
    >>     >     >
    >>     >     >         https://dist.apache.org/repos/dist/dev/oodt/
    >>     >     >
    >>     >     >         For more detailed information, see the included
    >> CHANGES.txt
    >>     > file
    >>     >     > for details on release contents and latest changes. The
    >> release was
    >>     > made
    >>     >     > using the OODT release process, documented on the Wiki here:
    >>     >     >
    >>     >     >
    >>     > https://cwiki.apache.org/confluence/display/OODT/Release+Process
    >>     >     >
    >>     >     >         The release was made from the OODT 1.1 tag at:
    >>     >     >
    >>     >     >         https://github.com/apache/oodt/tree/1.1
    >>     >     >
    >>     >     >         A staged Maven repository is available at:
    >>     >     >
    >>     >     >
    >>     >     >
    >>     > https://repository.apache.org/content/repositories/orgapache
    >> oodt-1012/
    >>     >     >
    >>     >     >         Please vote on releasing these packages as Apache
    >> OODT 1.1.
    >>     > The
    >>     >     > vote is open for at least the next 72 hours.
    >>     >     >
    >>     >     >         Only votes from OODT PMC are binding, but folks are
    >> welcome
    >>     > to
    >>     >     > check the release candidate and voice their approval or
    >> disapproval.
    >>     > The
    >>     >     > vote passes if at least three binding +1 votes are cast.
    >>     >     >
    >>     >     >         [ ] +1 Release the packages as Apache OODT 1.1
    >>     >     >
    >>     >     >         [ ] -1 Do not release the packages because...
    >>     >     >
    >>     >     >         Thanks!
    >>     >     >
    >>     >     >         Chris Mattmann
    >>     >     >
    >>     >     >         P.S. Here is my +1.
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >     >
    >>     >
    >>     >
    >>     >
    >>     >
    >>
    >>
    >>
    >>
    



Mime
View raw message