Return-Path: X-Original-To: apmail-oodt-dev-archive@www.apache.org Delivered-To: apmail-oodt-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 483CE997D for ; Wed, 28 Dec 2011 23:00:57 +0000 (UTC) Received: (qmail 41179 invoked by uid 500); 28 Dec 2011 23:00:57 -0000 Delivered-To: apmail-oodt-dev-archive@oodt.apache.org Received: (qmail 41159 invoked by uid 500); 28 Dec 2011 23:00:57 -0000 Mailing-List: contact dev-help@oodt.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@oodt.apache.org Delivered-To: mailing list dev@oodt.apache.org Received: (qmail 41151 invoked by uid 99); 28 Dec 2011 23:00:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Dec 2011 23:00:57 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of holenoter@me.com designates 17.148.16.99 as permitted sender) Received: from [17.148.16.99] (HELO asmtpout024.mac.com) (17.148.16.99) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Dec 2011 23:00:52 +0000 MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_UGev/hWVLBHjt9EfkL0JZQ)" Received: from spool001.mac.com ([10.150.69.51]) by asmtp024.mac.com (Oracle Communications Messaging Server 7u4-23.01 (7.0.4.23.0) 64bit (built Aug 10 2011)) with ESMTP id <0LWX004QHRW74460@asmtp024.mac.com> for dev@oodt.apache.org; Wed, 28 Dec 2011 15:00:10 -0800 (PST) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.5.7110,1.0.211,0.0.0000 definitions=2011-12-28_08:2011-12-28,2011-12-28,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=1 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1012030000 definitions=main-1112280166 Received: from localhost ([10.150.79.226]) by spool001.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTP id <0LWX00ME6RVSIP50@spool001.mac.com> for dev@oodt.apache.org; Wed, 28 Dec 2011 15:00:08 -0800 (PST) X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=1 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1012030000 definitions=main-1112280167 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.5.7110,1.0.211,0.0.0000 definitions=2011-12-28_08:2011-12-28,2011-12-28,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=1 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1012030000 definitions=main-1112280167 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.5.7110,1.0.211,0.0.0000 definitions=2011-12-28_08:2011-12-28,2011-12-28,1970-01-01 signatures=0 To: dev@oodt.apache.org Cc: "dev@oodt.apache.org" From: holenoter Subject: Re: PushPull Question.... Date: Wed, 28 Dec 2011 22:59:52 +0000 (GMT) X-Mailer: MobileMe Mail (1C323414) X-Originating-IP: [74.125.59.129] Message-id: In-reply-to: --Boundary_(ID_UGev/hWVLBHjt9EfkL0JZQ) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT i believe i did this integration for peate before i left... there is a peate-pushpull which depends on cas-pushpull and adds the non-apache protocols... also can you send me your tar.gz of the pushpull you are currently using... and i will see if i can figure out what is going on. -brian On Dec 28, 2011, at 02:16 PM, "Mattmann, Chris A (388J)" wrote: Hey Cam, On Dec 27, 2011, at 8:48 PM, Cameron Goodale wrote: > Brian, > > Thanks for the info. I don't have my eclipse setup properly to run that > java file as an application so I will have to try that with some help from > Paul or Chris next week perhaps. The best way to run it would be to: 1. cd /usr/local/pushpull 2. java -Djava.ext.dirs=lib org.apache.oodt.cas.pushpull.util.ExpressionValidator > I looked up the regex for Java and the \w > is supposed to be a word character matching alphanumeric and underscore. > > I ran my regex against this online tool I found and it seems to be a valid > match. > http://www.regexplanet.com/advanced/java/index.html > Regex: TES-Aura_L2-CO2-Nadir_r\d{10}\w{2}\d{2}\w\d{2}\.he5 > > Input: TES-Aura_L2-CO2-Nadir_r0000002931_F06_08.he5 > > > Thanks for the feedback and help on this. I have tried so many different > tweaks with backslashes, etc. I am going to just walk away now and look at > it later. I think it's a protocol issue. Let me see if I can throw up those LGPL plugins somewhere b/c we can't host them at Apache. Or better yet, if folks have a better ALv2 compatible FTP library, I'll happily write a PushPull plugin and integrate it. Cheers, Chris > > > -Cam > > > On Tue, Dec 27, 2011 at 2:58 PM, Brian Foster wrote: > >> Hey cameron... I think the problem is with ur >> DirStructXmlParserFiles/TESL2CO2.xml file... for the file name regExp u are >> specifying whitespace (i.e. \w instead of underscore)... pushpull looks >> like it is behaving normally... it is acting like it is not finding any >> files that match ur regExps... I wrote an app which may help u with your >> regular expressions... in the pushpull util package in eclipse right click >> on ExpressionValidator.java and Run As: Java Application... you can test >> your regExp in the GUI that pops up. >> >> -brian >> >> Cameron Goodale wrote: >> >>> Hey Devs, >>> >>> I have been working on building an OODT PushPull User Guide from an >> Example >>> I am trying to get working for a project, and I have encountered a little >>> issue (at least I hope it is little). >>> >>> Here is the guide (work in progress): >>> >> https://cwiki.apache.org/confluence/display/OODT/OODT+Push-Pull+User+Guide >>> >>> The setups listed in the User Guide are the same ones I am working >> against. >>> Now here is the log messages I am seeing. >>> >>> Dec 27, 2011 10:59:00 AM org.apache.oodt.cas.pushpull.daemon.DaemonManager >>> setAsRunning >>> INFO: Daemon with ID = 91131 was given permission to run >>> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser >>> Dec 27, 2011 10:59:00 AM >>> orgapache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >>> INFO: Testing protocol >>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient . . . this >>> may take a few minutes . . . >>> Dec 27, 2011 10:59:00 AM org.apache.oodt.cas.pushpull.protocol.Protocol >>> cdToHOME >>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true' >>> Dec 27, 2011 10:59:00 AM >>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >>> SEVERE: Protocol >>> orgapache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed >>> compatibility test : [CogJGlobusFtpClient] Failed to get list of files : >>> Server refused performing the request. Custom message: (error code 1) >>> [Nested exception message: Custom message: Unexpected reply: 500 'ORT >>> 97,93,139,127,210,161': command not understood.] >>> Dec 27, 2011 10:59:00 AM >>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler >>> getAppropriateProtocolBySite >>> WARNING: ProtocolFactory >>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory is >> not >>> compatible with server at ftp://l4ftl01.larc.nasa.gov >>> Dec 27, 2011 10:59:01 AM >>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >>> INFO: Testing protocol >>> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient . . . this >>> may take a few minutes . . . >>> Dec 27, 2011 10:59:01 AM org.apache.oodt.cas.pushpull.protocol.Protocol >>> cdToHOME >>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true' >>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol >>> cdToROOT >>> INFO: Changing to ROOT directory '/' >>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol >>> cdToHOME >>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true' >>> Dec 27, 2011 10:59:02 AM >>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect >>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with protocol >>> 'org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient' and >>> username 'anonymous' >>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol cd >>> INFO: Changing to directory >>> 'ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005isDir=true' >>> PageSize: 8 PageLoc: 0 >>> FileList size: 491 >>> Dec 27, 2011 10:59:03 AM >>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect >>> INFO: Disconnecting protocol from ftp://l4ftl01.larc.nasa.gov >>> Dec 27, 2011 10:59:03 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep >>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to sleep >>> until Tue Dec 27 11:00:00 PST 2011 >>> >>> >>> First PushPull tries to use the CogJGlobusFtpClient and it encounters an >>> error so it switches to CommonsNetFtpClient does a couple cd commands, >>> connects successfully then disconnects and goes back to sleep. It will >>> then wake up after the 3 minutes and repeat the same process. Has anyone >>> seen this issue or have any suggestions for a fix? >>> >>> Thank you in advance. >>> >>> -Cameron >>> >>> P.S. If you have any feedback on the PushPull User Guide be sure to leave >>> comments or make changes to the wiki. >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattmann@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ --Boundary_(ID_UGev/hWVLBHjt9EfkL0JZQ) Content-type: multipart/related; boundary="Boundary_(ID_RfiUT+yMJ7B9u6J5kIH6pg)"; type="text/html" --Boundary_(ID_RfiUT+yMJ7B9u6J5kIH6pg) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT
i believe i did this integration for peate before i left... there is a peate-pushpull which depends on cas-pushpull and adds the non-apache protocols... also can you send me your tar.gz of the pushpull you are currently using... and i will see if i can figure out what is going on.

-brian

On Dec 28, 2011, at 02:16 PM, "Mattmann, Chris A (388J)" <chris.a.mattmann@jpl.nasa.gov> wrote:

Hey Cam,

On Dec 27, 2011, at 8:48 PM, Cameron Goodale wrote:

> Brian,
>
> Thanks for the info. I don't have my eclipse setup properly to run that
> java file as an application so I will have to try that with some help from
> Paul or Chris next week perhaps.

The best way to run it would be to:

1. cd /usr/local/pushpull
2. java -Djava.ext.dirs=lib org.apache.oodt.cas.pushpull.util.ExpressionValidator

> I looked up the regex for Java and the \w
> is supposed to be a word character matching alphanumeric and underscore.
>
> I ran my regex against this online tool I found and it seems to be a valid
> match.
> http://www.regexplanet.com/advanced/java/index.html
> Regex: TES-Aura_L2-CO2-Nadir_r\d{10}\w{2}\d{2}\w\d{2}\.he5
>
> Input: TES-Aura_L2-CO2-Nadir_r0000002931_F06_08.he5
>
>
> Thanks for the feedback and help on this. I have tried so many different
> tweaks with backslashes, etc. I am going to just walk away now and look at
> it later.

I think it's a protocol issue. Let me see if I can throw up those LGPL plugins
somewhere b/c we can't host them at Apache. Or better yet, if folks have a
better ALv2 compatible FTP library, I'll happily write a PushPull plugin and
integrate it.

Cheers,
Chris


>
>
> -Cam
>
>
> On Tue, Dec 27, 2011 at 2:58 PM, Brian Foster <holenoter@me.com> wrote:
>
>> Hey cameron.. I think the problem is with ur
>> DirStructXmlParserFiles/TESL2CO2.xml file... for the file name regExp u are
>> specifying whitespace (i.e. \w instead of underscore)... pushpull looks
>> like it is behaving normally... it is acting like it is not finding any
>> files that match ur regExps... I wrote an app which may help u with your
>> regular expressions... in the pushpull util package in eclipse right click
>> on ExpressionValidator.java and Run As: Java Application... you can test
>> your regExp in the GUI that pops up.
>>
>> -brian
>>
>> Cameron Goodale <goodale@apache.org> wrote:
>>
>>> Hey Devs,
>>>
>>> I have been working on building an OODT PushPull User Guide from an
>> Example
>>> I am trying to get working for a project, and I have encountered a little
>>> issue (at least I hope it is little).
>>>
>>> Here is the guide (work in progress):
>>>
>> https://cwiki.apache.org/confluence/display/OODT/OODT+Push-Pull+User+Guide
>>>
>>> The setups listed in the User Guide are the same ones I am working
>> against.
>>> Now here is the log messages I am seeing.
>>>
>>> Dec 27, 2011 10:59:00 AM org.apache.oodt.cas.pushpull.daemon.DaemonManager
>>> setAsRunning
>>> INFO: Daemon with ID = 91131 was given permission to run
>>> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser
>>> Dec 27, 2011 10:59:00 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>>> INFO: Testing protocol
>>> org.apache.oodtcas.pushpull.protocol.ftp.CogJGlobusFtpClient . . . this
>>> may take a few minutes . . .
>>> Dec 27, 2011 10:59:00 AM org.apache.oodt.cas.pushpull.protocol.Protocol
>>> cdToHOME
>>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true'
>>> Dec 27, 2011 10:59:00 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>>> SEVERE: Protocol
>>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed
>>> compatibility test : [CogJGlobusFtpClient] Failed to get list of files :
>>> Server refused performing the request. Custom message: (error code 1)
>>> [Nested exception message: Custom message: Unexpected reply: 500 'ORT
>>> 97,93,139,127,210,161': command not understood.]
>>> Dec 27, 2011 10:59:00 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler
>>> getAppropriateProtocolBySite
>>> WARNING: ProtocolFactory
>>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory is
>> not
>>> compatible with server at ftp://l4ftl01.larc.nasa.gov
>>> Dec 27, 2011 10:59:01 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>>> INFO: Testing protocol
>>> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient . . . this
>>> may take a few minutes . . .
>>> Dec 27, 2011 10:59:01 AM org.apache.oodt.cas.pushpull.protocol.Protocol
>>> cdToHOME
>>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true'
>>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol
>>> cdToROOT
>>> INFO: Changing to ROOT directory '/'
>>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol
>>> cdToHOME
>>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/isDir=true'
>>> Dec 27, 2011 10:59:02 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with protocol
>>> 'org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient' and
>>> username 'anonymous'
>>> Dec 27, 2011 10:59:02 AM org.apache.oodt.cas.pushpull.protocol.Protocol cd
>>> INFO: Changing to directory
>>> 'ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005isDir=true'
>>> PageSize: 8 PageLoc: 0
>>> FileList size: 491
>>> Dec 27, 2011 10:59:03 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect
>>> INFO: Disconnecting protocol from ftp://l4ftl01.larc.nasa.gov
>>> Dec 27, 2011 10:59:03 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep
>>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to sleep
>>> until Tue Dec 27 11:00:00 PST 2011
>>>
>>>
>>> First PushPull tries to use the CogJGlobusFtpClient and it encounters an
>>> error so it switches to CommonsNetFtpClient does a couple cd commands,
>>> connects successfully then disconnects and goes back to sleep. It will
>>> then wake up after the 3 minutes and repeat the same process. Has anyone
>>> seen this issue or have any suggestions for a fix?
>>>
>>> Thank you in advance.
>>>
>>> -Cameron
>>>
>>> P.S. If you have any feedback on the PushPull User Guide be sure to leave
>>> comments or make changes to the wiki.
>>


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

--Boundary_(ID_RfiUT+yMJ7B9u6J5kIH6pg)-- --Boundary_(ID_UGev/hWVLBHjt9EfkL0JZQ)--