oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Problem when running pushpull
Date Tue, 24 Jul 2012 16:43:57 GMT
Hi YunHee,

Thanks, can you try some of the other XML file configs. Maybe it's being
blocked in your country, or something, but let's see if we can get some
of the other XML RemoteSpec configs to work?

Cheers,
Chris

On Jul 24, 2012, at 9:40 AM, YunHee Kang wrote:

> Hi Chris,
> 
> 
>> Try out the MODIS one there, does that work?
> No, it doesn't.
> I think  that the ftp site  e4ftl01u.ecs.nasa.gov has a connection problem.
> 
> $ ftp e4ftl01u.ecs.nasa.gov
> ftp: connect: Connection timed out
> ftp>
> 
> The message below also say the reason why the problem was occurred.
> WARNING: Error occurred while connecting to RemoteSite: alias =
> 'ModisFtp'  url = 'ftp://e4ftl01u.ecs.nasa.gov'  username =
> 'anonymous' cdTestDir = 'null' maxConnections = '-1' : Failed to
> connect to server : Connection timed out
> Jul 25, 2012 1:31:25 AM
> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
> INFO: Will retry connecting to RemoteSite: alias = 'ModisFtp'  url =
> 'ftp://e4ftl01u.ecs.nasa.gov'  username = 'anonymous' cdTestDir =
> 'null' maxConnections = '-1' in 5 seconds
> Waiting . . . . .
> 
> Thanks,
> Yunhee
> 
> 2012/7/25 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>> Hi YunHee,
>> 
>> OK, can you try some of the other files, e.g., the ones in:
>> 
>> http://s.apache.org/ks0
>> 
>> Try out the MODIS one there, does that work?
>> 
>> Cheers,
>> Chris
>> 
>> On Jul 24, 2012, at 8:38 AM, YunHee Kang wrote:
>> 
>>> Hi Chris,
>>> 
>>> I added the header of XML as follows:
>>> <?xml version="1.0"?>
>>> 
>>> Next the value of elements was changed into specific one.
>>> The following is the output of the XML:
>>> <?xml version="1.0"?>
>>> <root>
>>>   <dirstruct starting_path="/TES/TL2CO2N.005">
>>>       <nofiles/>
>>>       <!--dir name="\d{4}\.\d{2}\.\d{2}"-->
>>>       <dir name="2004\.09\.20">
>>>           <nodirs/>
>>>           <!--file
>>> name="TES-Aura\_L2-CO2-Nadir\_r\d{10}\w{2}\d{2}\w\d{2}\.he5"/-->
>>>           <file name="TES-Aura\_L2-CO2-Nadir\_r0000002147\_F06\_09\.he5"/>
>>>       </dir>
>>>   </dirstruct>
>>> </root>
>>> 
>>> Unfortunately I got the same result before:
>>> ...
>>> INFO: Testing protocol
>>> org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . . this ma
>>>      y take a few minutes . . .
>>> Jul 25, 2012 12:30:03 AM
>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
>>> protocol 'org.apache.oodt.
>>> cas.protocol.ftp.CommonsNetFtpProtocol' and username 'anonymous'
>>> Jul 25, 2012 12:30:03 AM
>>> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup retrie
>>>    veFiles
>>> SEVERE: Failed to finish downloading per property files
>>> /home/yhkang/oodt/cas-pushpull/etc/
>>> examples/DirStructXmlParserFiles/TESL2CO2.xml : null
>>> Jul 25, 2012 12:30:04 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep
>>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to
>>> sleep until Wed Jul 25 0
>>> ...
>>> 
>>> 
>>> Thanks,
>>> Yunhee
>>> 
>>> 
>>> 2012/7/25 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>>>> Hi YunHee,
>>>> 
>>>> Can you try simply putting a specific <dir name="..."> instead of \d{4},
etc., and
>>>> then can you put a specific file name in the <file name="..">. If that
works and
>>>> downloads, then we know there is something up with the regex somehow.
>>>> Also can you try adding the <?xml version="1.0"?> start to your file?
>>>> 
>>>> Cheers,
>>>> Chris
>>>> 
>>>> On Jul 24, 2012, at 8:16 AM, YunHee Kang wrote:
>>>> 
>>>>> Hi Chris,
>>>>> 
>>>>> 
>>>>> The following is the content of  TESL2CO2.xml.
>>>>> $ cat /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>>> <root>
>>>>>  <dirstruct starting_path="/TES/TL2CO2N.005">
>>>>>      <nofiles/>
>>>>>      <dir name="\d{4}\.\d{2}\.\d{2}">
>>>>>          <nodirs/>
>>>>>          <file name="TES-Aura\_L2-CO2-Nadir\_r\d{10}\w{2}\d{2}\w\d{2}\.he5"/>
>>>>>      </dir>
>>>>>  </dirstruct>
>>>>> </root>
>>>>> 
>>>>> Let me know if there is a problem in the configuration file.
>>>>> 
>>>>> Thanks,
>>>>> Yunhee
>>>>> 
>>>>> 2012/7/24 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>>>>>> Hi YunHee,
>>>>>> 
>>>>>> On Jul 24, 2012, at 12:30 AM, YunHee Kang wrote:
>>>>>> 
>>>>>>> Hi Chris,
>>>>>>> 
>>>>>>> I am stuck on the problem related with  "pushpull" script  occurred
>>>>>>> the following error message:
>>>>>>> ....
>>>>>>> INFO: Testing protocol
>>>>>>> org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . .
this may
>>>>>>> take a few minutes . . .
>>>>>>> Jul 24, 2012 4:05:02 PM
>>>>>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>>>>>>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
>>>>>>> protocol 'org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol'
and
>>>>>>> username 'anonymous'
>>>>>>> Jul 24, 2012 4:05:02 PM
>>>>>>> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup
>>>>>>> retrieveFiles
>>>>>>> SEVERE: Failed to finish downloading per property files
>>>>>>> /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>>>>> : null
>>>>>> 
>>>>>> Can you do me a favor and simply run this command:
>>>>>> 
>>>>>> cat /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>>>> 
>>>>>> What do you get?
>>>>>> 
>>>>>>> Jul 24, 2012 4:05:02 PM org.apache.oodt.cas.pushpull.daemon.Daemon
sleep
>>>>>>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going
to
>>>>>>> sleep until Tue Jul 24 16:06:00 KST 2012
>>>>>>> Jul 24, 2012 4:06:00 PM
>>>>>>> org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning
>>>>>>> INFO: Daemon with ID = 91131 was given permission to run
>>>>>>> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser
>>>>>>> 
>>>>>>> First of all, I would like test the pushpull framework  to move
data
>>>>>>> written in NetCDF from a remote site by a network protocols (like
ftp,
>>>>>>> http, etc).
>>>>>> 
>>>>>> Yep, sounds good.
>>>>>> 
>>>>>>> I think  that there is a test suite which can be used to  check
the
>>>>>>> pushpull framework.
>>>>>>> Can you give  configuration files(a set of XML files)  that I
can test
>>>>>>> downloading  files from a  FTP site (public anonymous site) ?
>>>>>> 
>>>>>> The examples in the resources directory that you found should work
>>>>>> so we'll just need to figure out why they aren't working on your
system.
>>>>>> 
>>>>>>> or
>>>>>>> Please let me know  a reference implementation of pushpull framework
>>>>>>> that I can use to move NetCDF files.
>>>>>> 
>>>>>> Thanks we'll figure this out.
>>>>>> 
>>>>>> Cheers,
>>>>>> Chris
>>>>>> 
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Chris Mattmann, Ph.D.
>>>>>> Senior Computer Scientist
>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>> WWW:   http://sunset.usc.edu/~mattmann/
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> 
>>>> 
>>>> 
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Chris Mattmann, Ph.D.
>>>> Senior Computer Scientist
>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 171-266B, Mailstop: 171-246
>>>> Email: chris.a.mattmann@nasa.gov
>>>> WWW:   http://sunset.usc.edu/~mattmann/
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Adjunct Assistant Professor, Computer Science Department
>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> 
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message