oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YunHee Kang <yunh.k...@gmail.com>
Subject Re: Problem when running pushpull
Date Tue, 24 Jul 2012 16:40:47 GMT
Hi Chris,


> Try out the MODIS one there, does that work?
No, it doesn't.
I think  that the ftp site  e4ftl01u.ecs.nasa.gov has a connection problem.

$ ftp e4ftl01u.ecs.nasa.gov
ftp: connect: Connection timed out
ftp>

The message below also say the reason why the problem was occurred.
WARNING: Error occurred while connecting to RemoteSite: alias =
'ModisFtp'  url = 'ftp://e4ftl01u.ecs.nasa.gov'  username =
'anonymous' cdTestDir = 'null' maxConnections = '-1' : Failed to
connect to server : Connection timed out
Jul 25, 2012 1:31:25 AM
org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
INFO: Will retry connecting to RemoteSite: alias = 'ModisFtp'  url =
'ftp://e4ftl01u.ecs.nasa.gov'  username = 'anonymous' cdTestDir =
'null' maxConnections = '-1' in 5 seconds
Waiting . . . . .

Thanks,
Yunhee

2012/7/25 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
> Hi YunHee,
>
> OK, can you try some of the other files, e.g., the ones in:
>
> http://s.apache.org/ks0
>
> Try out the MODIS one there, does that work?
>
> Cheers,
> Chris
>
> On Jul 24, 2012, at 8:38 AM, YunHee Kang wrote:
>
>> Hi Chris,
>>
>> I added the header of XML as follows:
>> <?xml version="1.0"?>
>>
>> Next the value of elements was changed into specific one.
>> The following is the output of the XML:
>> <?xml version="1.0"?>
>> <root>
>>    <dirstruct starting_path="/TES/TL2CO2N.005">
>>        <nofiles/>
>>        <!--dir name="\d{4}\.\d{2}\.\d{2}"-->
>>        <dir name="2004\.09\.20">
>>            <nodirs/>
>>            <!--file
>> name="TES-Aura\_L2-CO2-Nadir\_r\d{10}\w{2}\d{2}\w\d{2}\.he5"/-->
>>            <file name="TES-Aura\_L2-CO2-Nadir\_r0000002147\_F06\_09\.he5"/>
>>        </dir>
>>    </dirstruct>
>> </root>
>>
>> Unfortunately I got the same result before:
>> ...
>> INFO: Testing protocol
>> org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . . this ma
>>       y take a few minutes . . .
>> Jul 25, 2012 12:30:03 AM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
>> protocol 'org.apache.oodt.
>> cas.protocol.ftp.CommonsNetFtpProtocol' and username 'anonymous'
>> Jul 25, 2012 12:30:03 AM
>> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup retrie
>>     veFiles
>> SEVERE: Failed to finish downloading per property files
>> /home/yhkang/oodt/cas-pushpull/etc/
>> examples/DirStructXmlParserFiles/TESL2CO2.xml : null
>> Jul 25, 2012 12:30:04 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep
>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to
>> sleep until Wed Jul 25 0
>> ...
>>
>>
>> Thanks,
>> Yunhee
>>
>>
>> 2012/7/25 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>>> Hi YunHee,
>>>
>>> Can you try simply putting a specific <dir name="..."> instead of \d{4},
etc., and
>>> then can you put a specific file name in the <file name="..">. If that
works and
>>> downloads, then we know there is something up with the regex somehow.
>>> Also can you try adding the <?xml version="1.0"?> start to your file?
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Jul 24, 2012, at 8:16 AM, YunHee Kang wrote:
>>>
>>>> Hi Chris,
>>>>
>>>>
>>>> The following is the content of  TESL2CO2.xml.
>>>> $ cat /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>> <root>
>>>>   <dirstruct starting_path="/TES/TL2CO2N.005">
>>>>       <nofiles/>
>>>>       <dir name="\d{4}\.\d{2}\.\d{2}">
>>>>           <nodirs/>
>>>>           <file name="TES-Aura\_L2-CO2-Nadir\_r\d{10}\w{2}\d{2}\w\d{2}\.he5"/>
>>>>       </dir>
>>>>   </dirstruct>
>>>> </root>
>>>>
>>>> Let me know if there is a problem in the configuration file.
>>>>
>>>> Thanks,
>>>> Yunhee
>>>>
>>>> 2012/7/24 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>>>>> Hi YunHee,
>>>>>
>>>>> On Jul 24, 2012, at 12:30 AM, YunHee Kang wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> I am stuck on the problem related with  "pushpull" script  occurred
>>>>>> the following error message:
>>>>>> ....
>>>>>> INFO: Testing protocol
>>>>>> org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . . this
may
>>>>>> take a few minutes . . .
>>>>>> Jul 24, 2012 4:05:02 PM
>>>>>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>>>>>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
>>>>>> protocol 'org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol'
and
>>>>>> username 'anonymous'
>>>>>> Jul 24, 2012 4:05:02 PM
>>>>>> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup
>>>>>> retrieveFiles
>>>>>> SEVERE: Failed to finish downloading per property files
>>>>>> /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>>>> : null
>>>>>
>>>>> Can you do me a favor and simply run this command:
>>>>>
>>>>> cat /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
>>>>>
>>>>> What do you get?
>>>>>
>>>>>> Jul 24, 2012 4:05:02 PM org.apache.oodt.cas.pushpull.daemon.Daemon
sleep
>>>>>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to
>>>>>> sleep until Tue Jul 24 16:06:00 KST 2012
>>>>>> Jul 24, 2012 4:06:00 PM
>>>>>> org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning
>>>>>> INFO: Daemon with ID = 91131 was given permission to run
>>>>>> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser
>>>>>>
>>>>>> First of all, I would like test the pushpull framework  to move data
>>>>>> written in NetCDF from a remote site by a network protocols (like
ftp,
>>>>>> http, etc).
>>>>>
>>>>> Yep, sounds good.
>>>>>
>>>>>> I think  that there is a test suite which can be used to  check the
>>>>>> pushpull framework.
>>>>>> Can you give  configuration files(a set of XML files)  that I can
test
>>>>>> downloading  files from a  FTP site (public anonymous site) ?
>>>>>
>>>>> The examples in the resources directory that you found should work
>>>>> so we'll just need to figure out why they aren't working on your system.
>>>>>
>>>>>> or
>>>>>> Please let me know  a reference implementation of pushpull framework
>>>>>> that I can use to move NetCDF files.
>>>>>
>>>>> Thanks we'll figure this out.
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Chris Mattmann, Ph.D.
>>>>> Senior Computer Scientist
>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>> Office: 171-266B, Mailstop: 171-246
>>>>> Email: chris.a.mattmann@nasa.gov
>>>>> WWW:   http://sunset.usc.edu/~mattmann/
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW:   http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Mime
View raw message