oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YunHee Kang <yunh.k...@gmail.com>
Subject Re: Problem when running pushpull
Date Thu, 19 Jul 2012 18:19:35 GMT
I just found a problem that is in a file
"$CAS_PP_HOME/policy/ProtocolFactoryInfo.xml" in  oodt 0.4.

The problem is caused by missing classes related with an element
"protocolFactory" in the file described above.
To fix this problem,  It needs to replace a wrong class name with  a
correct class name.
For example  there is no class named
org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpClientFactory in any jar
file in the directory,  $CAS_PP_HOME/lib.  To run the script
$CAS_PP_HOME/bin/pushpull,   CogJGlobusFtpClientFactory  needs to be
replaced with CogJGlobusFtpProtocolFactory.

But I don't know how to replace
org.apache.oodt.cas.protocol.local.LocalClientFactory.
To be honest, I am not familiar with OODT.


 <protocol type="ftp">
        <!--protocolFactory
class="org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpClientFactory"/-->
        <!-- cas-protocol-ftp-0.4.jar -->
        <protocolFactory
class="org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpProtocolFactory"/>
        <!--protocolFactory
class="org.apache.oodt.cas.protocol.ftp.CommonsNetFtpClientFactory"/-->
        <protocolFactory
class="org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocolFactory"/>
    </protocol>

 <protocol type="http">
        <!--protocolFactory
class="org.apache.oodt.cas.protocol.http.HttpClientFactory"/-->
        <!-- cas-protocol-http-0.4.jar-->
        <protocolFactory
class="org.apache.oodt.cas.protocol.http.HttpProtocolFactory"/>
    </protocol>

    <protocol type="sftp">
        <!--protocolFactory
class="org.apache.oodt.cas.protocol.sftp.JschSftpClientFactory"/-->
        <!-- cas-protocol-sftp-0.4.jar -->
        <protocolFactory
class="org.apache.oodt.cas.protocol.sftp.JschSftpProtocolFactory"/>
    </protocol>

    <protocol type="imaps">
        <!--protocolFactory
class="org.apache.oodt.cas.protocol.imaps.ImapsClientFactory"/ -->
        <!-- cas-protocol-sftp-0.4.jar -->
        <protocolFactory
class="org.apache.oodt.cas.protocol.imaps.ImapsProtocolFactory"/>
    </protocol>

    <!--protocol type="file">
        <protocolFactory
class="org.apache.oodt.cas.protocol.local.LocalClientFactory"/>
    </protocol>
    -->


But  up to now I am having a trouble to run the script
$CAS_PP_HOME/bin/pushpull  after I changed the file
"$CAS_PP_HOME/policy/ProtocolFactoryInfo.xml".

The following is the part of the error message when running the script.
SEVERE: Failed to finish downloading per property files
/home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
: null

I think that the error may occur the file RetrievalSetup.java in the
following directory:
./apache-oodt-0.4-src/pushpull/src/main/java/org/apache/oodt/cas/pushpull/retrievalsystem/RetrievalSetup.java

Please let me know how to fix the problem.


yhkang@yunhee-kisti:~/oodt/cas-pushpull/bin$ ./pushpull
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
INFO: Configuring DaemonLauncher. . .
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config
loadProtocolTypes
INFO: Associating protocol types with ProtocolFactories . . .
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
INFO: Assiging protocol 'ftp' with class 'class
org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocolFactory'
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
INFO: Assiging protocol 'http' with class 'class
org.apache.oodt.cas.protocol.http.HttpProtocolFactory'
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
INFO: Assiging protocol 'sftp' with class 'class
org.apache.oodt.cas.protocol.sftp.JschSftpProtocolFactory'
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
INFO: Assiging protocol 'imaps' with class 'class
org.apache.oodt.cas.protocol.imaps.ImapsProtocolFactory'
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config
loadParserInfo
INFO: Associating parsers with RetrievalMethods . . .
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo
loadParserInfo
INFO: Assiging parser
'org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser'
with retrievalmethod
'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler'
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo
loadParserInfo
INFO: Assiging parser
'org.apache.oodt.cas.pushpull.filerestrictions.parsers.FileListParser'
with retrievalmethod
'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever'
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo
loadParserInfo
INFO: Assiging parser
'org.apache.oodt.cas.pushpull.filerestrictions.parsers.ClassNoaaEmailParser'
with retrievalmethod
'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever'
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadIngester
INFO: Configuring and building ingester:
[org.apache.oodt.cas.filemgr.ingest.StdIngester]: data transfer:
[org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory]:
to ingest to file manager: [http://localhost:9000]
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadIngester
INFO: Configuring Ingester cache: []
TRANSFER: org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config
loadProductTypeDetection
INFO: Loading product type detection file:
/home/yhkang/oodt/cas-pushpull/policy/mimetypes.xml
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
INFO: Loading SiteInfo file
'/home/yhkang/oodt/cas-pushpull/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml'
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons
INFO: Launching Daemons . . .
Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.Daemon
registerRMIServer
INFO: Created Daemon ID = 91131 on RMI registry port 9113
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons
INFO: Creating Daemon with ID = 1
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning
INFO: Daemon with ID = 91131 was given permission to run
org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser
Jul 20, 2012 3:06:44 AM
org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup
retrieveFiles
INFO: Creating 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler'
to download data files
Jul 20, 2012 3:06:49 AM
org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
INFO: Testing protocol
org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . . this may
take a few minutes . . .
Jul 20, 2012 3:06:59 AM
org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
protocol 'org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol' and
username 'anonymous'
Jul 20, 2012 3:06:59 AM
org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup
retrieveFiles
SEVERE: Failed to finish downloading per property files
/home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml
: null
Jul 20, 2012 3:06:59 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep
INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to
sleep until Fri Jul 20 03:09:00 KST 2012



Thanks in advance,

Best Regards,
Yunhee Kang

2012/7/19 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
> Hi YunHee,
>
> Thanks for your email! Would you be OK with me CC'ing dev@oodt.apache.org
> and user@oodt.apache.org in my reply to you so that the rest of the community
> can benefit?
>
> Thank you!
>
> Cheers,
> Chris
>
> On Jul 19, 2012, at 1:51 AM, YunHee Kang wrote:
>
>> Hi  Dr. Mattmann,
>>
>> My name is Yunhee Kang.
>> I am working  as an assistent professor  at Baekseok university in Korea.
>>
>> I installed oodt0.4 to build an application for aggregating dataset
>> genereated from a remote site.
>> The dataset is the result of a climent model simulation, which is
>> composed of files written in netCDF format.
>>
>> But it was not working while I ran the script pushpull.
>> The follow is a message I got.
>> Let me know how to handle this problem.
>>
>> ./pushpull
>> Jul 19, 2012 4:13:17 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
>> INFO: Configuring DaemonLauncher. . .
>> Jul 19, 2012 4:13:18 PM org.apache.oodt.cas.pushpull.config.Config
>> loadProtocolTypes
>> INFO: Associating protocol types with ProtocolFactories . . .
>> Jul 19, 2012 4:13:18 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
>> SEVERE: Failed to configure DaemonLauncher : [Config] Failed to get
>> properties from properties file : [Config] Failed to load
>> ProtocolFactory config file
>> /Users/yunheekang/oodt/cas-pushpull/policy/ProtocolFactoryInfo.xml :
>> [ProtocolInfo] Failed to load ProtocolFactory info for protocol types
>> : org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpClientFactory
>> Jul 19, 2012 4:13:18 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons
>> INFO: Launching Daemons . . .
>> Exception in thread "main" java.lang.NullPointerException
>>       at org.apache.oodt.cas.pushpull.daemon.DaemonLauncher.launchDaemons(DaemonLauncher.java:116)
>>       at org.apache.oodt.cas.pushpull.daemon.DaemonLauncher.main(DaemonLauncher.java:183)
>>
>>
>>
>> After I got the problem menetioned above, I installed oodt0.3.  But I
>> got the following error messge related with the ftp connections:
>> - SEVERE: Protocol
>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed
>> compatibility tes
>> - org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect
>>
>> Can I get a ftp client that is running well  on oodt 0.3 ?
>>
>>
>> ./pushpull
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
>> INFO: Configuring DaemonLauncher. . .
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config
>> loadProtocolTypes
>> INFO: Associating protocol types with ProtocolFactories . . .
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'ftp' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'ftp' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClientFactory'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'http' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.http.HttpClientFactory'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'sftp' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.sftp.JschSftpClientFactory'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'imaps' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.imaps.ImapsClientFactory'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType
>> INFO: Assiging protocol 'file' with class 'class
>> org.apache.oodt.cas.pushpull.protocol.local.LocalClientFactory'
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config
>> loadParserInfo
>> INFO: Associating parsers with RetrievalMethods . . .
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo
>> loadParserInfo
>> INFO: Assiging parser
>> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser'
>> with retrievalmethod
>> 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler'
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo
>> loadParserInfo
>> INFO: Assiging parser
>> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.FileListParser'
>> with retrievalmethod
>> 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever'
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo
>> loadParserInfo
>> INFO: Assiging parser
>> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.ClassNoaaEmailParser'
>> with retrievalmethod
>> 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever'
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config loadIngester
>> INFO: Configuring and building ingester:
>> [org.apache.oodt.cas.filemgr.ingest.StdIngester]: data transfer:
>> [org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory]:
>> to ingest to file manager: [http://localhost:9000]
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config loadIngester
>> INFO: Configuring Ingester cache: []
>> TRANSFER: org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config
>> loadProductTypeDetection
>> INFO: Loading product type detection file:
>> /Users/yunheekang/oodt/cas-pushpull/policy/mimetypes.xml
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure
>> INFO: Loading SiteInfo file
>> '/Users/yunheekang/oodt/cas-pushpull/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml'
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons
>> INFO: Launching Daemons . . .
>> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.daemon.Daemon
>> registerRMIServer
>> INFO: Created Daemon ID = 91131 on RMI registry port 9113
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons
>> INFO: Creating Daemon with ID = 1
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning
>> INFO: Daemon with ID = 91131 was given permission to run
>> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser
>> Jul 19, 2012 4:30:29 PM
>> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup
>> retrieveFiles
>> INFO: Creating 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler'
>> to download data files
>> Jul 19, 2012 4:30:36 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>> INFO: Testing protocol
>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient . . .
>> this may take a few minutes . . .
>> Jul 19, 2012 4:30:36 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME
>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true'
>> Jul 19, 2012 4:30:39 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>> SEVERE: Protocol
>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed
>> compatibility test : [CogJGlobusFtpClient] Failed to get list of files
>> :  Custom message: Could not create FileInfo [Nested exception
>> message:  Custom message: Invalid number of tokens in the list reply
>> [ACRIM_III:]]
>> Jul 19, 2012 4:30:39 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler
>> getAppropriateProtocolBySite
>> WARNING: ProtocolFactory
>> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory
>> is not compatible with server at ftp://l4ftl01.larc.nasa.gov
>> Jul 19, 2012 4:30:44 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol
>> INFO: Testing protocol
>> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient . . .
>> this may take a few minutes . . .
>> Jul 19, 2012 4:30:44 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME
>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true'
>> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToROOT
>> INFO: Changing to ROOT directory '/'
>> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME
>> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true'
>> Jul 19, 2012 4:30:45 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect
>> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with
>> protocol 'org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient'
>> and username 'anonymous'
>> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cd
>> INFO: Changing to directory
>> 'ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005 isDir=true'
>> PageSize: 8 PageLoc: 0
>> FileList size: 897
>> Jul 19, 2012 4:30:48 PM
>> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect
>> INFO: Disconnecting protocol from ftp://l4ftl01.larc.nasa.gov
>> Jul 19, 2012 4:30:48 PM org.apache.oodt.cas.pushpull.daemon.Daemon sleep
>> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to
>> sleep until Thu Jul 19 16:33:00 YAKST 2012
>>
>>
>> Thanks in advance,
>>
>> Best Regards,
>>
>> Yunhee Kang
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> Phone: +1 (818) 354-8810
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Mime
View raw message