Return-Path: X-Original-To: apmail-oodt-dev-archive@www.apache.org Delivered-To: apmail-oodt-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2925ED554 for ; Thu, 19 Jul 2012 19:08:57 +0000 (UTC) Received: (qmail 85627 invoked by uid 500); 19 Jul 2012 19:08:57 -0000 Delivered-To: apmail-oodt-dev-archive@oodt.apache.org Received: (qmail 85571 invoked by uid 500); 19 Jul 2012 19:08:56 -0000 Mailing-List: contact dev-help@oodt.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@oodt.apache.org Delivered-To: mailing list dev@oodt.apache.org Delivered-To: moderator for dev@oodt.apache.org Received: (qmail 36487 invoked by uid 99); 19 Jul 2012 18:20:01 -0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yunh.kang@gmail.com designates 209.85.213.43 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=znZT95lCDP0ebHl0tuNvueLviiYZVQzlrAbDWquKuVI=; b=ftYQzgY0P4a990rWWHn5vSkClPT1cEvzC0s9kMKWHxoN0Dno7KrONoCxpxQCD4kurz vTY60M71+Qe2jNj/J/sqMR9/WJUhwvgekdaFBBsP5pwmJ37XSOD6eYeGGwTkak8y8nDn wzF5w21QptKBTWxlGZ8if6W1SkrPj5nauIXAicxm2FnzqgEG8oMRST8Z0zPyc2LlJvbB p/cNp/qlSHl33Kn15jCSAZrhLuIreyPf40rN80p0VDTSpOK7aG4zqUUY408+cnE7Toql nMcKsgFMmPbaylxbAeXTKqFtgLk0djh56UeXPTwqQ1w9B4+oQ01FXm0Jdg1wiHriK8Vj MEkA== MIME-Version: 1.0 In-Reply-To: <008731C8-06C2-4222-84AB-44F2A3ECBF7D@jpl.nasa.gov> References: <008731C8-06C2-4222-84AB-44F2A3ECBF7D@jpl.nasa.gov> Date: Fri, 20 Jul 2012 03:19:35 +0900 Message-ID: Subject: Re: Problem when running pushpull From: YunHee Kang To: "Mattmann, Chris A (388J)" Cc: dev@oodt.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org I just found a problem that is in a file "$CAS_PP_HOME/policy/ProtocolFactoryInfo.xml" in oodt 0.4. The problem is caused by missing classes related with an element "protocolFactory" in the file described above. To fix this problem, It needs to replace a wrong class name with a correct class name. For example there is no class named org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpClientFactory in any jar file in the directory, $CAS_PP_HOME/lib. To run the script $CAS_PP_HOME/bin/pushpull, CogJGlobusFtpClientFactory needs to be replaced with CogJGlobusFtpProtocolFactory. But I don't know how to replace org.apache.oodt.cas.protocol.local.LocalClientFactory. To be honest, I am not familiar with OODT. But up to now I am having a trouble to run the script $CAS_PP_HOME/bin/pushpull after I changed the file "$CAS_PP_HOME/policy/ProtocolFactoryInfo.xml". The following is the part of the error message when running the script. SEVERE: Failed to finish downloading per property files /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml : null I think that the error may occur the file RetrievalSetup.java in the following directory: ./apache-oodt-0.4-src/pushpull/src/main/java/org/apache/oodt/cas/pushpull/retrievalsystem/RetrievalSetup.java Please let me know how to fix the problem. yhkang@yunhee-kisti:~/oodt/cas-pushpull/bin$ ./pushpull Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure INFO: Configuring DaemonLauncher. . . Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadProtocolTypes INFO: Associating protocol types with ProtocolFactories . . . Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType INFO: Assiging protocol 'ftp' with class 'class org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocolFactory' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType INFO: Assiging protocol 'http' with class 'class org.apache.oodt.cas.protocol.http.HttpProtocolFactory' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType INFO: Assiging protocol 'sftp' with class 'class org.apache.oodt.cas.protocol.sftp.JschSftpProtocolFactory' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType INFO: Assiging protocol 'imaps' with class 'class org.apache.oodt.cas.protocol.imaps.ImapsProtocolFactory' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadParserInfo INFO: Associating parsers with RetrievalMethods . . . Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo loadParserInfo INFO: Assiging parser 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser' with retrievalmethod 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo loadParserInfo INFO: Assiging parser 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.FileListParser' with retrievalmethod 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.ParserInfo loadParserInfo INFO: Assiging parser 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.ClassNoaaEmailParser' with retrievalmethod 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadIngester INFO: Configuring and building ingester: [org.apache.oodt.cas.filemgr.ingest.StdIngester]: data transfer: [org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory]: to ingest to file manager: [http://localhost:9000] Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadIngester INFO: Configuring Ingester cache: [] TRANSFER: org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.config.Config loadProductTypeDetection INFO: Loading product type detection file: /home/yhkang/oodt/cas-pushpull/policy/mimetypes.xml Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure INFO: Loading SiteInfo file '/home/yhkang/oodt/cas-pushpull/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml' Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons INFO: Launching Daemons . . . Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.Daemon registerRMIServer INFO: Created Daemon ID = 91131 on RMI registry port 9113 Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons INFO: Creating Daemon with ID = 1 Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning INFO: Daemon with ID = 91131 was given permission to run org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser Jul 20, 2012 3:06:44 AM org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup retrieveFiles INFO: Creating 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler' to download data files Jul 20, 2012 3:06:49 AM org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol INFO: Testing protocol org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol . . . this may take a few minutes . . . Jul 20, 2012 3:06:59 AM org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with protocol 'org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol' and username 'anonymous' Jul 20, 2012 3:06:59 AM org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup retrieveFiles SEVERE: Failed to finish downloading per property files /home/yhkang/oodt/cas-pushpull/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml : null Jul 20, 2012 3:06:59 AM org.apache.oodt.cas.pushpull.daemon.Daemon sleep INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to sleep until Fri Jul 20 03:09:00 KST 2012 Thanks in advance, Best Regards, Yunhee Kang 2012/7/19 Mattmann, Chris A (388J) : > Hi YunHee, > > Thanks for your email! Would you be OK with me CC'ing dev@oodt.apache.org > and user@oodt.apache.org in my reply to you so that the rest of the community > can benefit? > > Thank you! > > Cheers, > Chris > > On Jul 19, 2012, at 1:51 AM, YunHee Kang wrote: > >> Hi Dr. Mattmann, >> >> My name is Yunhee Kang. >> I am working as an assistent professor at Baekseok university in Korea. >> >> I installed oodt0.4 to build an application for aggregating dataset >> genereated from a remote site. >> The dataset is the result of a climent model simulation, which is >> composed of files written in netCDF format. >> >> But it was not working while I ran the script pushpull. >> The follow is a message I got. >> Let me know how to handle this problem. >> >> ./pushpull >> Jul 19, 2012 4:13:17 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure >> INFO: Configuring DaemonLauncher. . . >> Jul 19, 2012 4:13:18 PM org.apache.oodt.cas.pushpull.config.Config >> loadProtocolTypes >> INFO: Associating protocol types with ProtocolFactories . . . >> Jul 19, 2012 4:13:18 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure >> SEVERE: Failed to configure DaemonLauncher : [Config] Failed to get >> properties from properties file : [Config] Failed to load >> ProtocolFactory config file >> /Users/yunheekang/oodt/cas-pushpull/policy/ProtocolFactoryInfo.xml : >> [ProtocolInfo] Failed to load ProtocolFactory info for protocol types >> : org.apache.oodt.cas.protocol.ftp.CogJGlobusFtpClientFactory >> Jul 19, 2012 4:13:18 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons >> INFO: Launching Daemons . . . >> Exception in thread "main" java.lang.NullPointerException >> at org.apache.oodt.cas.pushpull.daemon.DaemonLauncher.launchDaemons(DaemonLauncher.java:116) >> at org.apache.oodt.cas.pushpull.daemon.DaemonLauncher.main(DaemonLauncher.java:183) >> >> >> >> After I got the problem menetioned above, I installed oodt0.3. But I >> got the following error messge related with the ftp connections: >> - SEVERE: Protocol >> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed >> compatibility tes >> - org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect >> >> Can I get a ftp client that is running well on oodt 0.3 ? >> >> >> ./pushpull >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure >> INFO: Configuring DaemonLauncher. . . >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config >> loadProtocolTypes >> INFO: Associating protocol types with ProtocolFactories . . . >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'ftp' with class 'class >> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'ftp' with class 'class >> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClientFactory' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'http' with class 'class >> org.apache.oodt.cas.pushpull.protocol.http.HttpClientFactory' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'sftp' with class 'class >> org.apache.oodt.cas.pushpull.protocol.sftp.JschSftpClientFactory' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'imaps' with class 'class >> org.apache.oodt.cas.pushpull.protocol.imaps.ImapsClientFactory' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.config.ProtocolInfo addClassForType >> INFO: Assiging protocol 'file' with class 'class >> org.apache.oodt.cas.pushpull.protocol.local.LocalClientFactory' >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config >> loadParserInfo >> INFO: Associating parsers with RetrievalMethods . . . >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo >> loadParserInfo >> INFO: Assiging parser >> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser' >> with retrievalmethod >> 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler' >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo >> loadParserInfo >> INFO: Assiging parser >> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.FileListParser' >> with retrievalmethod >> 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever' >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.ParserInfo >> loadParserInfo >> INFO: Assiging parser >> 'org.apache.oodt.cas.pushpull.filerestrictions.parsers.ClassNoaaEmailParser' >> with retrievalmethod >> 'org.apache.oodt.cas.pushpull.retrievalmethod.ListRetriever' >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config loadIngester >> INFO: Configuring and building ingester: >> [org.apache.oodt.cas.filemgr.ingest.StdIngester]: data transfer: >> [org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory]: >> to ingest to file manager: [http://localhost:9000] >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config loadIngester >> INFO: Configuring Ingester cache: [] >> TRANSFER: org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.config.Config >> loadProductTypeDetection >> INFO: Loading product type detection file: >> /Users/yunheekang/oodt/cas-pushpull/policy/mimetypes.xml >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher configure >> INFO: Loading SiteInfo file >> '/Users/yunheekang/oodt/cas-pushpull/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml' >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons >> INFO: Launching Daemons . . . >> Jul 19, 2012 4:30:29 PM org.apache.oodt.cas.pushpull.daemon.Daemon >> registerRMIServer >> INFO: Created Daemon ID = 91131 on RMI registry port 9113 >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonLauncher launchDaemons >> INFO: Creating Daemon with ID = 1 >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.daemon.DaemonManager setAsRunning >> INFO: Daemon with ID = 91131 was given permission to run >> org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser >> Jul 19, 2012 4:30:29 PM >> org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup >> retrieveFiles >> INFO: Creating 'org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler' >> to download data files >> Jul 19, 2012 4:30:36 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >> INFO: Testing protocol >> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient . . . >> this may take a few minutes . . . >> Jul 19, 2012 4:30:36 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME >> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true' >> Jul 19, 2012 4:30:39 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >> SEVERE: Protocol >> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClient failed >> compatibility test : [CogJGlobusFtpClient] Failed to get list of files >> : Custom message: Could not create FileInfo [Nested exception >> message: Custom message: Invalid number of tokens in the list reply >> [ACRIM_III:]] >> Jul 19, 2012 4:30:39 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler >> getAppropriateProtocolBySite >> WARNING: ProtocolFactory >> org.apache.oodt.cas.pushpull.protocol.ftp.CogJGlobusFtpClientFactory >> is not compatible with server at ftp://l4ftl01.larc.nasa.gov >> Jul 19, 2012 4:30:44 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler isOkProtocol >> INFO: Testing protocol >> org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient . . . >> this may take a few minutes . . . >> Jul 19, 2012 4:30:44 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME >> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true' >> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToROOT >> INFO: Changing to ROOT directory '/' >> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cdToHOME >> INFO: Changing to HOME directory 'ftp://l4ftl01.larc.nasa.gov/ isDir=true' >> Jul 19, 2012 4:30:45 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler connect >> INFO: Successfully connected to ftp://l4ftl01.larc.nasa.gov with >> protocol 'org.apache.oodt.cas.pushpull.protocol.ftp.CommonsNetFtpClient' >> and username 'anonymous' >> Jul 19, 2012 4:30:45 PM org.apache.oodt.cas.pushpull.protocol.Protocol cd >> INFO: Changing to directory >> 'ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005 isDir=true' >> PageSize: 8 PageLoc: 0 >> FileList size: 897 >> Jul 19, 2012 4:30:48 PM >> org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect >> INFO: Disconnecting protocol from ftp://l4ftl01.larc.nasa.gov >> Jul 19, 2012 4:30:48 PM org.apache.oodt.cas.pushpull.daemon.Daemon sleep >> INFO: Daemon with ID = 91131 on RMI registry port 9113 is going to >> sleep until Thu Jul 19 16:33:00 YAKST 2012 >> >> >> Thanks in advance, >> >> Best Regards, >> >> Yunhee Kang > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: chris.a.mattmann@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > Phone: +1 (818) 354-8810 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >