Return-Path: X-Original-To: apmail-oodt-dev-archive@www.apache.org Delivered-To: apmail-oodt-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id ED4BF10B7E for ; Fri, 14 Nov 2014 04:20:04 +0000 (UTC) Received: (qmail 78973 invoked by uid 500); 14 Nov 2014 04:20:04 -0000 Delivered-To: apmail-oodt-dev-archive@oodt.apache.org Received: (qmail 78935 invoked by uid 500); 14 Nov 2014 04:20:04 -0000 Mailing-List: contact dev-help@oodt.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@oodt.apache.org Delivered-To: mailing list dev@oodt.apache.org Delivered-To: moderator for dev@oodt.apache.org Received: (qmail 48069 invoked by uid 99); 14 Nov 2014 01:40:35 -0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type; bh=JQkCgdfXXGgrcGDEnYgLK/LQzpel+vHrKQ31HUrsNAk=; b=TMvPZs9lGr1ggU7K/oS8iJtLu6IIkmT/okzjizajGUBd1jVSBBv7MVITXjA0P/IJuw RxuXTlkAmvsYfn5ocYRbZ9AZ5A8Lavvz2H10JpPa09a+cQ53rqHrEi5PEW4ARubj/N9y Xh6mIAFqJGyuyFOFEEYHJNY/57zFKmVI0wupZpHOJo0wLm4CmgIG+XZJLNqqW3qrdaJ3 oxMun7XpN2emhYWAB319CCuZM4rKGCmll0RHqXEvRxYctconp+eYTFqXyqqNvLQwkYIU TJbtAig5Jwvn1TUlwW/n2Hgx32RwTWG63p10PETAd7r/CgIi++t/gU8aUO3ejmtuIVsq sHiA== X-Gm-Message-State: ALoCoQkgHTZrZdpVMNT04rNoRALNB98133YEqOkJ9p5Jl2PWUP3rbUG0Zx04rYQrqrfeS9PM3lwk X-Received: by 10.180.79.196 with SMTP id l4mr3145253wix.72.1415929164061; Thu, 13 Nov 2014 17:39:24 -0800 (PST) Message-ID: <54655D49.40904@meteorite.bi> Date: Fri, 14 Nov 2014 01:39:21 +0000 From: Tom Barber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: dev@oodt.apache.org Subject: Re: more info for OODT-751 OPSUI Pages constantly expire References: <5231e3948c064ca7822f8a832e8c2a9f@aplex01.dom1.jhuapl.edu> <8f142c5b4e40464c845cc5942fdb71ab@aplex01.dom1.jhuapl.edu>,<5464D0F4.1070308@meteorite.bi> <79cc3a51e97b44ab879c483b9d568c3f@aplex01.dom1.jhuapl.edu> <54655B48.1030600@meteorite.bi> In-Reply-To: <54655B48.1030600@meteorite.bi> Content-Type: multipart/alternative; boundary="------------020109000101080508050506" X-Virus-Checked: Checked by ClamAV on apache.org --------------020109000101080508050506 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Hi Val, I tried a 0.7 OpsUI over 0.6 filemgr to see if that made a difference but it really doesn't like that. I can probably have a prod about over the weekend to see if I can figure out whats knackered. Don't despair, we'll get you your demo! ;) Tom On 14/11/14 01:30, Tom Barber wrote: > Okay well, I have some good news and some bad news (and its 1:30 am so > I'm off to bed in a moment, so this is a one shot email :) ) > > I have a stock 0.7 radix install with nothing in the filemanager and I > see what you see, its always been pretty picky and needs replacing > with something better, I keep prodding people about that, it will > happen one day ;) > > For some reason Wicket has decided a file that has not changed in 4 > years, now is the worst thing on the planet and it refuses to change > the Product information into something the browser can render. > Caused by: java.io.NotSerializableException: > org.apache.oodt.cas.filemgr.structs.ProductType > > My 0.7 install has no products in it. > > But, I have a 0.6 Radix install that a used for ApacheCon last year, I > haven't done much too it apart from put some products into the file > manager, and that works fine(mostly, some of the OpsUI stuff is just > endlessly broken). > > http://ibin.co/1h9uVtTs0XuX > http://ibin.co/1h9uge8eqLkU > http://ibin.co/1h9uowI0RaDO > > I've not changed anything in the old one, it just uses the GenericFile > type, so it does work. Just not all the time. > > Out of curiosity have you got any products ingested or are you running > on an empty repo? > > Tom > > > > On 14/11/14 00:52, Mallder, Valerie wrote: >> Hi Tom, >> >> I am running with the Radix installation of version 0.7. Although, when I tried running with version 0.6 opsui I did not do a full reinstall of the other oodt components that come with Radix too. >> >> Val >> >> >> >> Sent from my iPhone. >> ________________________________ >> From: Tom Barber >> Sent: Thursday, November 13, 2014 10:40:36 AM >> To:dev@oodt.apache.org >> Subject: Re: more info for OODT-751 OPSUI Pages constantly expire >> >> Hi Valerie, >> >> Is this with or without Radix? >> >> Tom >> >> On 13/11/14 13:10, Mallder, Valerie wrote: >>> For those of you who are using the pcs-opsui in an operational environment without getting expired pages please tell me what version you are using, the OS you are running it on, and the browser you are using. >>> >>> I need to demonstrate "something" to my project manager to show the usefulness of OODT in the Jedi instrument science data pipeline here at APL. And right now, I have nothing to "show" for my last few months of work. If someone can tell me a configuration using opsui that is working then maybe I can try to mimic that and get something useful up and running. So far, for me, versions 0.6, 0.7 and the current trunk are showing only expired pages. >>> >>> Thanks very much! >>> Val >>> >>> >>> >>> Sent from my iPhone. >>> ________________________________ >>> From: Mallder, Valerie >>> Sent: Tuesday, November 11, 2014 2:55:12 PM >>> To:dev@oodt.apache.org >>> Subject: more info for OODT-751 OPSUI Pages constantly expire >>> >>> Hi Chris, >>> >>> I know you are working on this, but I wanted to let you know that I tried to use version 0.6 and am having the same problem in version 0.6. But, I can't guarantee that I installed version 0.6 correctly so I would like to run this by you. Based on your email below here's what I did: >>> >>> 1. Downloaded pcs-opsui-0.76.war from: >>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ >>> and saved it in a new folder named $OODT_HOME/bin/opsui >>> >>> 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the following variables (this is output from my script): >>> Using OODT_BASE: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy >>> Using OODT_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy >>> Using OODT_TMPDIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp >>> Using FILEMGR_URL:http://localhost:9000 >>> Using WORKFLOW_URL:http://localhost:9001 >>> Using RESMGR_URL:http://localhost:9002 >>> Using WORKFLOW_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow >>> Using RESMGR_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr >>> Using CRAWLER_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler >>> Using TOMCAT_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat >>> Using PCS_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs >>> Using PGE_HOME: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge >>> Using PGE_JOBS_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs >>> Using FEI_DROP_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetry >>> Using JEDI_L0_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0 >>> Using JEDI_L2_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2 >>> Using ARCHIVE_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive >>> Using BACKUP_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met >>> Using FAILURE_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure >>> Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline >>> Using SNAPSHOT_DIR: /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs/snapshot >>> >>> (Then, I followed the steps at the bottom of:https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI) >>> >>> 3. Downloadedhttp://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml >>> And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml >>> >>> 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line from: >>> >>> To >>> >>> >>> 5. Killed any process with "tomcat" in its name to ensure that tomcat is not running. >>> >>> 6. Executed the following command to create a symbolic link: >>> cd $OODT_HOME/bin/opsui/ >>> ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml >>> >>> 7. Then, in my script that sets the environment variables, I added the following command to start tomcat. >>> exec "$OODT_BASE"/tomcat/bin/catalina.sh start >>> >>> 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script >>> >>> 9. Then, I started firefox and went to "localhost:8080/pcs-opsui" >>> >>> >>> >>> The home page comes up, and the PCS Status page comes up, but all other pages are expired. I should note that the "radix" installation of version 0.7 points me to localhost:8080/opsui, while step 10 of the instructions from the quick start guide of the wiki point me to localhost:8080/pcs-opsui. And, no matter which one I go to, the pages expire. >>> >>> I also tried clearing out my cache and browser history and restarting firefox and topcat, etc. and nothing seems to help. >>> >>> If I have gotten the version 0.6 up and running correctly, I thought you would be interested to know that I am seeing the error. If I did not get version 0.6 up and running, then I need more hints on how to do it. :) >>> >>> Thanks, >>> Val >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Valerie A. Mallder >>> New Horizons Deputy Mission System Engineer >>> Johns Hopkins University/Applied Physics Laboratory >>> >>> >>>> -----Original Message----- >>>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov] >>>> Sent: Friday, October 03, 2014 1:07 PM >>>> To:dev@oodt.apache.org >>>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly >>>> >>>> Awesome Val! :) >>>> >>>> I think you�re running into this: >>>> >>>> https://issues.apache.org/jira/browse/OODT-751 >>>> >>>> >>>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here: >>>> >>>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ >>>> >>>> >>>> Grab the WAR file and drop it into your favorite container. >>>> Make sure you have all of these environment variables installed >>>> *before* starting Tomcat or Jetty, etc.: >>>> >>>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI >>>> >>>> >>>> (see steps at bottom and replace 0.5 with 0.6) >>>> >>>> I�m working on a fix for OODT-751, at which point RADIX will be pretty buff. Next >>>> steps at that point: >>>> >>>> 1. Release 0.7 and then encourage folks to get started by using the Vagrant build, >>>> e.g., >>>> >>>> git clonehttps://github.com/apache/oodt cd vagrant/radix vagrant up >>>> >>>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. Work >>>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release in 0.8. >>>> 4. Conquer and win. >>>> >>>> Thanks! >>>> >>>> Cheers, >>>> Chris >>>> >>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> +++++ >>>> Chris Mattmann, Ph.D. >>>> Chief Architect >>>> Instrument Software and Science Data Systems Section (398) NASA Jet >>>> Propulsion Laboratory Pasadena, CA 91109 USA >>>> Office: 168-519, Mailstop: 168-527 >>>> Email:chris.a.mattmann@nasa.gov >>>> WWW:http://sunset.usc.edu/~mattmann/ >>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> +++++ >>>> Adjunct Associate Professor, Computer Science Department University of >>>> Southern California, Los Angeles, CA 90089 USA >>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> +++++ >>>> >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: , Valerie >>>> Reply-To:"dev@oodt.apache.org" >>>> Date: Friday, October 3, 2014 at 9:53 AM >>>> To:"dev@oodt.apache.org" >>>> Subject: Success! RE: how to use MetadataBasedFileVersioner properly >>>> >>>>> Hi Chris, >>>>> >>>>> Yes, that indeed fixed it! Thanks so much! I now have 18 engineering >>>>> files ingested. Whoo hoo!! >>>>> >>>>> Ok, so now, I would like to see what the opsui has to say about my >>>>> ingested files. And here comes probably a really stupid question. I am >>>>> using Firefox on a Redhat Linux box, and I'm not that familiar with >>>>> Firefox and it's settings, so this might be a browser setting issue. >>>>> When I startup the opsui and select "File Catalog Browse" it shows that >>>>> I have >>>>> 18 EngineeringFiles. Then, when I select File Catalog Browse from the >>>>> strip of options under the logo, I get a message saying the page has >>>>> expired. And here's the link that shows up in the address bar: >>>>> http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL >>>>> ist >>>>> ener:: >>>>> >>>>> And I get the page expired message for all of the options that I select. >>>>> Any idea's on this one?? >>>>> >>>>> In the meantime though, I will start playing with an action for post >>>>> ingestion success that simply makes another copy of all these >>>>> engineering files and puts them another folder for Level 0 files. >>>>> >>>>> Thanks, >>>>> Val >>>>> >>>>> >>>>> >>>>> >>>>> Valerie A. Mallder >>>>> New Horizons Deputy Mission System Engineer Johns Hopkins >>>>> University/Applied Physics Laboratory >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com] >>>>>> Sent: Friday, October 03, 2014 3:03 AM >>>>>> To:dev@oodt.apache.org >>>>>> Subject: Re: how to use MetadataBasedFileVersioner properly >>>>>> >>>>>> You?re almost there Val! >>>>>> >>>>>> Unfortunately Versioners right now aren?t configurable from product >>>>>> type policy (would be great to capture this in a JIRA issue, here: >>>>>> https://issues.apache.org/jira/browse/OODT). If they were, it would >>>>>> have picked up your >>>>> It?s been on my TODO list for a long time. >>>>>> >>>>>> Instead I created this: >>>>>> >>>>>> https://issues.apache.org/jira/browse/OODT-639 >>>>>> >>>>>> >>>>>> So you can amend your definition below (also note you always have to >>>>>> include /[Filename] at the end to get the filename you want). >>>>>> >>>>>> >>>>>> >>>>>> >>>>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/> >>>>>> The default product type for any kind of >>>>>> file. >>>>>> >>>>>> >>>>> >>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> value="ProductReceivedTime,ProductName,ProductId" /> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> filePathSpec >>>>>> /[YearDir]/[DoyDir]/[Filename] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> See if that fixes it! >>>>>> >>>>>> Cheers, >>>>>> Chris >>>>>> >>>>>> ------------------------ >>>>>> Chris Mattmann >>>>>> chris.mattmann@gmail.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: "Mallder, Valerie" >>>>>> Reply-To: >>>>>> Date: Thursday, October 2, 2014 at 2:56 PM >>>>>> To:"dev@oodt.apache.org" >>>>>> Subject: how to use MetadataBasedFileVersioner properly >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I am trying to use the MetadataBasedFileVersioner to store my files >>>>>>> in a subdirectory of the data/archive folder based on the values of >>>>>>> two metadata elements. Has anyone does this before, and if so, can >>>>>>> you give me some hints. >>>>>>> >>>>>>> My files need to be organized in subdirectories by year and day of >>>>>> year. >>>>>>> So, lets say the file name is 'myfile', the year is 2014, and the >>>>>>> doy of year is 002. Then, the end result that I am looking for is to >>>>>>> have the final location of my file be: >>>>>> 'data/archive/ops/eng/2014/002/myfile. >>>>>>> However, in my log file, the INFO messages indicate that the >>>>>>> generated final location reference is 'data/archive/ops/eng/myfile'. >>>>>>> And it doesn't include the year and day of year at all. And the >>>>>>> incorrect location leads to other errors. So I want to solve this >>>>>>> one first. Has anyone tried to do something like this before? >>>>>>> >>>>>>> Here is what I have done so far: >>>>>>> >>>>>>> Added two new elements to the .met file 'YearDir' and 'DoyDir'. >>>>>>> Added these new elements to the elements.xml file. >>>>>>> Added these new elements to the product in the >>>>>>> product-type-element-map.xml file. >>>>>>> And, I have made changes to the product-types.xml file, but here's >>>>>>> where I am not sure I've done this properly. >>>>>>> >>>>>>> Here's what product type definition looks like: >>>>>>> >>>>>>> >>>>>>> >>>>>> class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio >>>>>>> ner >>>>>>> "> >>>>>>> >>>>>>> >>>>>>> The default product type for any kind of >>>>>>> file. >>>>>>> >>>>>>> >>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> value="ProductReceivedTime,ProductName,ProductId" /> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks in advance for any help or ideas you might have! >>>>>>> >>>>>>> Valerie >>>>>>> >>>>>>> >>>>>>> >>>>>>> Valerie A. Mallder >>>>>>> >>>>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins >>>>>>> University/Applied Physics Laboratory >>>>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 >>>>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry) >>>>>>> >> -- >> *Tom Barber* | Technical Director >> >> meteorite bi >> *T:* +44 20 8133 3730 >> *W:*www.meteorite.bi | *Skype:* meteorite.consulting >> *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK >> > > > -- > *Tom Barber* | Technical Director > > meteorite bi > *T:* +44 20 8133 3730 > *W:* www.meteorite.bi | *Skype:* meteorite.consulting > *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 > 7YG, UK -- *Tom Barber* | Technical Director meteorite bi *T:* +44 20 8133 3730 *W:* www.meteorite.bi | *Skype:* meteorite.consulting *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK --------------020109000101080508050506--