Return-Path: X-Original-To: apmail-oodt-dev-archive@www.apache.org Delivered-To: apmail-oodt-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D0C109326 for ; Thu, 12 Apr 2012 11:47:23 +0000 (UTC) Received: (qmail 65696 invoked by uid 500); 12 Apr 2012 11:47:23 -0000 Delivered-To: apmail-oodt-dev-archive@oodt.apache.org Received: (qmail 65668 invoked by uid 500); 12 Apr 2012 11:47:23 -0000 Mailing-List: contact user-help@oodt.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@oodt.apache.org Delivered-To: mailing list user@oodt.apache.org Received: (qmail 65660 invoked by uid 99); 12 Apr 2012 11:47:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Apr 2012 11:47:23 +0000 X-ASF-Spam-Status: No, hits=3.1 required=5.0 tests=HK_RANDOM_ENVFROM,HK_RANDOM_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lmzxq.tom@gmail.com designates 209.85.217.171 as permitted sender) Received: from [209.85.217.171] (HELO mail-lb0-f171.google.com) (209.85.217.171) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Apr 2012 11:47:16 +0000 Received: by lbon10 with SMTP id n10so340855lbo.16 for ; Thu, 12 Apr 2012 04:46:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=oZrCy4m3ZdAmCWpSWd2JuNPNhz6ZfieSzDJHWnFZeKY=; b=Fn52NyvyWJqaUaayJxpVHyW9KeZ+VepLqKyU2qmUTxyKkBhgJsTr7z04/lPrwm3kYt RnPLuBMJ2mXoN6e/Xb1mEpT2DQBLxux45OH6pcAy7CTkCy5ZRDrAN85HGQ8nBCIfGtvy lMm9Oo/TG6tfj5Xs/B7tWaJ/WMGor8GQzCnuaECffhyRWVTbwsyJzXmEmCVXucNi0AmG VZW3iKYeyUvo9xlTMAs8Z3jGlZPhhgxOPYVvUqGP7nwhv8oqRpxcOBoCykFpLx4/g5tk 2R7Yl8iEkEvUehFSX4lbxdFvip2tR7RD/FZbWGUezimKNo1JQu5Y0zy+eCJcHIKCepwS RJqw== MIME-Version: 1.0 Received: by 10.112.28.6 with SMTP id x6mr253981lbg.46.1334231215779; Thu, 12 Apr 2012 04:46:55 -0700 (PDT) Received: by 10.112.69.75 with HTTP; Thu, 12 Apr 2012 04:46:55 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 Apr 2012 13:46:55 +0200 Message-ID: Subject: Re: query syntax From: Thomas Bennett To: user@oodt.apache.org Content-Type: multipart/alternative; boundary=bcaec554db7caa318904bd79e97f --bcaec554db7caa318904bd79e97f Content-Type: text/plain; charset=ISO-8859-1 Hey Paul, Thanks for the reply! That explains it then. Useful tool - I'm glad this made its way into the code base :). I'll create a JIRA issue and patch. Cheers, Tom On 12 April 2012 12:57, Ramirez, Paul M (388J) wrote: > Hey Tom, > > To follow up on this lets enter a Jira issue and the patch should be to > that outer " if (metadata.getMetadata("UUID") != null) {" and then > change the logging messages there to have > metadata.getMetadata("CAS.ProductId"). In addition, the else part of that > statement can be dropped. > > I also found one more place where the uuid appears and that is in the > indeMetFile method. Here this should be updated to > metadata.getMetadata("CAS.ProductId"). > > Thanks, > Paul > > From: "Ramirez, Paul M (388J)" > Date: Thu, 12 Apr 2012 03:32:27 -0700 > To: "" > Cc: "user@oodt.apache.org" > Subject: Re: query syntax > > This UUID stuff should have been dropped when it was ported to Apache. > This was a project specific thing where the team that developed it had a > different unique id (UUID). > > --Paul > > On Apr 11, 2012, at 6:06 PM, "Thomas Bennett" > wrote: > > Hey Chris, > > Here's a basic summary of what I've discovered so far getting my > metadata into a Solr index... > > SolrIndexer queries metadata for a single or multiple products from a > File Manager and put them into a Solr index. It does this by an http POST > to solr/update. Very cool. > > Because it's xml on the wire, its really easy to tcpdump and view the > actual POST and see what the content is :). This is how I discovered that > the indexAll was not updating solr (see below). > > Okay so for a simple test, I set up the following mapping, since these > fields already exist in the Solr schema.xml file: > > map.ExperimentID=id > map.Observer=author > map.Description=description > > Note: It seems a good idea to have an "id" field. If you look in the > solr scheme.xml file, I saw the following configuration by default: > > > id > > Okay, so I finally ran the SolrIndexer with the -all. > > This piece of code (starting at line 209 of SolrIndexer.java) was never > executed. It's checking for metadata "UUID". I don't have this metadata > key. > > if (metadata.getMetadata("UUID") != null) { > > if (metadata.getMetadata("Deleted") == null > > || !"true".equals(metadata.getMetadata("Deleted"))) { > > try { > > server.add(this.getSolrDocument(metadata)); > > server.commit(); > > LOG.info("Indexed " + metadata.getMetadata("UUID")); > > } catch (Exception e) { > > LOG.severe("Could not index " + metadata.getMetadata( > "UUID") > > + " " + e.getMessage()); > > } > > } else { > > LOG.info("Skipping Deleted: " + metadata.getMetadata( > "UUID")); > > } > > } > Question: Deleted looks optional, so no problem there, but what should I > be doing with UUID? I've changed this to be CAS.ProductId in the code, just > to check nothing else was causing a problem. I thought it would be worth > asking why UUID is specified... > > I've tried the single product index, which works just fine. > > Cheers, > Tom > > On 30 March 2012 20:38, Mattmann, Chris A (388J) < > chris.a.mattmann@jpl.nasa.gov> wrote: > >> No worries...based on your email below, did you update the Solr schema? I >> think you need >> to tell it to allow for all fields, by uncommenting the dynamicField >> block in the schema.xml file >> to say "allow all fields" to come through, or modify the schema.xml in >> Solr's conf directory to >> specify which fields to send to Solr. >> >> Does that make sense? I can provide more detailed help and will follow up >> later, but >> see if that helps for now. >> >> Cheers, >> Chris > > --bcaec554db7caa318904bd79e97f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hey Paul,

Thanks for the reply!

That explains it then. Useful tool - I'm glad this made its way into t= he code base :).

I'll create a JIRA issue and = patch.

Cheers,
Tom

On 12 April 2012 12:57, Ramirez, Paul M (388J) <paul.m.ramirez@= jpl.nasa.gov> wrote:
Hey Tom,

To follow up on this lets enter a Jira issue and the patch should be t= o that outer "=A0=A0if=A0(metadata.getMetadata(&quo= t;UUID") !=3D=A0null) {" and then change the = logging messages there to have metadata.getMetadata("CAS.ProductId"). In addition, the else par= t of that statement can be dropped.=A0

I also found one more place where the uuid appears and that is in the = indeMetFile method. Here this should be updated to metadata.getMetadata(&qu= ot;CAS.ProductId").

Thanks,
Paul

From: "Ramirez, Paul M (388J)&= quot; <= paul.m.ramirez@jpl.nasa.gov>
Date: Thu, 12 Apr 2012 03:32:27 -07= 00
To: "<user@oodt.apache.org>" <= ;user@oodt.apache= .org>
Cc: "user@oodt.apache.org" <user@oodt.apache.org= >
Subject: Re: query syntax

This UUID stuff should have been dropped when it was ported to Apache.= This was a project specific thing where the team that developed it had a d= ifferent unique id (UUID).=A0

--Paul=A0

On Apr 11, 2012, at 6:06 PM, "Thomas Bennett" <lmzxq.tom@gmail.com> = wrote:

Hey Chris,

Here's a basic summary of what I've discovered so far getting = my metadata into a Solr index...

SolrIndexer queries metadata for a single or multiple products from a = File Manager and put them into a Solr index. It does this by an http POST t= o solr/update. Very cool.

Because it's xml on the wire, its really easy to tcpdump and view = the actual POST and see what the content is :). This is how I discovered th= at the indexAll was not updating solr (see below).

Okay so for a simple test, I set up the following mapping, since these= fields already exist in the Solr schema.xml file:

map.ExperimentID=3Did
map.Observer=3Dauthor
map.Description=3Ddescription

Note: It seems a good idea =A0to have an "id" field. If you = look in the solr scheme.xml file, I saw the following configuration by defa= ult:

=A0<!-- Field to use to determine and enforce document uniqueness.= =A0
=A0 =A0 =A0 Unless this field is marked with required=3D"false&qu= ot;, it will be a required field
=A0 =A0-->
=A0<uniqueKey>id</uniqueKey>

Okay, so I finally ran the SolrIndexer with the -all.

This piece of code (starting at line 209 =A0of SolrIndexer.java) was n= ever executed. It's checking for metadata "UUID". =A0I don= 9;t have this metadata key.

=A0 =A0 =A0 =A0 =A0 =A0=A0if (metadata.getMetadata(= "UUID") !=3D null) {

=A0 =A0 =A0 =A0 =A0 =A0 =A0 if (metadata.getMetadata(= "Deleted") =3D=3D null

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 || !"true".eq= uals(metadata.getMetadata("Deleted"))) {

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 try {

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 server.add(this.getSolrDocument(metadata));

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 server.commit();

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 LOG.info("In= dexed " + metadata.getMetadata("UUID"));=

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } catch (Exception e) {

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 LOG.severe("= Could not index " + metadata.getMetadata("UUID"= )

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 + " "= + e.getMessage());

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 }

=A0 =A0 =A0 =A0 =A0 =A0 =A0 } else {

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 LOG.info("Skippi= ng Deleted: " + metadata.getMetadata("UUID"));

=A0 =A0 =A0 =A0 =A0 =A0 =A0 }

=A0 =A0 =A0 =A0 =A0 =A0 }

Question: Deleted looks optional, so no problem there, but what should= I be doing with UUID? I've changed this to be CAS.ProductId in the cod= e, just to check nothing else was causing a problem. =A0I thought it would = be worth asking why UUID is specified...

I've tried the single product index, which works just fine.

Cheers,
Tom=A0

On 30 March 2012 20:38, Mattmann, Chris A (388J)= <chri= s.a.mattmann@jpl.nasa.gov> wrote:
No worries...based on your email below, did you update the Solr schema? I t= hink you need
to tell it to allow for all fields, by uncommenting the dynamicField block = in the schema.xml file
to say "allow all fields" to come through, or modify the schema.x= ml in Solr's conf directory to
specify which fields to send to Solr.

Does that make sense? I can provide more detailed help and will follow up l= ater, but
see if that helps for now.

Cheers,
Chris

--bcaec554db7caa318904bd79e97f--