oodt-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris A. Mattmann (JIRA)" <j...@apache.org>
Subject [jira] Commented: (OODT-72) Unable to set Metadata based off Product Versioning during Product ingestion
Date Wed, 08 Dec 2010 15:22:01 GMT

    [ https://issues.apache.org/jira/browse/OODT-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969339#action_12969339
] 

Chris A. Mattmann commented on OODT-72:
---------------------------------------

bq. your counter example creates a xml-rpc client connection within an xml-rpc client connection
. . . out of a 100 concurrent threads allowed to a filemgr server you are now making each
ingest utilize 2 of those threads

1. 100 is *not* a hard limit. See here:

http://ws.apache.org/xmlrpc/xmlrpc2/apidocs/org/apache/xmlrpc/XmlRpc.html#setMaxThreads%28int%29

It's totally configurable.

2. Even with #1 being true, we also aren't strictly limited to that number. One possible solution
is simply to stand up another filemgr (or N filemgrs) to load balance the # of threads available,
but make them all talk to the same backend catalog (e.g., Oracle or some DB). 

So, no, I don't agree with you that in all cases we've used 2 of only 100 threads here.

bq. you metextractor is faulty: Product p = Product.blankProduct(); . . . what if your Versioner
uses product.getProductId()? . . . you FileLocation metadata field is not going to be accurate

If the Versioner used product.getProductId(), then I would have flowed it through in my example.
My example didn't make your assumption. 

bq. yes . . . either on the client or the server . . . it will be dynamically loaded . . .
otherwise the product will never get versioned

I'll agree to disagree with you on this.

bq. on the server side in the ingestProduct method, modify it so that it clones the Metadata
object before calling the Versioner

I'm against cloning because it creates another Object when it's not needed.


> Unable to set Metadata based off Product Versioning during Product ingestion
> ----------------------------------------------------------------------------
>
>                 Key: OODT-72
>                 URL: https://issues.apache.org/jira/browse/OODT-72
>             Project: OODT
>          Issue Type: Improvement
>          Components: file manager
>    Affects Versions: 0.1-incubating
>         Environment: none
>            Reporter: Brian Foster
>            Assignee: Brian Foster
>            Priority: Minor
>             Fix For: 0.2
>
>
> Ingest order:
>   - add product to catalog
>   - metextractors run
>   - add metadata to catalog
>   - versioner runs (on client or on server)
>   - add references to catalog
> This prevents Metadata elements such as FileLocation from being set based off Product
References since metadata is added to catalog pre-versioning
> - A known work-around is to run the versioner twice (once in a MetExtractor and again
when the Versioner is run in the ingest process) . . . however this is very much a work-around
and setting elements such as FileLocation is a very common functionality.
> I purpose that Versioning take place before MetExtractors run.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message