lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Inger (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1612) Use java.sql.Array to populate Multivalue field in DataImportHandler
Date Wed, 02 Dec 2009 19:39:20 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784970#action_12784970
] 

Matt Inger commented on SOLR-1612:
----------------------------------

Thanks for the suggestion, and I'm in the process of doing that, but I have noticed a few
things wrong with the general custom transformer implementation:

1.  If I package my custom transformer up in it's own jar file, it seems the ClassLoader is
unable to find the Transformer class from dataimporter jar file, so I have
     to add that to my example/example-DIH/lib directory as well in order to get it to find
the Transformer class.

2.  The implementation of the loading of the Transformer class has a bug in it in regards
to how it deals with Simple transformers.  It seems if you extend the Transformer class, you
still have to implement the simpler version of the transformRow method:

{code}
public Object transformRow(Map<String,Object>);
{code}

This slice of code, which is in EntityProcessorWrapper, around line 100, is the root of the
problem.

{code}
      try {
        Class clazz = DocBuilder.loadClass(trans, context.getSolrCore());
        if (clazz.newInstance() instanceof Transformer) {
          transformers.add((Transformer) clazz.newInstance());
        } else {
          final Method meth = clazz.getMethod(TRANSFORM_ROW, Map.class);
          if (meth == null) {
            String msg = "Transformer :"
                    + trans
                    + "does not implement Transformer interface or does not have a transformRow(Map
m)method";
            log.error(msg);
            throw new DataImportHandlerException(
                    SEVERE, msg);
          }
          transformers.add(new ReflectionTransformer(meth, clazz, trans));
        }
{code}

In particular, the class to "clazz.getMethod" DOES NOT return a null value if the method does
not exist (at least in JDK 1.5).  It in fact, throws a NoSuchMethodException, which means
that I have to have the simpler method no matter whether i'm using it or not.



> Use java.sql.Array to populate Multivalue field in DataImportHandler
> --------------------------------------------------------------------
>
>                 Key: SOLR-1612
>                 URL: https://issues.apache.org/jira/browse/SOLR-1612
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>    Affects Versions: 1.4
>            Reporter: Matt Inger
>             Fix For: 1.5
>
>         Attachments: SOLR-1612.patch
>
>
> I am attaching a patch to the JdbcDataSource class which allows the it to populate a
multiValued field from a query which returns a java.sql.Array type for a column value.
> In order to make this work, you need to do the following:
> 1.  In your schema.xml file, set the "multiValued" attribute to the value "true"
>          <field name=foo" type="string"  ... multiValued="true" />
> 2.  Have your sql query return a sql type of java.sql.Types.ARRAY.  How to do this is
very database dependent, but in oracle, you first create a type:
>              create or replace type FOO_TYPE IS TABLE OF VARCHAR2(255);
>       and then when you select the values, you have to use MULTISET and cast to the created
type (assuming here that FOO_PARENT_ID is the condition to
>       limit which foos belong to the current record):
>              CAST(MULTISET(SELECT FOO FROM FOOS WHERE FOO_PARENT=FOO_PARENT_ID) AS FOO_TYPE
) MY_FOOS
> The main advantage this has is it can save you a ton of queries over the current method
which will fire off an individual query for each document in order to retrieve the values
for each multiValued field.  This method executes much faster, and saves alot of database
resources.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message