lucene-pylucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Brown <peter.bro...@griffithuni.edu.au>
Subject Re: JCC SimilarityBase
Date Fri, 05 May 2017 02:27:44 GMT
Andi, you are a champion! :)
Thanks so much for your timely and helpful response.

Unfortunately, I had no luck adding the score function to the makefile. 
However, I was able to resolve the issue by defining the method as public.


Peter Brown - s2623892
School of Information and Communication Technology,
Griffith University - Gold Coast Campus

> On 5 May 2017, at 11:58 am, Andi Vajda <vajda@apache.org> wrote:
> 
> 
> On Fri, 5 May 2017, Peter Brown wrote:
> 
>> Hi all,
>> 
>> I was just wondering if anybody else has run into a similar issue before.
>> I?m trying to create a wrapper class for the org.apache.lucene.search.similarities.SimilarityBase
class within PyLucene (similar to the existing wrapper for PythonClassicSimilarity).
>> 
>> However, upon re-building and re-installing PyLucene, when trying to use the wrapper
class it fails with the following exception:
>> 
>> java.lang.UnsatisfiedLinkError: org.apache.pylucene.search.similarities.PythonSimilarityBase.score(Lorg/apache/lucene/search/similarities/BasicStats;FF)F
>> 
>> Upon examination of the wrap0*.cpp files, and comparing with PythonClassicSimilarity,
I found that the score method is not being generated. E.g.
>> 
>> static PyMethodDef t_PythonSimilarityBase__methods_[] = {
>>           DECLARE_METHOD(t_PythonSimilarityBase, cast_, METH_O | METH_CLASS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, instance_, METH_O | METH_CLASS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, finalize, METH_NOARGS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, pythonExtension, METH_VARARGS),
>>           { NULL, NULL, 0, NULL }
>>         };
>> 
>> and
>> 
>> JNINativeMethod methods[] = {
>>             { "pythonDecRef", "()V", (void *) t_PythonSimilarityBase_pythonDecRef0
},
>>             { "toString", "()Ljava/lang/String;", (void *) t_PythonSimilarityBase_toString1
},
>>           };
>> 
>> But it is present in the header file enum?
>> class PythonSimilarityBase : public ::org::apache::lucene::search::similarities::SimilarityBase
{
>>         public:
>>           enum {
>>             mid_init$_7353aa4eb69a6f70,
>>             mid_finalize_7353aa4eb69a6f70,
>>             mid_pythonDecRef_7353aa4eb69a6f70,
>>             mid_pythonExtension_aae05472994e7446,
>>             mid_pythonExtension_f41cd3dc7c7d5d78,
>>             mid_toString_5df7257cf25bf439,
>>             mid_score_a6c8466c4349b376,
>>             max_mid
>>           };
>> 
>> But not in the header file methods.
>> PythonSimilarityBase();
>> 
>>           void finalize() const;
>>           void pythonDecRef() const;
>>           jlong pythonExtension() const;
>>           void pythonExtension(jlong) const;
>>           ::java::lang::String toString() const;
>> 
>> 
>> Compared to
>> static PyMethodDef t_PythonClassicSimilarity__methods_[] = {
>>           DECLARE_METHOD(t_PythonClassicSimilarity, cast_, METH_O | METH_CLASS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, instance_, METH_O | METH_CLASS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, coord, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, finalize, METH_NOARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, idf, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, idfExplain, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, lengthNorm, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, pythonExtension, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, queryNorm, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, sloppyFreq, METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, tf, METH_VARARGS),
>>           { NULL, NULL, 0, NULL }
>>         };
>> 
>> and
>> 
>> JNINativeMethod methods[] = {
>>             { "coord", "(II)F", (void *) t_PythonClassicSimilarity_coord0 },
>>             { "idf", "(JJ)F", (void *) t_PythonClassicSimilarity_idf1 },
>>             { "idfExplain", "(Lorg/apache/lucene/search/CollectionStatistics;[Lorg/apache/lucene/search/TermStatistics;)Lorg/apache/lucene/search/Explanation;",
(void *) t_PythonClassicSimilarity_idfExplain2 },
>>             { "lengthNorm", "(Lorg/apache/lucene/index/FieldInvertState;)F", (void
*) t_PythonClassicSimilarity_lengthNorm3 },
>>             { "pythonDecRef", "()V", (void *) t_PythonClassicSimilarity_pythonDecRef4
},
>>             { "queryNorm", "(F)F", (void *) t_PythonClassicSimilarity_queryNorm5
},
>>             { "sloppyFreq", "(I)F", (void *) t_PythonClassicSimilarity_sloppyFreq6
},
>>             { "tf", "(F)F", (void *) t_PythonClassicSimilarity_tf7 },
>>           };
>> 
>> 
>> enum {
>>             mid_init$_7353aa4eb69a6f70,
>>             mid_coord_78a3b70e2a43b9c7,
>>             mid_finalize_7353aa4eb69a6f70,
>>             mid_idf_0af4b58f982c0dd1,
>>             mid_idfExplain_08e4e4733921c6f1,
>>             mid_lengthNorm_a7d67c4cd2e3b6a2,
>>             mid_pythonDecRef_7353aa4eb69a6f70,
>>             mid_pythonExtension_aae05472994e7446,
>>             mid_pythonExtension_f41cd3dc7c7d5d78,
>>             mid_queryNorm_eecff9f314592ce9,
>>             mid_sloppyFreq_6b1faadec3ed3278,
>>             mid_tf_eecff9f314592ce9,
>>             max_mid
>>           };
>> 
>> and
>> 
>> jfloat coord(jint, jint) const;
>>           void finalize() const;
>>           jfloat idf(jlong, jlong) const;
>>           ::org::apache::lucene::search::Explanation idfExplain(const ::org::apache::lucene::search::CollectionStatistics
&, const JArray< ::org::apache::lucene::search::TermStatistics > &) const;
>>           jfloat lengthNorm(const ::org::apache::lucene::index::FieldInvertState
&) const;
>>           void pythonDecRef() const;
>>           jlong pythonExtension() const;
>>           void pythonExtension(jlong) const;
>>           jfloat queryNorm(jfloat) const;
>>           jfloat sloppyFreq(jint) const;
>>           jfloat tf(jfloat) const;
>> 
>> Here is the code for PythonSimilarityBase.java (stored in the same directory as PythonClassicSimilarity)
which compiles without issues:
>> package org.apache.pylucene.search.similarities;
>> 
>> import org.apache.lucene.search.similarities.BasicStats;
>> import org.apache.lucene.search.similarities.SimilarityBase;
>> 
>> 
>> 
>> public class PythonSimilarityBase extends SimilarityBase {
>> 
>>   private long pythonObject;
>> 
>>   public PythonSimilarityBase()
>>   {
>>   }
>> 
>>   public void pythonExtension(long pythonObject)
>>   {
>>       this.pythonObject = pythonObject;
>>   }
>>   public long pythonExtension()
>>   {
>>       return this.pythonObject;
>>   }
>> 
>>   public void finalize()
>>       throws Throwable
>>   {
>>       pythonDecRef();
>>   }
>> 
>>   public native void pythonDecRef();
>> 
>> 
>>   @Override
>>   protected native float score(BasicStats stats, float freq, float docLen);
>> 
>>   @Override
>>   public native String toString();
>> }
>> 
>> 
>> I?ve also tried variations of adding the BasicStats class, the Similarity$SimWeight
class and the whole similarities package into the JCC command, for example
>> JCC=$(PYTHON) -m jcc.__main__ --shared --arch x86_64 --package org.apache.lucene.search.similarities
>> (Trying to follow advice in https://issues.apache.org/jira/browse/PYLUCENE-21 <https://issues.apache.org/jira/browse/PYLUCENE-21>
<https://issues.apache.org/jira/browse/PYLUCENE-21 <https://issues.apache.org/jira/browse/PYLUCENE-21>>)
>> 
>> 
>> My feeling is that it has something to do with that BasicStats object?
>> 
>> 
>> If anyone is able to assist it would be greatly appreciated!!
> 
> The 'score' method you're trying to get wrapped is marked 'protected' in the source code
you showed above.
> 
> By default, JCC only wraps public methods and fields.
> 
> You can either make it public or force a wrapper for it by explicitely listing it on
the JCC command line.
> 
> At line 326 in the pylucene Makefile you can see two such non public things being force-wrapped
already via listing:
>           org.apache.lucene.index.IndexWriter:getReader \
>           org.apache.lucene.analysis.Tokenizer:input \
> Just add your score method to the list there...
> 
> Andi..
> 
>> 
>> Thanks,
>> Peter Brown
>> School of Information and Communication Technology,
>> Griffith University - Gold Coast Campus


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message