Mailing-List: contact pylucene-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: pylucene-dev@lucene.apache.org
From: Peter Brown <peter.brown2@griffithuni.edu.au>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_F8556FB6-9F21-47C3-A302-D39AB8E0F402"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: JCC SimilarityBase
Date: Fri, 5 May 2017 12:27:44 +1000
References: <F3A7BF1C-4918-4323-8B30-9E2C5C909B59@griffithuni.edu.au>
 <alpine.OSX.2.01.1705041855040.27684@yuzu.local>
To: pylucene-dev@lucene.apache.org,
 Andi Vajda <vajda@apache.org>
In-Reply-To: <alpine.OSX.2.01.1705041855040.27684@yuzu.local>
Message-Id: <CE32E0FF-2C57-4D65-B259-29B512FF45C1@griffithuni.edu.au>
archived-at: Fri, 05 May 2017 02:34:38 -0000

--Apple-Mail=_F8556FB6-9F21-47C3-A302-D39AB8E0F402
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Andi, you are a champion! :)
Thanks so much for your timely and helpful response.

Unfortunately, I had no luck adding the score function to the makefile.=20=

However, I was able to resolve the issue by defining the method as =
public.


Peter Brown - s2623892
School of Information and Communication Technology,
Griffith University - Gold Coast Campus

> On 5 May 2017, at 11:58 am, Andi Vajda <vajda@apache.org> wrote:
>=20
>=20
> On Fri, 5 May 2017, Peter Brown wrote:
>=20
>> Hi all,
>>=20
>> I was just wondering if anybody else has run into a similar issue =
before.
>> I?m trying to create a wrapper class for the =
org.apache.lucene.search.similarities.SimilarityBase class within =
PyLucene (similar to the existing wrapper for PythonClassicSimilarity).
>>=20
>> However, upon re-building and re-installing PyLucene, when trying to =
use the wrapper class it fails with the following exception:
>>=20
>> java.lang.UnsatisfiedLinkError: =
org.apache.pylucene.search.similarities.PythonSimilarityBase.score(Lorg/ap=
ache/lucene/search/similarities/BasicStats;FF)F
>>=20
>> Upon examination of the wrap0*.cpp files, and comparing with =
PythonClassicSimilarity, I found that the score method is not being =
generated. E.g.
>>=20
>> static PyMethodDef t_PythonSimilarityBase__methods_[] =3D {
>>           DECLARE_METHOD(t_PythonSimilarityBase, cast_, METH_O | =
METH_CLASS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, instance_, METH_O | =
METH_CLASS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, finalize, =
METH_NOARGS),
>>           DECLARE_METHOD(t_PythonSimilarityBase, pythonExtension, =
METH_VARARGS),
>>           { NULL, NULL, 0, NULL }
>>         };
>>=20
>> and
>>=20
>> JNINativeMethod methods[] =3D {
>>             { "pythonDecRef", "()V", (void *) =
t_PythonSimilarityBase_pythonDecRef0 },
>>             { "toString", "()Ljava/lang/String;", (void *) =
t_PythonSimilarityBase_toString1 },
>>           };
>>=20
>> But it is present in the header file enum?
>> class PythonSimilarityBase : public =
::org::apache::lucene::search::similarities::SimilarityBase {
>>         public:
>>           enum {
>>             mid_init$_7353aa4eb69a6f70,
>>             mid_finalize_7353aa4eb69a6f70,
>>             mid_pythonDecRef_7353aa4eb69a6f70,
>>             mid_pythonExtension_aae05472994e7446,
>>             mid_pythonExtension_f41cd3dc7c7d5d78,
>>             mid_toString_5df7257cf25bf439,
>>             mid_score_a6c8466c4349b376,
>>             max_mid
>>           };
>>=20
>> But not in the header file methods.
>> PythonSimilarityBase();
>>=20
>>           void finalize() const;
>>           void pythonDecRef() const;
>>           jlong pythonExtension() const;
>>           void pythonExtension(jlong) const;
>>           ::java::lang::String toString() const;
>>=20
>>=20
>> Compared to
>> static PyMethodDef t_PythonClassicSimilarity__methods_[] =3D {
>>           DECLARE_METHOD(t_PythonClassicSimilarity, cast_, METH_O | =
METH_CLASS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, instance_, METH_O =
| METH_CLASS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, coord, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, finalize, =
METH_NOARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, idf, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, idfExplain, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, lengthNorm, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, pythonExtension, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, queryNorm, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, sloppyFreq, =
METH_VARARGS),
>>           DECLARE_METHOD(t_PythonClassicSimilarity, tf, =
METH_VARARGS),
>>           { NULL, NULL, 0, NULL }
>>         };
>>=20
>> and
>>=20
>> JNINativeMethod methods[] =3D {
>>             { "coord", "(II)F", (void *) =
t_PythonClassicSimilarity_coord0 },
>>             { "idf", "(JJ)F", (void *) t_PythonClassicSimilarity_idf1 =
},
>>             { "idfExplain", =
"(Lorg/apache/lucene/search/CollectionStatistics;[Lorg/apache/lucene/searc=
h/TermStatistics;)Lorg/apache/lucene/search/Explanation;", (void *) =
t_PythonClassicSimilarity_idfExplain2 },
>>             { "lengthNorm", =
"(Lorg/apache/lucene/index/FieldInvertState;)F", (void *) =
t_PythonClassicSimilarity_lengthNorm3 },
>>             { "pythonDecRef", "()V", (void *) =
t_PythonClassicSimilarity_pythonDecRef4 },
>>             { "queryNorm", "(F)F", (void *) =
t_PythonClassicSimilarity_queryNorm5 },
>>             { "sloppyFreq", "(I)F", (void *) =
t_PythonClassicSimilarity_sloppyFreq6 },
>>             { "tf", "(F)F", (void *) t_PythonClassicSimilarity_tf7 },
>>           };
>>=20
>>=20
>> enum {
>>             mid_init$_7353aa4eb69a6f70,
>>             mid_coord_78a3b70e2a43b9c7,
>>             mid_finalize_7353aa4eb69a6f70,
>>             mid_idf_0af4b58f982c0dd1,
>>             mid_idfExplain_08e4e4733921c6f1,
>>             mid_lengthNorm_a7d67c4cd2e3b6a2,
>>             mid_pythonDecRef_7353aa4eb69a6f70,
>>             mid_pythonExtension_aae05472994e7446,
>>             mid_pythonExtension_f41cd3dc7c7d5d78,
>>             mid_queryNorm_eecff9f314592ce9,
>>             mid_sloppyFreq_6b1faadec3ed3278,
>>             mid_tf_eecff9f314592ce9,
>>             max_mid
>>           };
>>=20
>> and
>>=20
>> jfloat coord(jint, jint) const;
>>           void finalize() const;
>>           jfloat idf(jlong, jlong) const;
>>           ::org::apache::lucene::search::Explanation idfExplain(const =
::org::apache::lucene::search::CollectionStatistics &, const JArray< =
::org::apache::lucene::search::TermStatistics > &) const;
>>           jfloat lengthNorm(const =
::org::apache::lucene::index::FieldInvertState &) const;
>>           void pythonDecRef() const;
>>           jlong pythonExtension() const;
>>           void pythonExtension(jlong) const;
>>           jfloat queryNorm(jfloat) const;
>>           jfloat sloppyFreq(jint) const;
>>           jfloat tf(jfloat) const;
>>=20
>> Here is the code for PythonSimilarityBase.java (stored in the same =
directory as PythonClassicSimilarity) which compiles without issues:
>> package org.apache.pylucene.search.similarities;
>>=20
>> import org.apache.lucene.search.similarities.BasicStats;
>> import org.apache.lucene.search.similarities.SimilarityBase;
>>=20
>>=20
>>=20
>> public class PythonSimilarityBase extends SimilarityBase {
>>=20
>>   private long pythonObject;
>>=20
>>   public PythonSimilarityBase()
>>   {
>>   }
>>=20
>>   public void pythonExtension(long pythonObject)
>>   {
>>       this.pythonObject =3D pythonObject;
>>   }
>>   public long pythonExtension()
>>   {
>>       return this.pythonObject;
>>   }
>>=20
>>   public void finalize()
>>       throws Throwable
>>   {
>>       pythonDecRef();
>>   }
>>=20
>>   public native void pythonDecRef();
>>=20
>>=20
>>   @Override
>>   protected native float score(BasicStats stats, float freq, float =
docLen);
>>=20
>>   @Override
>>   public native String toString();
>> }
>>=20
>>=20
>> I?ve also tried variations of adding the BasicStats class, the =
Similarity$SimWeight class and the whole similarities package into the =
JCC command, for example
>> JCC=3D$(PYTHON) -m jcc.__main__ --shared --arch x86_64 --package =
org.apache.lucene.search.similarities
>> (Trying to follow advice in =
https://issues.apache.org/jira/browse/PYLUCENE-21 =
<https://issues.apache.org/jira/browse/PYLUCENE-21> =
<https://issues.apache.org/jira/browse/PYLUCENE-21 =
<https://issues.apache.org/jira/browse/PYLUCENE-21>>)
>>=20
>>=20
>> My feeling is that it has something to do with that BasicStats =
object?
>>=20
>>=20
>> If anyone is able to assist it would be greatly appreciated!!
>=20
> The 'score' method you're trying to get wrapped is marked 'protected' =
in the source code you showed above.
>=20
> By default, JCC only wraps public methods and fields.
>=20
> You can either make it public or force a wrapper for it by explicitely =
listing it on the JCC command line.
>=20
> At line 326 in the pylucene Makefile you can see two such non public =
things being force-wrapped already via listing:
>           org.apache.lucene.index.IndexWriter:getReader \
>           org.apache.lucene.analysis.Tokenizer:input \
> Just add your score method to the list there...
>=20
> Andi..
>=20
>>=20
>> Thanks,
>> Peter Brown
>> School of Information and Communication Technology,
>> Griffith University - Gold Coast Campus


--Apple-Mail=_F8556FB6-9F21-47C3-A302-D39AB8E0F402--