lucene-pylucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andi Vajda <va...@apache.org>
Subject Re: Problem using generic types?
Date Mon, 03 Feb 2014 00:22:19 GMT

On Sun, 2 Feb 2014, Andi Vajda wrote:

>
> On Sun, 2 Feb 2014, Petrus Hyvönen wrote:
>
>> Yes, the confusing thing is that it works well under windows. I compared
>> the code in __wrap__ and it looks different in the windows version:
>> 
>> static int t_PointVectorValuePair_init_(t_PointVectorValuePair *self,
>> PyObject *args, PyObject *kwds)
>>          {
>>            switch (PyTuple_GET_SIZE(args)) {
>>             case 2:
>>              {
>>                JArray< jdouble > a0((jobject) NULL);
>>                JArray< jdouble > a1((jobject) NULL);
>>                PointVectorValuePair object((jobject) NULL);
>>
>>                if (!parseArgs(args, "[D[D", &a0, &a1))
>>                {
>>                  INT_CALL(object = PointVectorValuePair(a0, a1));
>>                  self->object = object;
>>                  break;
>>                }
>>              }
>> 
>> 
>> So no Parameters[] stuff in windows version.. On mac I'm using java v
>> 1.7.0_45 and 1.6.0_35 under windows. could that make a difference?
>
> That could also make a difference but there is a bug nonetheless.

And I've now reproduced it by adding s SimpleClass3 class to the example you 
had sent in last time that looks like this:

public class SimpleClass3 extends SimpleClass<double[]>{

public SimpleClass3(){}
     public void testInJava(){
     System.out.println(this.return_null());
     } 
}

Andi..

>
> Andi..
>
>> 
>> I'm using a mvn downloaded version of the apache.math jar, but source is:
>> 
>> public class PointVectorValuePair extends Pair<double[], double[]>
>> implements Serializable {
>>    /** Serializable UID. */
>>    private static final long serialVersionUID = 20120513L;
>>
>>    /**
>>     ...
>>     */
>>    public PointVectorValuePair(final double[] point,
>>                                final double[] value) {
>>        this(point, value, true);
>>    }
>>
>>    /**
>>    ...
>>     */
>>    public PointVectorValuePair(final double[] point,
>>                                final double[] value,
>>                                final boolean copyArray) {
>>        super(copyArray ?
>>              ((point == null) ? null :
>>               point.clone()) :
>>              point,
>>              copyArray ?
>>              ((value == null) ? null :
>>               value.clone()) :
>>              value);
>>    }
>> 
>> ...
>> 
>> full code at
>> http://commons-math/jacoco/org.apache.commons.math3.optimization/PointVectorValuePair.java.html
>> 
>> 
>> 
>> Regards
>> /petrus
>> 
>> 
>> On 02 Feb 2014, at 22:54 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Feb 2, 2014, at 13:41, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>> 
>> Hi,
>> 
>> It's part of a class "PointVectorValuePair" and "PointValuePair" in the
>> apache math commons (
>> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/org/apache/commons/math3/optimization/PointVectorValuePair.html
>> )
>> 
>> I have tried to reserve a bunch of things like 'Point', 'Value', 'Pair'
>> without any difference. If I go through the __wrap__ file, it looks like
>> this [D object to wrap is a very odd one, the other PY_TYPE wrapping seems
>> to have 'proper' names without special characters. It's the functions
>> PointValuePair and PointVectorValuePair that seems to have this strage
>> class wrapping.  I cannot find a class named D. From the API doc's I would
>> guess it's trying to set a double [] type.
>> 
>> extract from __wrap__:
>> 
>> under namespace org {
>> namespace apache {
>>  namespace commons {
>>    namespace math3 {
>>      namespace optimization {
>> 
>> ...
>>
>>        static int t_PointVectorValuePair_init_(t_PointVectorValuePair
>> *self, PyObject *args, PyObject *kwds)
>>        {
>>          switch (PyTuple_GET_SIZE(args)) {
>>           case 2:
>>            {
>>              JArray< jdouble > a0((jobject) NULL);
>>              JArray< jdouble > a1((jobject) NULL);
>>              PointVectorValuePair object((jobject) NULL);
>>
>>              if (!parseArgs(args, "[D[D", &a0, &a1))
>> 
>> 
>> It could just be that you found a bug in jcc's handling of generic
>> parameters that are arrays. I suspect you have some double[] (or worse:
>> double[][]) somewhere in the java source.
>> But then this should fail the same way on Windows.
>> Can you please post the original java source for that class here ?
>> 
>> Andi..
>>
>>              {
>>                INT_CALL(object = PointVectorValuePair(a0, a1));
>>                self->object = object;
>>                self->parameters[0] = &::PY_TYPE([D);
>>                self->parameters[1] = &::PY_TYPE([D);
>>                break;
>>              }
>> 
>> Regards
>> /petrus
>> 
>> 
>> 
>> = &::PY_TYPE([D);
>>                                                 ^
>> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
>> note:
>>    expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                    ^
>> build/_orekit/__wrap__.cpp:58329:44: error: use of undeclared identifier
>>    'D$$Type'
>>                self->parameters[0] = &::PY_TYPE([D);
>>                                         ^
>> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
>> note:
>>    expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                    ^
>> <scratch space>:109:1: note: expanded from here
>> D$$Type
>> ^
>> build/_orekit/__wrap__.cpp:58329:55: error: expected ']'
>>                self->parameters[0] = &::PY_TYPE([D);
>>                                                    ^
>> build/_orekit/__wrap__.cpp:58329:52: note: to match this '['
>>                self->parameters[0] = &::PY_TYPE([D);
>> 
>> 
>> On 02 Feb 2014, at 19:48 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Feb 2, 2014, at 10:08, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>> 
>> Hi Andi and Others,
>> 
>> The latest trunk jcc works and builds very fine on my windows machine, and
>> happy about the extendend support of generics. But I am experiencing the
>> problem below when building on my mac, using same wrap parameters as on the
>> windows platform. To me it seems like some compiler problem related to
>> macros, but don't know.
>> 
>> Does anyone have experience of these failures?
>> 
>> Regards
>> /Petrus
>> 
>> 
>> error: expected unqualified-id
>>              self->parameters[0] = &::PY_TYPE([D);
>>                                               ^
>> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
>> note:
>>  expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                  ^
>> build/_orekit/__wrap__.cpp:2344:44: error: use of undeclared identifier
>>  'D$$Type'
>>              self->parameters[0] = &::PY_TYPE([D);
>> 
>> 
>> Take a look at the code around line 2344 in that __wrap__.cpp file and
>> figure out what java code this corresponds to. You might be using a
>> classname that's a C macro on some platforms - do you have a class named D
>> for example ?
>> If you're hitting such a name conflict, add that name to the --reserved
>> list passed to jcc.
>> 
>> Andi..
>>
>>                                       ^
>> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
>> note:
>>  expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                  ^
>> <scratch space>:73:1: note: expanded from here
>> D$$Type
>> ^
>> build/_orekit/__wrap__.cpp:2344:55: error: expected ']'
>>              self->parameters[0] = &::PY_TYPE([D);
>>                                                  ^
>> build/_orekit/__wrap__.cpp:2344:52: note: to match this '['
>>              self->parameters[0] = &::PY_TYPE([D);
>> 
>> 
>> 
>> On 14 Jan 2014, at 19:02 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> Hi Petrus,
>> 
>> On Jan 14, 2014, at 4:24, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>> 
>> Dear Andi,
>> 
>> Many many thanks for looking into this!
>> 
>> I am on travel for a week and do not have the computer with the special
>> case with me to test. I did now however run it on the library that I am
>> wrapping, and there get some errors in the creation of the wrapped module
>> (orekit):
>> 
>> build/_orekit/__wrap__.cpp:86504:89: error: no member named
>>  'HarmonicOscillator$Parametric$$Type' in namespace
>> 
>> 
>> I believe I fixed this bug in rev 1557613, header files for classes for
>> inherited fixed parameters were not included as required.
>> 
>> Andi..
>>
>>  'org::apache::commons::math3::analysis::function'
>> ...=
>> &::org::apache::commons::math3::analysis::function::PY_TYPE(HarmonicOs...
>>    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
>> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
>> note:
>>  expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                  ^
>> <scratch space>:63:1: note: expanded from here
>> HarmonicOscillator$Parametric$$Type
>> ^
>> build/_orekit/__wrap__.cpp:217752:55: error: no member named
>>  'OrekitMessages$$Type' in namespace 'org::orekit::errors'
>>    self->parameters[0] = &::org::orekit::errors::PY_TYPE(OrekitMessages);
>>                           ~~~~~~~~~~~~~~~~~~~~~~~^
>> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
>> note:
>>  expanded from macro 'PY_TYPE'
>> #define PY_TYPE(name) name##$$Type
>>                  ^
>> <scratch space>:9:1: note: expanded from here
>> OrekitMessages$$Type
>> ^
>> 2 errors generated.
>> error: command 'gcc' failed with exit status 1
>> 
>> 
>> 
>> The OrekitMessages is defined as a "public enum OrekitMessages implements
>> Localizable { ..."
>> and the "public class HarmonicOscillator implements
>> UnivariateDifferentiable, DifferentiableUnivariateFunction".
>> 
>> Maybe this says you something, I will try to test more and run my test case
>> in .
>> 
>> Best Regards and again, many thanks again!
>> /petrus
>> 
>> 
>> 
>> 
>> 
>> On 11 Jan 2014, at 14:23 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> Hi Petrus,
>> 
>> On Mon, 30 Dec 2013, Petrus Hyvönen wrote:
>> 
>> Andi, great to hear that you could reproduce it. I'm very thankful if you
>> could have a look at it, I've been struggling to understand how the
>> machinery behind this works, but far from it still..
>> 
>> 
>> I think I fixed the problem and added an improvement to generics support
>> along the way. Please check out rev 1557423 from jcc's trunk, rebuild your
>> module with it and let me know how it's working for you.
>> 
>> The bug had to do with SimpleClass2's wrapper class missing its parameters
>> slot which it should get from its superclass, SimpleClass. When calling a
>> method on SimpleClass from SimpleClass2, the missing parameters in the
>> wrapper's struct would cause memory to get trashed.
>> 
>> In addition to fixing the bug, I also improved support for fixed class
>> parameters. Now, if you change SimpleClass's return_null() method to return
>> new Integer(12) instead, when calling it on SimpleClass2, 12 is returned
>> instead of <Object: 12>.
>> 
>> Andi..
>> 
>> public class SimpleClass<T> { public SimpleClass() {
>> System.out.println("Created SimpleClass"); }
>> public T return_null() {
>>   return (T) new Integer(12);
>> }
>> 
>> }
>> 
>> 
>> 
>> Best Regards
>> /Petrus
>> 
>> 
>> 
>> On Mon, Dec 30, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Sun, 29 Dec 2013, Petrus Hyvönen wrote:
>> 
>> I have distilled the library that I have some trouble with and I think I
>> 
>> have an example that is failing due to same problem I think. I am not good
>> in java, but have tried to follow the logic from the library I'm wrapping.
>> The function of the example does not make sense in itself.
>> 
>> public class SimpleClass<T> {
>> public SimpleClass() {
>> System.out.println("Created SimpleClass");
>> }
>> public T return_null() {
>>  return  null;
>> }
>> 
>> }
>> 
>> 
>> 
>> public class SimpleClass2 extends SimpleClass<Integer>{
>> 
>> public SimpleClass2(){}
>> public void testInJava(){
>> System.out.println(this.return_null());
>> }
>> }
>> 
>> 
>> It seems to me that there is some problem with methods inherited that
>> returns a generic type, failing in wrapType when this is to be wrapped.
>> 
>> The python script that fails:
>> 
>> a= SimpleClass()
>> print a.return_null()
>> 
>> b = SimpleClass2()
>> b.testInJava()
>> 
>> print b.return_null()  #Fails in wrapType
>> 
>> I don't know if the return null is a bad thing to do in java, but the
>> error
>> seems very similar to what I experience in the larger library. I have a
>> skeleton of that this is slightly larger, but not returning null, but
>> trying to keep the lenght of example low :)
>> 
>> Any comments highly appriciated :)
>> 
>> 
>> I've been able to reproduce the problem.
>> Thank you for providing an isolated test case !
>> 
>> Andi..
>> 
>> 
>> 
>> best Regards
>> /Petrus
>> 
>> 
>> 
>> 
>> 
>> 
>> On Fri, Dec 27, 2013 at 6:25 PM, Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Dec 27, 2013, at 17:36, Petrus Hyvönen <petrus.hyvonen@gmail.com>
>> wrote:
>> 
>> 
>> Dear Andi,
>> 
>> I am working on debugging the failure and try to understand a bit how
>> JCC
>> works internally. I haven't gone very far but in case you have some
>> pointers from these early debugging sessions I would be very thankful. I
>> know it's complex, and I should try to make some smaller test cases, but
>> 
>> I
>> 
>> don't really have a grasp yet where the problem might be.
>> 
>> Writing this might help me also to get some structure in my thinking :)
>> 
>> 
>> 
>> The main crash seems to be in the last line, wrapType(), of
>> __wrap__.cpp:
>> 
>> static PyObject
>> 
>> *t_AbstractReconfigurableDetector_withHandler(t_
>> 
>> AbstractReconfigurableDetector
>> 
>> *self, PyObject *arg)
>>  {
>>    ::org::orekit::propagation::events::handlers::EventHandler
>> a0((jobject) NULL);
>>    PyTypeObject **p0;
>>    ::org::orekit::propagation::events::EventDetector
>> result((jobject) NULL);
>>
>>    if (!parseArg(arg, "K",
>> 
>> ::org::orekit::propagation::events::handlers::
>> 
>> EventHandler::initializeClass,
>> 
>> &a0, &p0,
>> 
>> ::org::orekit::propagation::events::handlers::t_
>> 
>> EventHandler::parameters_))
>>
>>    {
>>      OBJ_CALL(result = self->object.withHandler(a0));
>>      return self->parameters[0] != NULL ?
>> wrapType(self->parameters[0], result.this$) :
>> ::org::orekit::propagation::events::t_EventDetector::wrap_
>> Object(result);
>>    }
>> 
>> The parameters[0] does not seem to be null, but neither is it a valid
>> object, in my debugger it says 0xbaadf00d {ob_refcnt=??? ob_type=???
>> ob_size=??? ...}     _typeobject *, wrapType is called and when trying
>> to
>> access the wrapfn it crashes.
>> 
>> The main python lines are:
>> tmp1 = ElevationDetector(sta1Frame)                    #
>> 
>> ElevationDetector
>> 
>> is a java public class ElevationDetector extends
>> AbstractReconfigurableDetector<ElevationDetector>
>> hand = ContinueOnEvent().of_(ElevationDetector)   # a
>> java ContinueOnEvent<ElevationDetector> object
>> elDetector = tmp1.withHandler(hand)     #Crash. withHandler is a method
>> that is inherited from AbstractReconfigurableDetector to
>> 
>> ElevationDetector
>> 
>> 
>> This crashes when interactively entered on the python prompt (or in
>> other
>> interactive consoles), but seems to work if executed directly without
>> interactivity. This difference makes me think that it might be something
>> with garbage collection, but don't know.
>> 
>> Any comments appriciated, I know this is likely very difficult to
>> comment
>> on as it's not very encapsulated.
>> 
>> 
>> Right, so unless you can isolate this into something I can reproduce, I'm
>> afraid there isn't much I can comment.
>> It is quite likely that by the time you have that reproducible test case
>> ready, you also have the solution to the problem. Or I might be able to
>> help then...
>> 
>> Andi..
>> 
>> 
>> Regards
>> /Petrus
>> 
>> 
>> 
>> 
>> 
>> 
>> On Sun, Dec 15, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Dec 15, 2013, at 5:43, Petrus Hyvönen <petrus.hyvonen@gmail.com>
>> 
>> wrote:
>> 
>> Hi Andi,
>> 
>> I see your point and have now kept in the pure python domain.
>> 
>> If I run my script from the shell by "python script.py" it does not
>> 
>> crash. However if I execute it line-by-line in python it crashes (or in
>> other tools such as ipython notebook).
>> 
>> All classes used are non-wrapped java classes, but I get the same
>> 
>> effect
>> 
>> 
>> with classes made for python subclassing.
>> 
>> 
>> I am getting this on both MacOSX 64-bit python and Windows 7 32-bit
>> 
>> python.
>> 
>> 
>> 
>> elDetector =
>> 
>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>> 
>> 
>> #
>> # A fatal error has been detected by the Java Runtime Environment:
>> #
>> #  SIGSEGV (0xb) at pc=0x000000010005da1a, pid=3318, tid=1287
>> #
>> # JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build
>> 
>> 1.7.0_45-b18)
>> 
>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode
>> 
>> bsd-amd64 compressed oops)
>> 
>> # Problematic frame:
>> # C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
>> #
>> 
>> 
>> from the stack it seems like there is somthing happening in "wrapType"
>> Stack: [0x00007fff5fb80000,0x00007fff5fc00000],
>> sp=0x00007fff5fbff470,
>> 
>> free space=509k
>> 
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
>> 
>> C=native code)
>> 
>> C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
>> C  [_orekit.so+0xa80878]  wrapType(_typeobject*, _jobject*
>> const&)+0x58
>> C  [_orekit.so+0x554400]
>> 
>> 
>> org::orekit::propagation::events::t_AbstractReconfigurableDetector
>> 
>> _withHandler(org::orekit::propagation::events::t_
>> AbstractReconfigurableDetector*,
>> 
>> _object*)+0x1c0
>> 
>> 
>> 
>> First, is the generic class assignment correct as if to write "new
>> 
>> ContinueOnEvent<ElevationDetector>()" in java? And is it ok to use
>> 
>> regular
>> 
>> 
>> java objects/types?
>> 
>> 
>> 
>> Any other comments to move forward highly appriciated.. Is it somehow
>> 
>> possible to get more log what is going wrong?
>> 
>> You could compile the whole thing for debugging, by adding --debug
>> after
>> 'build' in the jcc invocation and run it with gdb.
>> 
>> If you can isolate a reproducible crash into a small test case, I can
>> 
>> also
>> 
>> 
>> take a look at it.
>> 
>> 
>> Andi..
>> 
>> 
>> WIth best regards
>> /Petrus
>> 
>> 
>> 
>> On 15 Dec 2013, at 2:40 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> On Dec 14, 2013, at 19:14, Petrus Hyvönen <petrus.hyvonen@gmail.com
>> wrote:
>> 
>> 
>> 
>> Hi,
>> 
>> I'm having a problem with I think might be related to generic types,
>> 
>> but not sure at all.
>> 
>> 
>> 
>> I'm wrapping a orbit calculation library, which has been working
>> well
>> 
>> but in latest version is using generic types and I'm getting some
>> 
>> problems.
>> 
>> 
>> The script works when executed in plain python, but fails in ipython
>> 
>> notebook on this last line when executed as a couple of cells.
>> 
>> 
>> What is an 'ipython notebook' ?
>> 
>> Andi.,
>> 
>> 
>> The section with problem in my script is:
>> elDetector =
>> 
>> ElevationDetector(sta1Frame).withConstantElevation(math.
>> 
>> radians(5.0))
>> 
>> elDetector =
>> 
>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>> 
>> 
>> 
>> In Java it would typically look something like:
>> 
>> ElevationDetector detector = new ElevationDetector(topo)
>>                                   .withConstantElevation(x)
>>                                   .withHandler(new
>> 
>> ContinueOnEvent<ElevationDetector>());
>> 
>> 
>> 
>> It produces correct results in plain python, but crashes the kernel
>> 
>> in
>> 
>> 
>> ipython if executed as cells, and in exection from spyder I get an error
>> 
>> message:
>> 
>> 
>> " elDetector =
>> 
>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>> 
>> 
>> AttributeError: 'str' object has no attribute 'wrapfn_' "
>> 
>> 
>> As I have been using this setup stabely with lots of other functions
>> 
>> it feels like there is something with the generic type line, but I
>> 
>> don't
>> really know how to get any further? I'm confused by that the pauses in
>> 
>> the
>> 
>> 
>> execution could seem to affect the result.
>> 
>> 
>> 
>> Any comments highly appriciated...
>> 
>> Best Regards
>> /Petrus
>> 
>> 
>> 
>> --
>> _____________________________________________
>> Petrus Hyvönen, Uppsala, Sweden
>> Mobile Phone/SMS:+46 73 803 19 00
>> 
>> 
>> 
>> --
>> _____________________________________________
>> Petrus Hyvönen, Uppsala, Sweden
>> Mobile Phone/SMS:+46 73 803 19 00
>> 
>> 
>> 
>> -- 
>> _____________________________________________
>> Petrus Hyvönen, Uppsala, Sweden
>> Mobile Phone/SMS:+46 73 803 19 00
>
Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message