lucene-pylucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andi Vajda <va...@apache.org>
Subject Re: Problem using generic types?
Date Sun, 02 Feb 2014 18:48:18 GMT

> On Feb 2, 2014, at 10:08, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
> 
> Hi Andi and Others,
> 
> The latest trunk jcc works and builds very fine on my windows machine, and happy about
the extendend support of generics. But I am experiencing the problem below when building on
my mac, using same wrap parameters as on the windows platform. To me it seems like some compiler
problem related to macros, but don't know.
> 
> Does anyone have experience of these failures?
> 
> Regards
> /Petrus
> 
> 
> error: expected unqualified-id
>                  self->parameters[0] = &::PY_TYPE([D);
>                                                   ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
note: 
>      expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                      ^
> build/_orekit/__wrap__.cpp:2344:44: error: use of undeclared identifier
>      'D$$Type'
>                  self->parameters[0] = &::PY_TYPE([D);

Take a look at the code around line 2344 in that __wrap__.cpp file and figure out what java
code this corresponds to. You might be using a classname that's a C macro on some platforms
- do you have a class named D for example ?
If you're hitting such a name conflict, add that name to the --reserved list passed to jcc.

Andi..

>                                           ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
note: 
>      expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                      ^
> <scratch space>:73:1: note: expanded from here
> D$$Type
> ^
> build/_orekit/__wrap__.cpp:2344:55: error: expected ']'
>                  self->parameters[0] = &::PY_TYPE([D);
>                                                      ^
> build/_orekit/__wrap__.cpp:2344:52: note: to match this '['
>                  self->parameters[0] = &::PY_TYPE([D);
> 
> 
> 
>> On 14 Jan 2014, at 19:02 , Andi Vajda <vajda@apache.org> wrote:
>> 
>> 
>> Hi Petrus,
>> 
>>> On Jan 14, 2014, at 4:24, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>>> 
>>> Dear Andi,
>>> 
>>> Many many thanks for looking into this! 
>>> 
>>> I am on travel for a week and do not have the computer with the special case
with me to test. I did now however run it on the library that I am wrapping, and there get
some errors in the creation of the wrapped module (orekit):
>>> 
>>> build/_orekit/__wrap__.cpp:86504:89: error: no member named
>>>      'HarmonicOscillator$Parametric$$Type' in namespace
>> 
>> I believe I fixed this bug in rev 1557613, header files for classes for inherited
fixed parameters were not included as required.
>> 
>> Andi..
>> 
>>>      'org::apache::commons::math3::analysis::function'
>>>  ...= &::org::apache::commons::math3::analysis::function::PY_TYPE(HarmonicOs...
>>>        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
>>> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
note: 
>>>      expanded from macro 'PY_TYPE'
>>> #define PY_TYPE(name) name##$$Type
>>>                      ^
>>> <scratch space>:63:1: note: expanded from here
>>> HarmonicOscillator$Parametric$$Type
>>> ^
>>> build/_orekit/__wrap__.cpp:217752:55: error: no member named
>>>      'OrekitMessages$$Type' in namespace 'org::orekit::errors'
>>>        self->parameters[0] = &::org::orekit::errors::PY_TYPE(OrekitMessages);
>>>                               ~~~~~~~~~~~~~~~~~~~~~~~^
>>> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
note: 
>>>      expanded from macro 'PY_TYPE'
>>> #define PY_TYPE(name) name##$$Type
>>>                      ^
>>> <scratch space>:9:1: note: expanded from here
>>> OrekitMessages$$Type
>>> ^
>>> 2 errors generated.
>>> error: command 'gcc' failed with exit status 1
>>> 
>>> 
>>> 
>>> The OrekitMessages is defined as a "public enum OrekitMessages implements Localizable
{ ..."
>>> and the "public class HarmonicOscillator implements UnivariateDifferentiable,
DifferentiableUnivariateFunction". 
>>> 
>>> Maybe this says you something, I will try to test more and run my test case in
.
>>> 
>>> Best Regards and again, many thanks again!
>>> /petrus
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 11 Jan 2014, at 14:23 , Andi Vajda <vajda@apache.org> wrote:
>>>> 
>>>> 
>>>> Hi Petrus,
>>>> 
>>>>> On Mon, 30 Dec 2013, Petrus Hyvönen wrote:
>>>>> 
>>>>> Andi, great to hear that you could reproduce it. I'm very thankful if
you
>>>>> could have a look at it, I've been struggling to understand how the
>>>>> machinery behind this works, but far from it still..
>>>> 
>>>> I think I fixed the problem and added an improvement to generics support
along the way. Please check out rev 1557423 from jcc's trunk, rebuild your module with it
and let me know how it's working for you.
>>>> 
>>>> The bug had to do with SimpleClass2's wrapper class missing its parameters
slot which it should get from its superclass, SimpleClass. When calling a
>>>> method on SimpleClass from SimpleClass2, the missing parameters in the
>>>> wrapper's struct would cause memory to get trashed.
>>>> 
>>>> In addition to fixing the bug, I also improved support for fixed class parameters.
Now, if you change SimpleClass's return_null() method to return new Integer(12) instead, when
calling it on SimpleClass2, 12 is returned instead of <Object: 12>.
>>>> 
>>>> Andi..
>>>> 
>>>> public class SimpleClass<T> { public SimpleClass() { System.out.println("Created
SimpleClass"); }
>>>>   public T return_null() {
>>>>       return (T) new Integer(12);
>>>>   }
>>>> 
>>>> }
>>>> 
>>>> 
>>>>> 
>>>>> Best Regards
>>>>> /Petrus
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Mon, Dec 30, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org>
wrote:
>>>>>> 
>>>>>> 
>>>>>> On Sun, 29 Dec 2013, Petrus Hyvönen wrote:
>>>>>> 
>>>>>> I have distilled the library that I have some trouble with and I
think I
>>>>>>> have an example that is failing due to same problem I think.
I am not good
>>>>>>> in java, but have tried to follow the logic from the library
I'm wrapping.
>>>>>>> The function of the example does not make sense in itself.
>>>>>>> 
>>>>>>> public class SimpleClass<T> {
>>>>>>> public SimpleClass() {
>>>>>>> System.out.println("Created SimpleClass");
>>>>>>> }
>>>>>>>  public T return_null() {
>>>>>>>      return  null;
>>>>>>>  }
>>>>>>> 
>>>>>>> }
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> public class SimpleClass2 extends SimpleClass<Integer>{
>>>>>>> 
>>>>>>> public SimpleClass2(){}
>>>>>>>  public void testInJava(){
>>>>>>>  System.out.println(this.return_null());
>>>>>>>  }
>>>>>>> }
>>>>>>> 
>>>>>>> 
>>>>>>> It seems to me that there is some problem with methods inherited
that
>>>>>>> returns a generic type, failing in wrapType when this is to be
wrapped.
>>>>>>> 
>>>>>>> The python script that fails:
>>>>>>> 
>>>>>>> a= SimpleClass()
>>>>>>> print a.return_null()
>>>>>>> 
>>>>>>> b = SimpleClass2()
>>>>>>> b.testInJava()
>>>>>>> 
>>>>>>> print b.return_null()  #Fails in wrapType
>>>>>>> 
>>>>>>> I don't know if the return null is a bad thing to do in java,
but the
>>>>>>> error
>>>>>>> seems very similar to what I experience in the larger library.
I have a
>>>>>>> skeleton of that this is slightly larger, but not returning null,
but
>>>>>>> trying to keep the lenght of example low :)
>>>>>>> 
>>>>>>> Any comments highly appriciated :)
>>>>>> 
>>>>>> I've been able to reproduce the problem.
>>>>>> Thank you for providing an isolated test case !
>>>>>> 
>>>>>> Andi..
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> best Regards
>>>>>>> /Petrus
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Fri, Dec 27, 2013 at 6:25 PM, Andi Vajda <vajda@apache.org>
wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> On Dec 27, 2013, at 17:36, Petrus Hyvönen <petrus.hyvonen@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Dear Andi,
>>>>>>>>> 
>>>>>>>>> I am working on debugging the failure and try to understand
a bit how
>>>>>>>>> JCC
>>>>>>>>> works internally. I haven't gone very far but in case
you have some
>>>>>>>>> pointers from these early debugging sessions I would
be very thankful. I
>>>>>>>>> know it's complex, and I should try to make some smaller
test cases, but
>>>>>>>> I
>>>>>>>> 
>>>>>>>>> don't really have a grasp yet where the problem might
be.
>>>>>>>>> 
>>>>>>>>> Writing this might help me also to get some structure
in my thinking :)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> The main crash seems to be in the last line, wrapType(),
of
>>>>>>>>> __wrap__.cpp:
>>>>>>>>> 
>>>>>>>>>    static PyObject
>>>>>>>>> 
>>>>>>>>> *t_AbstractReconfigurableDetector_withHandler(t_
>>>>>>>> AbstractReconfigurableDetector
>>>>>>>> 
>>>>>>>>> *self, PyObject *arg)
>>>>>>>>>      {
>>>>>>>>>        ::org::orekit::propagation::events::handlers::EventHandler
>>>>>>>>> a0((jobject) NULL);
>>>>>>>>>        PyTypeObject **p0;
>>>>>>>>>        ::org::orekit::propagation::events::EventDetector
>>>>>>>>> result((jobject) NULL);
>>>>>>>>> 
>>>>>>>>>        if (!parseArg(arg, "K",
>>>>>>>>> 
>>>>>>>>> ::org::orekit::propagation::events::handlers::
>>>>>>>> EventHandler::initializeClass,
>>>>>>>> 
>>>>>>>>> &a0, &p0,
>>>>>>>>> 
>>>>>>>>> ::org::orekit::propagation::events::handlers::t_
>>>>>>>> EventHandler::parameters_))
>>>>>>>> 
>>>>>>>>>        {
>>>>>>>>>          OBJ_CALL(result = self->object.withHandler(a0));
>>>>>>>>>          return self->parameters[0] != NULL ?
>>>>>>>>> wrapType(self->parameters[0], result.this$) :
>>>>>>>>> ::org::orekit::propagation::events::t_EventDetector::wrap_
>>>>>>>>> Object(result);
>>>>>>>>>        }
>>>>>>>>> 
>>>>>>>>> The parameters[0] does not seem to be null, but neither
is it a valid
>>>>>>>>> object, in my debugger it says 0xbaadf00d {ob_refcnt=???
ob_type=???
>>>>>>>>> ob_size=??? ...}     _typeobject *, wrapType is called
and when trying
>>>>>>>>> to
>>>>>>>>> access the wrapfn it crashes.
>>>>>>>>> 
>>>>>>>>> The main python lines are:
>>>>>>>>> tmp1 = ElevationDetector(sta1Frame)                 
  #
>>>>>>>> ElevationDetector
>>>>>>>> 
>>>>>>>>> is a java public class ElevationDetector extends
>>>>>>>>> AbstractReconfigurableDetector<ElevationDetector>
>>>>>>>>> hand = ContinueOnEvent().of_(ElevationDetector)   # a
>>>>>>>>> java ContinueOnEvent<ElevationDetector> object
>>>>>>>>> elDetector = tmp1.withHandler(hand)     #Crash. withHandler
is a method
>>>>>>>>> that is inherited from AbstractReconfigurableDetector
to
>>>>>>>> ElevationDetector
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> This crashes when interactively entered on the python
prompt (or in
>>>>>>>>> other
>>>>>>>>> interactive consoles), but seems to work if executed
directly without
>>>>>>>>> interactivity. This difference makes me think that it
might be something
>>>>>>>>> with garbage collection, but don't know.
>>>>>>>>> 
>>>>>>>>> Any comments appriciated, I know this is likely very
difficult to
>>>>>>>>> comment
>>>>>>>>> on as it's not very encapsulated.
>>>>>>>> 
>>>>>>>> Right, so unless you can isolate this into something I can
reproduce, I'm
>>>>>>>> afraid there isn't much I can comment.
>>>>>>>> It is quite likely that by the time you have that reproducible
test case
>>>>>>>> ready, you also have the solution to the problem. Or I might
be able to
>>>>>>>> help then...
>>>>>>>> 
>>>>>>>> Andi..
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Regards
>>>>>>>>> /Petrus
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Sun, Dec 15, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org>
wrote:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Dec 15, 2013, at 5:43, Petrus Hyvönen <petrus.hyvonen@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Andi,
>>>>>>>>>>> 
>>>>>>>>>>> I see your point and have now kept in the pure
python domain.
>>>>>>>>>>> 
>>>>>>>>>>> If I run my script from the shell by "python
script.py" it does not
>>>>>>>>>> crash. However if I execute it line-by-line in python
it crashes (or in
>>>>>>>>>> other tools such as ipython notebook).
>>>>>>>>>> 
>>>>>>>>>>> All classes used are non-wrapped java classes,
but I get the same
>>>>>>>>>> effect
>>>>>>>> 
>>>>>>>>> with classes made for python subclassing.
>>>>>>>>>> 
>>>>>>>>>>> I am getting this on both MacOSX 64-bit python
and Windows 7 32-bit
>>>>>>>>>> python.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> elDetector =
>>>>>>>>>>>>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>>>>>>>>>> 
>>>>>>>>>>> #
>>>>>>>>>>> # A fatal error has been detected by the Java
Runtime Environment:
>>>>>>>>>>> #
>>>>>>>>>>> #  SIGSEGV (0xb) at pc=0x000000010005da1a, pid=3318,
tid=1287
>>>>>>>>>>> #
>>>>>>>>>>> # JRE version: Java(TM) SE Runtime Environment
(7.0_45-b18) (build
>>>>>>>>>> 1.7.0_45-b18)
>>>>>>>>>> 
>>>>>>>>>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM
(24.45-b08 mixed mode
>>>>>>>>>> bsd-amd64 compressed oops)
>>>>>>>>>> 
>>>>>>>>>>> # Problematic frame:
>>>>>>>>>>> # C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
>>>>>>>>>>> #
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> from the stack it seems like there is somthing
happening in "wrapType"
>>>>>>>>>>> Stack: [0x00007fff5fb80000,0x00007fff5fc00000],
>>>>>>>>>>> sp=0x00007fff5fbff470,
>>>>>>>>>> free space=509k
>>>>>>>>>> 
>>>>>>>>>>> Native frames: (J=compiled Java code, j=interpreted,
Vv=VM code,
>>>>>>>>>> C=native code)
>>>>>>>>>> 
>>>>>>>>>>> C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
>>>>>>>>>>> C  [_orekit.so+0xa80878]  wrapType(_typeobject*,
_jobject*
>>>>>>>>>>> const&)+0x58
>>>>>>>>>>> C  [_orekit.so+0x554400]
>>>>>>>>>> 
>>>>>>>>>> org::orekit::propagation::events::t_AbstractReconfigurableDetector
>>>>>>>> _withHandler(org::orekit::propagation::events::t_
>>>>>>>> AbstractReconfigurableDetector*,
>>>>>>>> 
>>>>>>>>> _object*)+0x1c0
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> First, is the generic class assignment correct
as if to write "new
>>>>>>>>>> ContinueOnEvent<ElevationDetector>()" in java?
And is it ok to use
>>>>>>>>> regular
>>>>>>>> 
>>>>>>>>> java objects/types?
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Any other comments to move forward highly appriciated..
Is it somehow
>>>>>>>>>> possible to get more log what is going wrong?
>>>>>>>>>> 
>>>>>>>>>> You could compile the whole thing for debugging,
by adding --debug
>>>>>>>>>> after
>>>>>>>>>> 'build' in the jcc invocation and run it with gdb.
>>>>>>>>>> 
>>>>>>>>>> If you can isolate a reproducible crash into a small
test case, I can
>>>>>>>>> also
>>>>>>>> 
>>>>>>>>> take a look at it.
>>>>>>>>>> 
>>>>>>>>>> Andi..
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> WIth best regards
>>>>>>>>>>> /Petrus
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On 15 Dec 2013, at 2:40 , Andi Vajda <vajda@apache.org>
wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Dec 14, 2013, at 19:14, Petrus Hyvönen
<petrus.hyvonen@gmail.com
>>>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm having a problem with I think might
be related to generic types,
>>>>>>>>>>>> but not sure at all.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> I'm wrapping a orbit calculation library,
which has been working
>>>>>>>>>>>>> well
>>>>>>>>>>>> but in latest version is using generic types
and I'm getting some
>>>>>>>>> problems.
>>>>>>>> 
>>>>>>>>> The script works when executed in plain python, but fails
in ipython
>>>>>>>>>> notebook on this last line when executed as a couple
of cells.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> What is an 'ipython notebook' ?
>>>>>>>>>>>> 
>>>>>>>>>>>> Andi.,
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> The section with problem in my script
is:
>>>>>>>>>>>>> elDetector =
>>>>>>>>>>>> ElevationDetector(sta1Frame).withConstantElevation(math.
>>>>>>>>>> radians(5.0))
>>>>>>>>>> 
>>>>>>>>>>> elDetector =
>>>>>>>>>>>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> In Java it would typically look something
like:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ElevationDetector detector = new ElevationDetector(topo)
>>>>>>>>>>>>>                                     
 .withConstantElevation(x)
>>>>>>>>>>>>>                                     
 .withHandler(new
>>>>>>>>>>>> ContinueOnEvent<ElevationDetector>());
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> It produces correct results in plain
python, but crashes the kernel
>>>>>>>>>>>> in
>>>>>>>> 
>>>>>>>>> ipython if executed as cells, and in exection from spyder
I get an error
>>>>>>>>>> message:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> " elDetector =
>>>>>>>>>>>> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>>>>>>>>>> 
>>>>>>>>>>> AttributeError: 'str' object has no attribute
'wrapfn_' "
>>>>>>>>>>>>> 
>>>>>>>>>>>>> As I have been using this setup stabely
with lots of other functions
>>>>>>>>>>>> it feels like there is something with the
generic type line, but I
>>>>>>>>>> don't
>>>>>>>>>> really know how to get any further? I'm confused
by that the pauses in
>>>>>>>>> the
>>>>>>>> 
>>>>>>>>> execution could seem to affect the result.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> Any comments highly appriciated...
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Best Regards
>>>>>>>>>>>>> /Petrus
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> _____________________________________________
>>>>>>>>> Petrus Hyvönen, Uppsala, Sweden
>>>>>>>>> Mobile Phone/SMS:+46 73 803 19 00
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> _____________________________________________
>>>>>>> Petrus Hyvönen, Uppsala, Sweden
>>>>>>> Mobile Phone/SMS:+46 73 803 19 00
>>>>> 
>>>>> 
>>>>> -- 
>>>>> _____________________________________________
>>>>> Petrus Hyvönen, Uppsala, Sweden
>>>>> Mobile Phone/SMS:+46 73 803 19 00
> 

Mime
View raw message