lucene-pylucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andi Vajda <va...@apache.org>
Subject Re: Problem using generic types?
Date Mon, 03 Feb 2014 00:13:01 GMT

On Sun, 2 Feb 2014, Petrus Hyvönen wrote:

> Yes, the confusing thing is that it works well under windows. I compared
> the code in __wrap__ and it looks different in the windows version:
>
> static int t_PointVectorValuePair_init_(t_PointVectorValuePair *self,
> PyObject *args, PyObject *kwds)
>          {
>            switch (PyTuple_GET_SIZE(args)) {
>             case 2:
>              {
>                JArray< jdouble > a0((jobject) NULL);
>                JArray< jdouble > a1((jobject) NULL);
>                PointVectorValuePair object((jobject) NULL);
>
>                if (!parseArgs(args, "[D[D", &a0, &a1))
>                {
>                  INT_CALL(object = PointVectorValuePair(a0, a1));
>                  self->object = object;
>                  break;
>                }
>              }
>
>
> So no Parameters[] stuff in windows version.. On mac I'm using java v
> 1.7.0_45 and 1.6.0_35 under windows. could that make a difference?
>
> I'm using a mvn downloaded version of the apache.math jar, but source is:
>
> public class PointVectorValuePair extends Pair<double[], double[]>

Ah yes, there it is: extends Pair<double[], double[]>
This has got to fail the same way on Windows unless you're using an older 
JCC version. I think the fix I did recently for your use case doesn't handle 
arrays correctly.

Andi..

> implements Serializable {
>    /** Serializable UID. */
>    private static final long serialVersionUID = 20120513L;
>
>    /**
>     ...
>     */
>    public PointVectorValuePair(final double[] point,
>                                final double[] value) {
>        this(point, value, true);
>    }
>
>    /**
>    ...
>     */
>    public PointVectorValuePair(final double[] point,
>                                final double[] value,
>                                final boolean copyArray) {
>        super(copyArray ?
>              ((point == null) ? null :
>               point.clone()) :
>              point,
>              copyArray ?
>              ((value == null) ? null :
>               value.clone()) :
>              value);
>    }
>
> ...
>
> full code at
> http://commons-math/jacoco/org.apache.commons.math3.optimization/PointVectorValuePair.java.html
>
>
>
> Regards
> /petrus
>
>
> On 02 Feb 2014, at 22:54 , Andi Vajda <vajda@apache.org> wrote:
>
>
> On Feb 2, 2014, at 13:41, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>
> Hi,
>
> It's part of a class "PointVectorValuePair" and "PointValuePair" in the
> apache math commons (
> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/org/apache/commons/math3/optimization/PointVectorValuePair.html
> )
>
> I have tried to reserve a bunch of things like 'Point', 'Value', 'Pair'
> without any difference. If I go through the __wrap__ file, it looks like
> this [D object to wrap is a very odd one, the other PY_TYPE wrapping seems
> to have 'proper' names without special characters. It's the functions
> PointValuePair and PointVectorValuePair that seems to have this strage
> class wrapping.  I cannot find a class named D. From the API doc's I would
> guess it's trying to set a double [] type.
>
> extract from __wrap__:
>
> under namespace org {
> namespace apache {
>  namespace commons {
>    namespace math3 {
>      namespace optimization {
>
> ...
>
>        static int t_PointVectorValuePair_init_(t_PointVectorValuePair
> *self, PyObject *args, PyObject *kwds)
>        {
>          switch (PyTuple_GET_SIZE(args)) {
>           case 2:
>            {
>              JArray< jdouble > a0((jobject) NULL);
>              JArray< jdouble > a1((jobject) NULL);
>              PointVectorValuePair object((jobject) NULL);
>
>              if (!parseArgs(args, "[D[D", &a0, &a1))
>
>
> It could just be that you found a bug in jcc's handling of generic
> parameters that are arrays. I suspect you have some double[] (or worse:
> double[][]) somewhere in the java source.
> But then this should fail the same way on Windows.
> Can you please post the original java source for that class here ?
>
> Andi..
>
>              {
>                INT_CALL(object = PointVectorValuePair(a0, a1));
>                self->object = object;
>                self->parameters[0] = &::PY_TYPE([D);
>                self->parameters[1] = &::PY_TYPE([D);
>                break;
>              }
>
> Regards
> /petrus
>
>
>
> = &::PY_TYPE([D);
>                                                 ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
> note:
>    expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                    ^
> build/_orekit/__wrap__.cpp:58329:44: error: use of undeclared identifier
>    'D$$Type'
>                self->parameters[0] = &::PY_TYPE([D);
>                                         ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
> note:
>    expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                    ^
> <scratch space>:109:1: note: expanded from here
> D$$Type
> ^
> build/_orekit/__wrap__.cpp:58329:55: error: expected ']'
>                self->parameters[0] = &::PY_TYPE([D);
>                                                    ^
> build/_orekit/__wrap__.cpp:58329:52: note: to match this '['
>                self->parameters[0] = &::PY_TYPE([D);
>
>
> On 02 Feb 2014, at 19:48 , Andi Vajda <vajda@apache.org> wrote:
>
>
> On Feb 2, 2014, at 10:08, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>
> Hi Andi and Others,
>
> The latest trunk jcc works and builds very fine on my windows machine, and
> happy about the extendend support of generics. But I am experiencing the
> problem below when building on my mac, using same wrap parameters as on the
> windows platform. To me it seems like some compiler problem related to
> macros, but don't know.
>
> Does anyone have experience of these failures?
>
> Regards
> /Petrus
>
>
> error: expected unqualified-id
>              self->parameters[0] = &::PY_TYPE([D);
>                                               ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
> note:
>  expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                  ^
> build/_orekit/__wrap__.cpp:2344:44: error: use of undeclared identifier
>  'D$$Type'
>              self->parameters[0] = &::PY_TYPE([D);
>
>
> Take a look at the code around line 2344 in that __wrap__.cpp file and
> figure out what java code this corresponds to. You might be using a
> classname that's a C macro on some platforms - do you have a class named D
> for example ?
> If you're hitting such a name conflict, add that name to the --reserved
> list passed to jcc.
>
> Andi..
>
>                                       ^
> /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23:
> note:
>  expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                  ^
> <scratch space>:73:1: note: expanded from here
> D$$Type
> ^
> build/_orekit/__wrap__.cpp:2344:55: error: expected ']'
>              self->parameters[0] = &::PY_TYPE([D);
>                                                  ^
> build/_orekit/__wrap__.cpp:2344:52: note: to match this '['
>              self->parameters[0] = &::PY_TYPE([D);
>
>
>
> On 14 Jan 2014, at 19:02 , Andi Vajda <vajda@apache.org> wrote:
>
>
> Hi Petrus,
>
> On Jan 14, 2014, at 4:24, Petrus Hyvönen <petrus.hyvonen@gmail.com> wrote:
>
> Dear Andi,
>
> Many many thanks for looking into this!
>
> I am on travel for a week and do not have the computer with the special
> case with me to test. I did now however run it on the library that I am
> wrapping, and there get some errors in the creation of the wrapped module
> (orekit):
>
> build/_orekit/__wrap__.cpp:86504:89: error: no member named
>  'HarmonicOscillator$Parametric$$Type' in namespace
>
>
> I believe I fixed this bug in rev 1557613, header files for classes for
> inherited fixed parameters were not included as required.
>
> Andi..
>
>  'org::apache::commons::math3::analysis::function'
> ...=
> &::org::apache::commons::math3::analysis::function::PY_TYPE(HarmonicOs...
>    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
> note:
>  expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                  ^
> <scratch space>:63:1: note: expanded from here
> HarmonicOscillator$Parametric$$Type
> ^
> build/_orekit/__wrap__.cpp:217752:55: error: no member named
>  'OrekitMessages$$Type' in namespace 'org::orekit::errors'
>    self->parameters[0] = &::org::orekit::errors::PY_TYPE(OrekitMessages);
>                           ~~~~~~~~~~~~~~~~~~~~~~~^
> /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23:
> note:
>  expanded from macro 'PY_TYPE'
> #define PY_TYPE(name) name##$$Type
>                  ^
> <scratch space>:9:1: note: expanded from here
> OrekitMessages$$Type
> ^
> 2 errors generated.
> error: command 'gcc' failed with exit status 1
>
>
>
> The OrekitMessages is defined as a "public enum OrekitMessages implements
> Localizable { ..."
> and the "public class HarmonicOscillator implements
> UnivariateDifferentiable, DifferentiableUnivariateFunction".
>
> Maybe this says you something, I will try to test more and run my test case
> in .
>
> Best Regards and again, many thanks again!
> /petrus
>
>
>
>
>
> On 11 Jan 2014, at 14:23 , Andi Vajda <vajda@apache.org> wrote:
>
>
> Hi Petrus,
>
> On Mon, 30 Dec 2013, Petrus Hyvönen wrote:
>
> Andi, great to hear that you could reproduce it. I'm very thankful if you
> could have a look at it, I've been struggling to understand how the
> machinery behind this works, but far from it still..
>
>
> I think I fixed the problem and added an improvement to generics support
> along the way. Please check out rev 1557423 from jcc's trunk, rebuild your
> module with it and let me know how it's working for you.
>
> The bug had to do with SimpleClass2's wrapper class missing its parameters
> slot which it should get from its superclass, SimpleClass. When calling a
> method on SimpleClass from SimpleClass2, the missing parameters in the
> wrapper's struct would cause memory to get trashed.
>
> In addition to fixing the bug, I also improved support for fixed class
> parameters. Now, if you change SimpleClass's return_null() method to return
> new Integer(12) instead, when calling it on SimpleClass2, 12 is returned
> instead of <Object: 12>.
>
> Andi..
>
> public class SimpleClass<T> { public SimpleClass() {
> System.out.println("Created SimpleClass"); }
> public T return_null() {
>   return (T) new Integer(12);
> }
>
> }
>
>
>
> Best Regards
> /Petrus
>
>
>
> On Mon, Dec 30, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org> wrote:
>
>
> On Sun, 29 Dec 2013, Petrus Hyvönen wrote:
>
> I have distilled the library that I have some trouble with and I think I
>
> have an example that is failing due to same problem I think. I am not good
> in java, but have tried to follow the logic from the library I'm wrapping.
> The function of the example does not make sense in itself.
>
> public class SimpleClass<T> {
> public SimpleClass() {
> System.out.println("Created SimpleClass");
> }
> public T return_null() {
>  return  null;
> }
>
> }
>
>
>
> public class SimpleClass2 extends SimpleClass<Integer>{
>
> public SimpleClass2(){}
> public void testInJava(){
> System.out.println(this.return_null());
> }
> }
>
>
> It seems to me that there is some problem with methods inherited that
> returns a generic type, failing in wrapType when this is to be wrapped.
>
> The python script that fails:
>
> a= SimpleClass()
> print a.return_null()
>
> b = SimpleClass2()
> b.testInJava()
>
> print b.return_null()  #Fails in wrapType
>
> I don't know if the return null is a bad thing to do in java, but the
> error
> seems very similar to what I experience in the larger library. I have a
> skeleton of that this is slightly larger, but not returning null, but
> trying to keep the lenght of example low :)
>
> Any comments highly appriciated :)
>
>
> I've been able to reproduce the problem.
> Thank you for providing an isolated test case !
>
> Andi..
>
>
>
> best Regards
> /Petrus
>
>
>
>
>
>
> On Fri, Dec 27, 2013 at 6:25 PM, Andi Vajda <vajda@apache.org> wrote:
>
>
> On Dec 27, 2013, at 17:36, Petrus Hyvönen <petrus.hyvonen@gmail.com>
> wrote:
>
>
> Dear Andi,
>
> I am working on debugging the failure and try to understand a bit how
> JCC
> works internally. I haven't gone very far but in case you have some
> pointers from these early debugging sessions I would be very thankful. I
> know it's complex, and I should try to make some smaller test cases, but
>
> I
>
> don't really have a grasp yet where the problem might be.
>
> Writing this might help me also to get some structure in my thinking :)
>
>
>
> The main crash seems to be in the last line, wrapType(), of
> __wrap__.cpp:
>
> static PyObject
>
> *t_AbstractReconfigurableDetector_withHandler(t_
>
> AbstractReconfigurableDetector
>
> *self, PyObject *arg)
>  {
>    ::org::orekit::propagation::events::handlers::EventHandler
> a0((jobject) NULL);
>    PyTypeObject **p0;
>    ::org::orekit::propagation::events::EventDetector
> result((jobject) NULL);
>
>    if (!parseArg(arg, "K",
>
> ::org::orekit::propagation::events::handlers::
>
> EventHandler::initializeClass,
>
> &a0, &p0,
>
> ::org::orekit::propagation::events::handlers::t_
>
> EventHandler::parameters_))
>
>    {
>      OBJ_CALL(result = self->object.withHandler(a0));
>      return self->parameters[0] != NULL ?
> wrapType(self->parameters[0], result.this$) :
> ::org::orekit::propagation::events::t_EventDetector::wrap_
> Object(result);
>    }
>
> The parameters[0] does not seem to be null, but neither is it a valid
> object, in my debugger it says 0xbaadf00d {ob_refcnt=??? ob_type=???
> ob_size=??? ...}     _typeobject *, wrapType is called and when trying
> to
> access the wrapfn it crashes.
>
> The main python lines are:
> tmp1 = ElevationDetector(sta1Frame)                    #
>
> ElevationDetector
>
> is a java public class ElevationDetector extends
> AbstractReconfigurableDetector<ElevationDetector>
> hand = ContinueOnEvent().of_(ElevationDetector)   # a
> java ContinueOnEvent<ElevationDetector> object
> elDetector = tmp1.withHandler(hand)     #Crash. withHandler is a method
> that is inherited from AbstractReconfigurableDetector to
>
> ElevationDetector
>
>
> This crashes when interactively entered on the python prompt (or in
> other
> interactive consoles), but seems to work if executed directly without
> interactivity. This difference makes me think that it might be something
> with garbage collection, but don't know.
>
> Any comments appriciated, I know this is likely very difficult to
> comment
> on as it's not very encapsulated.
>
>
> Right, so unless you can isolate this into something I can reproduce, I'm
> afraid there isn't much I can comment.
> It is quite likely that by the time you have that reproducible test case
> ready, you also have the solution to the problem. Or I might be able to
> help then...
>
> Andi..
>
>
> Regards
> /Petrus
>
>
>
>
>
>
> On Sun, Dec 15, 2013 at 10:36 AM, Andi Vajda <vajda@apache.org> wrote:
>
>
> On Dec 15, 2013, at 5:43, Petrus Hyvönen <petrus.hyvonen@gmail.com>
>
> wrote:
>
> Hi Andi,
>
> I see your point and have now kept in the pure python domain.
>
> If I run my script from the shell by "python script.py" it does not
>
> crash. However if I execute it line-by-line in python it crashes (or in
> other tools such as ipython notebook).
>
> All classes used are non-wrapped java classes, but I get the same
>
> effect
>
>
> with classes made for python subclassing.
>
>
> I am getting this on both MacOSX 64-bit python and Windows 7 32-bit
>
> python.
>
>
>
> elDetector =
>
> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>
>
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x000000010005da1a, pid=3318, tid=1287
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build
>
> 1.7.0_45-b18)
>
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode
>
> bsd-amd64 compressed oops)
>
> # Problematic frame:
> # C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
> #
>
>
> from the stack it seems like there is somthing happening in "wrapType"
> Stack: [0x00007fff5fb80000,0x00007fff5fc00000],
> sp=0x00007fff5fbff470,
>
> free space=509k
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
>
> C=native code)
>
> C  [libpython2.7.dylib+0x5aa1a]  PyObject_GetAttr+0x1a
> C  [_orekit.so+0xa80878]  wrapType(_typeobject*, _jobject*
> const&)+0x58
> C  [_orekit.so+0x554400]
>
>
> org::orekit::propagation::events::t_AbstractReconfigurableDetector
>
> _withHandler(org::orekit::propagation::events::t_
> AbstractReconfigurableDetector*,
>
> _object*)+0x1c0
>
>
>
> First, is the generic class assignment correct as if to write "new
>
> ContinueOnEvent<ElevationDetector>()" in java? And is it ok to use
>
> regular
>
>
> java objects/types?
>
>
>
> Any other comments to move forward highly appriciated.. Is it somehow
>
> possible to get more log what is going wrong?
>
> You could compile the whole thing for debugging, by adding --debug
> after
> 'build' in the jcc invocation and run it with gdb.
>
> If you can isolate a reproducible crash into a small test case, I can
>
> also
>
>
> take a look at it.
>
>
> Andi..
>
>
> WIth best regards
> /Petrus
>
>
>
> On 15 Dec 2013, at 2:40 , Andi Vajda <vajda@apache.org> wrote:
>
>
> On Dec 14, 2013, at 19:14, Petrus Hyvönen <petrus.hyvonen@gmail.com
> wrote:
>
>
>
> Hi,
>
> I'm having a problem with I think might be related to generic types,
>
> but not sure at all.
>
>
>
> I'm wrapping a orbit calculation library, which has been working
> well
>
> but in latest version is using generic types and I'm getting some
>
> problems.
>
>
> The script works when executed in plain python, but fails in ipython
>
> notebook on this last line when executed as a couple of cells.
>
>
> What is an 'ipython notebook' ?
>
> Andi.,
>
>
> The section with problem in my script is:
> elDetector =
>
> ElevationDetector(sta1Frame).withConstantElevation(math.
>
> radians(5.0))
>
> elDetector =
>
> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>
>
>
> In Java it would typically look something like:
>
> ElevationDetector detector = new ElevationDetector(topo)
>                                   .withConstantElevation(x)
>                                   .withHandler(new
>
> ContinueOnEvent<ElevationDetector>());
>
>
>
> It produces correct results in plain python, but crashes the kernel
>
> in
>
>
> ipython if executed as cells, and in exection from spyder I get an error
>
> message:
>
>
> " elDetector =
>
> elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector))
>
>
> AttributeError: 'str' object has no attribute 'wrapfn_' "
>
>
> As I have been using this setup stabely with lots of other functions
>
> it feels like there is something with the generic type line, but I
>
> don't
> really know how to get any further? I'm confused by that the pauses in
>
> the
>
>
> execution could seem to affect the result.
>
>
>
> Any comments highly appriciated...
>
> Best Regards
> /Petrus
>
>
>
> --
> _____________________________________________
> Petrus Hyvönen, Uppsala, Sweden
> Mobile Phone/SMS:+46 73 803 19 00
>
>
>
> --
> _____________________________________________
> Petrus Hyvönen, Uppsala, Sweden
> Mobile Phone/SMS:+46 73 803 19 00
>
>
>
> -- 
> _____________________________________________
> Petrus Hyvönen, Uppsala, Sweden
> Mobile Phone/SMS:+46 73 803 19 00
>
Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message