Return-Path: X-Original-To: apmail-lucene-pylucene-dev-archive@minotaur.apache.org Delivered-To: apmail-lucene-pylucene-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C26EE10504 for ; Mon, 3 Feb 2014 00:15:48 +0000 (UTC) Received: (qmail 98016 invoked by uid 500); 3 Feb 2014 00:15:48 -0000 Delivered-To: apmail-lucene-pylucene-dev-archive@lucene.apache.org Received: (qmail 97935 invoked by uid 500); 3 Feb 2014 00:15:48 -0000 Mailing-List: contact pylucene-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: pylucene-dev@lucene.apache.org Delivered-To: mailing list pylucene-dev@lucene.apache.org Received: (qmail 97927 invoked by uid 99); 3 Feb 2014 00:15:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Feb 2014 00:15:48 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [50.0.193.30] (HELO ovaltofu.org) (50.0.193.30) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Feb 2014 00:15:40 +0000 Received: from [192.168.0.34] ([192.168.0.34]) (authenticated bits=0) by ovaltofu.org (8.14.4/8.14.4) with ESMTP id s130FEsl024044 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 2 Feb 2014 16:15:14 -0800 (PST) Date: Sun, 2 Feb 2014 16:15:17 -0800 (PST) From: Andi Vajda X-X-Sender: vajda@yuzu.local Reply-To: Andi Vajda To: pylucene-dev Subject: Re: Problem using generic types? In-Reply-To: Message-ID: References: <510160A1-CBEB-4FC1-826F-5074410EF2F9@apache.org> <71323F49-7C8E-4E4A-957D-C4FA178B6855@gmail.com> <21EBE0F5-06CE-4969-978D-D773DED75B03@apache.org> <288ABD0F-E844-4491-A16A-A42B03E9CB62@gmail.com> <5AB48BB3-FF7E-4154-B6EB-BF5D1BB9B3B9@apache.org> <6A1EE011-889C-4F45-A68C-7363B2DD239B@apache.org> <6BB3166C-283E-4DFF-A2F0-E5DCBD1F44D2@apache.org> <568AF072-E8AF-4124-BB32-21946891FAEA@gmail.com> User-Agent: Alpine 2.01 (OSX 1266 2009-07-14) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1464266112-1391386517=:11010" X-Virus-Checked: Checked by ClamAV on apache.org --0-1464266112-1391386517=:11010 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT On Sun, 2 Feb 2014, Petrus Hyv�nen wrote: > Yes, the confusing thing is that it works well under windows. I compared > the code in __wrap__ and it looks different in the windows version: > > static int t_PointVectorValuePair_init_(t_PointVectorValuePair *self, > PyObject *args, PyObject *kwds) > { > switch (PyTuple_GET_SIZE(args)) { > case 2: > { > JArray< jdouble > a0((jobject) NULL); > JArray< jdouble > a1((jobject) NULL); > PointVectorValuePair object((jobject) NULL); > > if (!parseArgs(args, "[D[D", &a0, &a1)) > { > INT_CALL(object = PointVectorValuePair(a0, a1)); > self->object = object; > break; > } > } > > > So no Parameters[] stuff in windows version.. On mac I'm using java v > 1.7.0_45 and 1.6.0_35 under windows. could that make a difference? That could also make a difference but there is a bug nonetheless. Andi.. > > I'm using a mvn downloaded version of the apache.math jar, but source is: > > public class PointVectorValuePair extends Pair > implements Serializable { > /** Serializable UID. */ > private static final long serialVersionUID = 20120513L; > > /** > ... > */ > public PointVectorValuePair(final double[] point, > final double[] value) { > this(point, value, true); > } > > /** > ... > */ > public PointVectorValuePair(final double[] point, > final double[] value, > final boolean copyArray) { > super(copyArray ? > ((point == null) ? null : > point.clone()) : > point, > copyArray ? > ((value == null) ? null : > value.clone()) : > value); > } > > ... > > full code at > http://commons-math/jacoco/org.apache.commons.math3.optimization/PointVectorValuePair.java.html > > > > Regards > /petrus > > > On 02 Feb 2014, at 22:54 , Andi Vajda wrote: > > > On Feb 2, 2014, at 13:41, Petrus Hyv�nen wrote: > > Hi, > > It's part of a class "PointVectorValuePair" and "PointValuePair" in the > apache math commons ( > http://commons.apache.org/proper/commons-math/javadocs/api-3.2/org/apache/commons/math3/optimization/PointVectorValuePair.html > ) > > I have tried to reserve a bunch of things like 'Point', 'Value', 'Pair' > without any difference. If I go through the __wrap__ file, it looks like > this [D object to wrap is a very odd one, the other PY_TYPE wrapping seems > to have 'proper' names without special characters. It's the functions > PointValuePair and PointVectorValuePair that seems to have this strage > class wrapping. I cannot find a class named D. From the API doc's I would > guess it's trying to set a double [] type. > > extract from __wrap__: > > under namespace org { > namespace apache { > namespace commons { > namespace math3 { > namespace optimization { > > ... > > static int t_PointVectorValuePair_init_(t_PointVectorValuePair > *self, PyObject *args, PyObject *kwds) > { > switch (PyTuple_GET_SIZE(args)) { > case 2: > { > JArray< jdouble > a0((jobject) NULL); > JArray< jdouble > a1((jobject) NULL); > PointVectorValuePair object((jobject) NULL); > > if (!parseArgs(args, "[D[D", &a0, &a1)) > > > It could just be that you found a bug in jcc's handling of generic > parameters that are arrays. I suspect you have some double[] (or worse: > double[][]) somewhere in the java source. > But then this should fail the same way on Windows. > Can you please post the original java source for that class here ? > > Andi.. > > { > INT_CALL(object = PointVectorValuePair(a0, a1)); > self->object = object; > self->parameters[0] = &::PY_TYPE([D); > self->parameters[1] = &::PY_TYPE([D); > break; > } > > Regards > /petrus > > > > = &::PY_TYPE([D); > ^ > /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > build/_orekit/__wrap__.cpp:58329:44: error: use of undeclared identifier > 'D$$Type' > self->parameters[0] = &::PY_TYPE([D); > ^ > /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > :109:1: note: expanded from here > D$$Type > ^ > build/_orekit/__wrap__.cpp:58329:55: error: expected ']' > self->parameters[0] = &::PY_TYPE([D); > ^ > build/_orekit/__wrap__.cpp:58329:52: note: to match this '[' > self->parameters[0] = &::PY_TYPE([D); > > > On 02 Feb 2014, at 19:48 , Andi Vajda wrote: > > > On Feb 2, 2014, at 10:08, Petrus Hyv�nen wrote: > > Hi Andi and Others, > > The latest trunk jcc works and builds very fine on my windows machine, and > happy about the extendend support of generics. But I am experiencing the > problem below when building on my mac, using same wrap parameters as on the > windows platform. To me it seems like some compiler problem related to > macros, but don't know. > > Does anyone have experience of these failures? > > Regards > /Petrus > > > error: expected unqualified-id > self->parameters[0] = &::PY_TYPE([D); > ^ > /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > build/_orekit/__wrap__.cpp:2344:44: error: use of undeclared identifier > 'D$$Type' > self->parameters[0] = &::PY_TYPE([D); > > > Take a look at the code around line 2344 in that __wrap__.cpp file and > figure out what java code this corresponds to. You might be using a > classname that's a C macro on some platforms - do you have a class named D > for example ? > If you're hitting such a name conflict, add that name to the --reserved > list passed to jcc. > > Andi.. > > ^ > /Users/petrus/anaconda/envs/_build/lib/python2.7/site-packages/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > :73:1: note: expanded from here > D$$Type > ^ > build/_orekit/__wrap__.cpp:2344:55: error: expected ']' > self->parameters[0] = &::PY_TYPE([D); > ^ > build/_orekit/__wrap__.cpp:2344:52: note: to match this '[' > self->parameters[0] = &::PY_TYPE([D); > > > > On 14 Jan 2014, at 19:02 , Andi Vajda wrote: > > > Hi Petrus, > > On Jan 14, 2014, at 4:24, Petrus Hyv�nen wrote: > > Dear Andi, > > Many many thanks for looking into this! > > I am on travel for a week and do not have the computer with the special > case with me to test. I did now however run it on the library that I am > wrapping, and there get some errors in the creation of the wrapped module > (orekit): > > build/_orekit/__wrap__.cpp:86504:89: error: no member named > 'HarmonicOscillator$Parametric$$Type' in namespace > > > I believe I fixed this bug in rev 1557613, header files for classes for > inherited fixed parameters were not included as required. > > Andi.. > > 'org::apache::commons::math3::analysis::function' > ...= > &::org::apache::commons::math3::analysis::function::PY_TYPE(HarmonicOs... > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ > /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > :63:1: note: expanded from here > HarmonicOscillator$Parametric$$Type > ^ > build/_orekit/__wrap__.cpp:217752:55: error: no member named > 'OrekitMessages$$Type' in namespace 'org::orekit::errors' > self->parameters[0] = &::org::orekit::errors::PY_TYPE(OrekitMessages); > ~~~~~~~~~~~~~~~~~~~~~~~^ > /Users/petrus/anaconda/lib/python2.7/site-packages/JCC-2.18-py2.7-macosx-10.5-x86_64.egg/jcc/sources/macros.h:66:23: > note: > expanded from macro 'PY_TYPE' > #define PY_TYPE(name) name##$$Type > ^ > :9:1: note: expanded from here > OrekitMessages$$Type > ^ > 2 errors generated. > error: command 'gcc' failed with exit status 1 > > > > The OrekitMessages is defined as a "public enum OrekitMessages implements > Localizable { ..." > and the "public class HarmonicOscillator implements > UnivariateDifferentiable, DifferentiableUnivariateFunction". > > Maybe this says you something, I will try to test more and run my test case > in . > > Best Regards and again, many thanks again! > /petrus > > > > > > On 11 Jan 2014, at 14:23 , Andi Vajda wrote: > > > Hi Petrus, > > On Mon, 30 Dec 2013, Petrus Hyv�nen wrote: > > Andi, great to hear that you could reproduce it. I'm very thankful if you > could have a look at it, I've been struggling to understand how the > machinery behind this works, but far from it still.. > > > I think I fixed the problem and added an improvement to generics support > along the way. Please check out rev 1557423 from jcc's trunk, rebuild your > module with it and let me know how it's working for you. > > The bug had to do with SimpleClass2's wrapper class missing its parameters > slot which it should get from its superclass, SimpleClass. When calling a > method on SimpleClass from SimpleClass2, the missing parameters in the > wrapper's struct would cause memory to get trashed. > > In addition to fixing the bug, I also improved support for fixed class > parameters. Now, if you change SimpleClass's return_null() method to return > new Integer(12) instead, when calling it on SimpleClass2, 12 is returned > instead of . > > Andi.. > > public class SimpleClass { public SimpleClass() { > System.out.println("Created SimpleClass"); } > public T return_null() { > return (T) new Integer(12); > } > > } > > > > Best Regards > /Petrus > > > > On Mon, Dec 30, 2013 at 10:36 AM, Andi Vajda wrote: > > > On Sun, 29 Dec 2013, Petrus Hyv�nen wrote: > > I have distilled the library that I have some trouble with and I think I > > have an example that is failing due to same problem I think. I am not good > in java, but have tried to follow the logic from the library I'm wrapping. > The function of the example does not make sense in itself. > > public class SimpleClass { > public SimpleClass() { > System.out.println("Created SimpleClass"); > } > public T return_null() { > return null; > } > > } > > > > public class SimpleClass2 extends SimpleClass{ > > public SimpleClass2(){} > public void testInJava(){ > System.out.println(this.return_null()); > } > } > > > It seems to me that there is some problem with methods inherited that > returns a generic type, failing in wrapType when this is to be wrapped. > > The python script that fails: > > a= SimpleClass() > print a.return_null() > > b = SimpleClass2() > b.testInJava() > > print b.return_null() #Fails in wrapType > > I don't know if the return null is a bad thing to do in java, but the > error > seems very similar to what I experience in the larger library. I have a > skeleton of that this is slightly larger, but not returning null, but > trying to keep the lenght of example low :) > > Any comments highly appriciated :) > > > I've been able to reproduce the problem. > Thank you for providing an isolated test case ! > > Andi.. > > > > best Regards > /Petrus > > > > > > > On Fri, Dec 27, 2013 at 6:25 PM, Andi Vajda wrote: > > > On Dec 27, 2013, at 17:36, Petrus Hyv�nen > wrote: > > > Dear Andi, > > I am working on debugging the failure and try to understand a bit how > JCC > works internally. I haven't gone very far but in case you have some > pointers from these early debugging sessions I would be very thankful. I > know it's complex, and I should try to make some smaller test cases, but > > I > > don't really have a grasp yet where the problem might be. > > Writing this might help me also to get some structure in my thinking :) > > > > The main crash seems to be in the last line, wrapType(), of > __wrap__.cpp: > > static PyObject > > *t_AbstractReconfigurableDetector_withHandler(t_ > > AbstractReconfigurableDetector > > *self, PyObject *arg) > { > ::org::orekit::propagation::events::handlers::EventHandler > a0((jobject) NULL); > PyTypeObject **p0; > ::org::orekit::propagation::events::EventDetector > result((jobject) NULL); > > if (!parseArg(arg, "K", > > ::org::orekit::propagation::events::handlers:: > > EventHandler::initializeClass, > > &a0, &p0, > > ::org::orekit::propagation::events::handlers::t_ > > EventHandler::parameters_)) > > { > OBJ_CALL(result = self->object.withHandler(a0)); > return self->parameters[0] != NULL ? > wrapType(self->parameters[0], result.this$) : > ::org::orekit::propagation::events::t_EventDetector::wrap_ > Object(result); > } > > The parameters[0] does not seem to be null, but neither is it a valid > object, in my debugger it says 0xbaadf00d {ob_refcnt=??? ob_type=??? > ob_size=??? ...} _typeobject *, wrapType is called and when trying > to > access the wrapfn it crashes. > > The main python lines are: > tmp1 = ElevationDetector(sta1Frame) # > > ElevationDetector > > is a java public class ElevationDetector extends > AbstractReconfigurableDetector > hand = ContinueOnEvent().of_(ElevationDetector) # a > java ContinueOnEvent object > elDetector = tmp1.withHandler(hand) #Crash. withHandler is a method > that is inherited from AbstractReconfigurableDetector to > > ElevationDetector > > > This crashes when interactively entered on the python prompt (or in > other > interactive consoles), but seems to work if executed directly without > interactivity. This difference makes me think that it might be something > with garbage collection, but don't know. > > Any comments appriciated, I know this is likely very difficult to > comment > on as it's not very encapsulated. > > > Right, so unless you can isolate this into something I can reproduce, I'm > afraid there isn't much I can comment. > It is quite likely that by the time you have that reproducible test case > ready, you also have the solution to the problem. Or I might be able to > help then... > > Andi.. > > > Regards > /Petrus > > > > > > > On Sun, Dec 15, 2013 at 10:36 AM, Andi Vajda wrote: > > > On Dec 15, 2013, at 5:43, Petrus Hyv�nen > > wrote: > > Hi Andi, > > I see your point and have now kept in the pure python domain. > > If I run my script from the shell by "python script.py" it does not > > crash. However if I execute it line-by-line in python it crashes (or in > other tools such as ipython notebook). > > All classes used are non-wrapped java classes, but I get the same > > effect > > > with classes made for python subclassing. > > > I am getting this on both MacOSX 64-bit python and Windows 7 32-bit > > python. > > > > elDetector = > > elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector)) > > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x000000010005da1a, pid=3318, tid=1287 > # > # JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build > > 1.7.0_45-b18) > > # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode > > bsd-amd64 compressed oops) > > # Problematic frame: > # C [libpython2.7.dylib+0x5aa1a] PyObject_GetAttr+0x1a > # > > > from the stack it seems like there is somthing happening in "wrapType" > Stack: [0x00007fff5fb80000,0x00007fff5fc00000], > sp=0x00007fff5fbff470, > > free space=509k > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > > C=native code) > > C [libpython2.7.dylib+0x5aa1a] PyObject_GetAttr+0x1a > C [_orekit.so+0xa80878] wrapType(_typeobject*, _jobject* > const&)+0x58 > C [_orekit.so+0x554400] > > > org::orekit::propagation::events::t_AbstractReconfigurableDetector > > _withHandler(org::orekit::propagation::events::t_ > AbstractReconfigurableDetector*, > > _object*)+0x1c0 > > > > First, is the generic class assignment correct as if to write "new > > ContinueOnEvent()" in java? And is it ok to use > > regular > > > java objects/types? > > > > Any other comments to move forward highly appriciated.. Is it somehow > > possible to get more log what is going wrong? > > You could compile the whole thing for debugging, by adding --debug > after > 'build' in the jcc invocation and run it with gdb. > > If you can isolate a reproducible crash into a small test case, I can > > also > > > take a look at it. > > > Andi.. > > > WIth best regards > /Petrus > > > > On 15 Dec 2013, at 2:40 , Andi Vajda wrote: > > > On Dec 14, 2013, at 19:14, Petrus Hyv�nen wrote: > > > > Hi, > > I'm having a problem with I think might be related to generic types, > > but not sure at all. > > > > I'm wrapping a orbit calculation library, which has been working > well > > but in latest version is using generic types and I'm getting some > > problems. > > > The script works when executed in plain python, but fails in ipython > > notebook on this last line when executed as a couple of cells. > > > What is an 'ipython notebook' ? > > Andi., > > > The section with problem in my script is: > elDetector = > > ElevationDetector(sta1Frame).withConstantElevation(math. > > radians(5.0)) > > elDetector = > > elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector)) > > > > In Java it would typically look something like: > > ElevationDetector detector = new ElevationDetector(topo) > .withConstantElevation(x) > .withHandler(new > > ContinueOnEvent()); > > > > It produces correct results in plain python, but crashes the kernel > > in > > > ipython if executed as cells, and in exection from spyder I get an error > > message: > > > " elDetector = > > elDetector.withHandler(ContinueOnEvent().of_(ElevationDetector)) > > > AttributeError: 'str' object has no attribute 'wrapfn_' " > > > As I have been using this setup stabely with lots of other functions > > it feels like there is something with the generic type line, but I > > don't > really know how to get any further? I'm confused by that the pauses in > > the > > > execution could seem to affect the result. > > > > Any comments highly appriciated... > > Best Regards > /Petrus > > > > -- > _____________________________________________ > Petrus Hyv�nen, Uppsala, Sweden > Mobile Phone/SMS:+46 73 803 19 00 > > > > -- > _____________________________________________ > Petrus Hyv�nen, Uppsala, Sweden > Mobile Phone/SMS:+46 73 803 19 00 > > > > -- > _____________________________________________ > Petrus Hyv�nen, Uppsala, Sweden > Mobile Phone/SMS:+46 73 803 19 00 > --0-1464266112-1391386517=:11010--