apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Yan <da...@datatorrent.com>
Subject Re: Best practises to specify the Application
Date Fri, 29 Apr 2016 17:54:37 GMT
Hi Ananth,

Please always use the archetype to generate your project.
In the archetype, among many other things, it also enforces that the target
java class files to be compatible with JDK 1.7.
And it seems that your project is missing that as well and that's why you
see the UnsupportedClassVersionError  error.

David

On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:

> I had a look at the dtgateway.log and observed the following stacktrace :
> I guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
>>
>> 2016-04-29 08:04:45,181 ERROR
>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
>> com/tx/y/z/Operator : Unsupported major.minor version 52.0
>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>         at
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>         at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>         at
>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
>>         at
>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
>>         at
>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
>>         at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>>
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Thanks Shubham.
>>
>> Those properties were being set by the parent pom which are inherited by
>> the current pom.xml ( Kind of wanted all apex apps to have same version
>> structure etc) and hence added that property as part of parent pom.
>>
>> I am still stumped as to what is causing this.
>>
>> Regards,
>> Ananth
>>
>>
>>
>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com>
>> wrote:
>>
>>> Hi Ananth,
>>>
>>> I tried $ mvn archetype:generate  -DarchetypeGroupId=org.apache.apex
>>>  -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>>>  -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>>>  -Dversion=1.0-SNAPSHOT
>>> and it works for me as well.
>>>
>>> I compared the pom.xml file and noticed values for following properties
>>> were missing.
>>>      <manifestEntries>
>>>
>>>                    <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>>>                    <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>>
>>>      </manifestEntries>
>>>
>>> Could you add the following to pom.xml and try again.
>>>
>>>  <properties>
>>>     <!-- change this if you desire to use a different version of Apex Core
-->
>>>     <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>>>     <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>>>   </properties>
>>>
>>>
>>> Thanks,
>>> Shubham
>>>
>>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> I reattempted the packaging with the archetype based approach. I guess
>>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>>> pointing it out.
>>>>
>>>> The default package generated by the archetype command seems to be
>>>> working fine when uploading it but the moment I add in my code and
>>>> dependencies it goes back the issue I am seeing :
>>>>
>>>>
>>>> My app package looks like app.zip  ( I removed all of the
>>>> application.class for brevity sake ) . Please note that I deleted all the
>>>> jars under /lib folder and the contents of the /app folder in the apa
>>>> pacakge.
>>>>
>>>> I have also attached a copy of my pom.xml file.
>>>>
>>>> Could you please advise if you something amiss right away in the
>>>> structure below ? It may be noted that I retained the classes of the
>>>> original simple project generated by the archetype and even they stopped
>>>> showing with the launch button the moment I added in all of the application
>>>> dependencies in maven.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <david@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> I just tried with 3.3.0-incubating with the following and it's
>>>>> working:
>>>>>
>>>>> $ mvn archetype:generate  -DarchetypeGroupId=org.apache.apex
>>>>>  -DarchetypeArtifactId=apex-app-archetype
>>>>> -DarchetypeVersion=3.3.0-incubating  -DgroupId=com.example
>>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp  -Dversion=1.0-SNAPSHOT
>>>>>
>>>>> Let me know whether this works for you or not.
>>>>>
>>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>>> on the app package and it looks at all the jars in the app directory
and
>>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>>> app for 3.3.0 version and I was not able to specify an archetype
with the
>>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype
for
>>>>>> version 3.3.0 ? I was not sure if the archetype version should match
the
>>>>>> exact apex engine version . My presumption was archetype does not
matter if
>>>>>> the packaging is done right. So i went ahead and created a simple
maven
>>>>>> project.
>>>>>>
>>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>>> constructs to build the exact file layout of the app as mentioned
in the
>>>>>> output above.
>>>>>>
>>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>>
>>>>>> This brings me to a question. How does the engine know which is the
>>>>>> main class that is to be used for launching the Streaming App. I
do have a
>>>>>> class that extends the StreaminApplication that is annotated with
>>>>>> ApplicationAnnotation but does not seem to be picking up. By this
I mean, I
>>>>>> do not see a "launch" icon next to it. All I see is the top level
packaged
>>>>>> name in the web console but nothing beneath it for me to trigger
a launch.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <david@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>>> if the archetype is used. For example, the MANFEST.MF file from
the Pi demo
>>>>>>> in Malhar has this content:
>>>>>>>
>>>>>>> Manifest-Version: 1.0
>>>>>>> Archiver-Version: Plexus Archiver
>>>>>>> Built-By: david
>>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating)
Pi Demo
>>>>>>> Class-Path: lib/*.jar
>>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>> Created-By: Apache Maven 3.3.9
>>>>>>> Build-Jdk: 1.8.0_92
>>>>>>> DT-App-Package-Description: Apex demo applications that calculate
>>>>>>> the
>>>>>>>  value of Pi. This is a starting point to understand how Apex
works.
>>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>>> DT-App-Package-Name: pi-demo
>>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>
>>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello David,
>>>>>>>>
>>>>>>>> Please find the unzip result at the end of this email. The
jar is
>>>>>>>> in the app directory inside the apa.
>>>>>>>> As to why I did not use the archetype, I was building the
app for
>>>>>>>> 3.3.0 version and I was not able to specify an archetype
with the version
>>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does
not matter if
>>>>>>>> the packaging is done right. So i went ahead and created
a simple maven
>>>>>>>> project.
>>>>>>>>
>>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in
the
>>>>>>>> listing below ) . I also pasted the contents of the properties.xml
file at
>>>>>>>> the end of this email for your reference.
>>>>>>>>
>>>>>>>> Archive:  cassandra-events-1.2-SNAPSHOT.apa
>>>>>>>>     testing: META-INF/                OK
>>>>>>>>     testing: META-INF/MANIFEST.MF     OK
>>>>>>>>     testing: app/                     OK
>>>>>>>>     testing: app/cassandra-events-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/                     OK
>>>>>>>>     testing: lib/activation-1.1.jar   OK
>>>>>>>>     testing: lib/activemq-client-5.8.0.jar   OK
>>>>>>>>     testing: lib/antlr-2.7.7.jar      OK
>>>>>>>>     testing: lib/aopalliance-1.0.jar   OK
>>>>>>>>     testing: lib/app-commons-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/aspectjrt-1.8.7.jar   OK
>>>>>>>>     testing: lib/btf-1.2.jar          OK
>>>>>>>>     testing: lib/cassandra-driver-core-2.1.8.jar   OK
>>>>>>>>     testing: lib/commons-beanutils-1.8.3.jar   OK
>>>>>>>>     testing: lib/commons-compiler-2.7.8.jar   OK
>>>>>>>>     testing: lib/commons-lang3-3.1.jar   OK
>>>>>>>>     testing: lib/commons-logging-1.1.3.jar   OK
>>>>>>>>     testing: lib/curator-client-2.7.0.jar   OK
>>>>>>>>     testing: lib/curator-framework-2.7.0.jar   OK
>>>>>>>>     testing: lib/dom4j-1.6.1.jar      OK
>>>>>>>>     testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/dwh-kafka-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar   OK
>>>>>>>>     testing: lib/evo-inflector-1.2.1.jar   OK
>>>>>>>>     testing: lib/fastutil-6.6.4.jar   OK
>>>>>>>>     testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar
  OK
>>>>>>>>     testing: lib/geronimo-jms_1.1_spec-1.1.1.jar   OK
>>>>>>>>     testing: lib/guava-14.0.1.jar     OK
>>>>>>>>     testing: lib/hawtbuf-1.9.jar      OK
>>>>>>>>     testing: lib/hibernate-commons-annotations-5.0.0.Final.jar
  OK
>>>>>>>>     testing: lib/hibernate-core-5.0.4.Final.jar   OK
>>>>>>>>     testing: lib/hibernate-entitymanager-5.0.4.Final.jar
  OK
>>>>>>>>     testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar  
OK
>>>>>>>>     testing: lib/HikariCP-2.4.3.jar   OK
>>>>>>>>     testing: lib/jackson-annotations-2.6.4.jar   OK
>>>>>>>>     testing: lib/jackson-core-2.6.4.jar   OK
>>>>>>>>     testing: lib/jackson-coreutils-1.6.jar   OK
>>>>>>>>     testing: lib/jackson-databind-2.6.4.jar   OK
>>>>>>>>     testing: lib/jandex-2.0.0.CR1.jar   OK
>>>>>>>>     testing: lib/javassist-3.18.1-GA.jar   OK
>>>>>>>>     testing: lib/javax.mail-1.5.0.jar   OK
>>>>>>>>     testing: lib/jboss-logging-3.3.0.Final.jar   OK
>>>>>>>>     testing: lib/jcl-over-slf4j-1.7.13.jar   OK
>>>>>>>>     testing: lib/jersey-client-1.9.jar   OK
>>>>>>>>     testing: lib/jersey-core-1.9.jar   OK
>>>>>>>>     testing: lib/jetty-continuation-8.1.10.v20130312.jar
  OK
>>>>>>>>     testing: lib/jetty-http-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-io-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-security-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-server-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-servlet-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-util-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jetty-websocket-8.1.10.v20130312.jar   OK
>>>>>>>>     testing: lib/jline-2.11.jar       OK
>>>>>>>>     testing: lib/jms-api-1.1-rev-1.jar   OK
>>>>>>>>     testing: lib/joda-time-2.7.jar    OK
>>>>>>>>     testing: lib/json-patch-1.7.jar   OK
>>>>>>>>     testing: lib/jsr305-2.0.1.jar     OK
>>>>>>>>     testing: lib/jta-1.1.jar          OK
>>>>>>>>     testing: lib/kafka-clients-0.9.0.0.jar   OK
>>>>>>>>     testing: lib/log4j-1.2.17.jar     OK
>>>>>>>>     testing: lib/logback-classic-1.1.5.jar   OK
>>>>>>>>     testing: lib/logback-core-1.1.5.jar   OK
>>>>>>>>     testing: lib/lz4-1.2.0.jar        OK
>>>>>>>>     testing: lib/malhar-contrib-3.3.1-incubating.jar   OK
>>>>>>>>     testing: lib/malhar-kafka-3.3.1-incubating.jar   OK
>>>>>>>>     testing: lib/malhar-library-3.3.1-incubating.jar   OK
>>>>>>>>     testing: lib/metrics-core-3.0.2.jar   OK
>>>>>>>>     testing: lib/msg-simple-1.1.jar   OK
>>>>>>>>     testing: lib/mysql-connector-java-5.1.36.jar   OK
>>>>>>>>     testing: lib/named-regexp-0.2.3.jar   OK
>>>>>>>>     testing: lib/netty-3.6.2.Final.jar   OK
>>>>>>>>     testing: lib/netty-buffer-4.0.27.Final.jar   OK
>>>>>>>>     testing: lib/netty-codec-4.0.27.Final.jar   OK
>>>>>>>>     testing: lib/netty-common-4.0.27.Final.jar   OK
>>>>>>>>     testing: lib/netty-handler-4.0.27.Final.jar   OK
>>>>>>>>     testing: lib/netty-transport-4.0.27.Final.jar   OK
>>>>>>>>     testing: lib/opencsv-2.3.jar      OK
>>>>>>>>     testing: lib/slf4j-api-1.7.10.jar   OK
>>>>>>>>     testing: lib/snappy-java-1.1.1.7.jar   OK
>>>>>>>>     testing: lib/spring-aop-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-beans-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-context-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-core-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-data-commons-1.11.4.RELEASE.jar 
 OK
>>>>>>>>     testing: lib/spring-data-jpa-1.9.4.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar
  OK
>>>>>>>>     testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar
  OK
>>>>>>>>     testing: lib/spring-expression-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-hateoas-0.19.0.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-jdbc-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-orm-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-plugin-core-1.2.0.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-tx-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-web-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/spring-webmvc-4.1.9.RELEASE.jar   OK
>>>>>>>>     testing: lib/super-csv-2.3.1.jar   OK
>>>>>>>>     testing: lib/super-csv-joda-2.3.1.jar   OK
>>>>>>>>     testing: lib/xml-apis-1.0.b2.jar   OK
>>>>>>>>     testing: lib/zookeeper-3.4.6.jar   OK
>>>>>>>>     testing: conf/                    OK
>>>>>>>>     testing: conf/properties.xml      OK
>>>>>>>> No errors detected in compressed data of
>>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>>
>>>>>>>> Contents of the properties.xml file under the conf folder
is here:
>>>>>>>>
>>>>>>>> <?xml version="1.0"?>
>>>>>>>> <configuration>
>>>>>>>>     <property>
>>>>>>>>         <name>dt.application.CED.class</name>
>>>>>>>>         <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>>>         <description>An alias for the application</description>
>>>>>>>>     </property>
>>>>>>>>     <property>
>>>>>>>>         <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>>>         <value>512</value>
>>>>>>>>     </property>
>>>>>>>>
>>>>>>>>     <property>
>>>>>>>>         <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>>>         <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>>>     </property>
>>>>>>>>
>>>>>>>>
>>>>>>>>     <property>
>>>>>>>>         <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>>>         <value>20</value>
>>>>>>>>     </property>
>>>>>>>>
>>>>>>>> </configuration>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <david@datatorrent.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Ananth,
>>>>>>>>>
>>>>>>>>> Can you do an "unzip -t" on your apa file and paste the
output
>>>>>>>>> here?
>>>>>>>>> Make sure the jar containing the application classes
should be in
>>>>>>>>> the app directory inside the apa.
>>>>>>>>>
>>>>>>>>> Can I know why the apex-app-archetype is not used in
your apa
>>>>>>>>> project?
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula
<
>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello All,
>>>>>>>>>>
>>>>>>>>>> I am having the following issue with my application.
I have a
>>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>>> StreamingApplication.
>>>>>>>>>>
>>>>>>>>>> My maven packaging is ensuring that the final "apa"
extension
>>>>>>>>>> package has the right folder structure if exploded
on a disk.
>>>>>>>>>>
>>>>>>>>>> However I am not able to locate the Application name
and the
>>>>>>>>>> "launch" button next to it after uploading the "apa"
package via the
>>>>>>>>>> browser.
>>>>>>>>>>
>>>>>>>>>> I tried looking at the docs to see how to specify
a class as an
>>>>>>>>>> App that can be launched from a packaged application
but could not find
>>>>>>>>>> much. Could anyone of you please point me to the
right way of associating a
>>>>>>>>>> class as an application that will get enabled with
a launch button in the
>>>>>>>>>> web ui ? The class XYZ has the annotation called
@ApplicationAnnotation on
>>>>>>>>>> top of it.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ananth
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message