apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ananth Gundabattula <agundabatt...@gmail.com>
Subject Re: Best practises to specify the Application
Date Fri, 29 Apr 2016 05:42:48 GMT
Hello David,

I reattempted the packaging with the archetype based approach. I guess the
3.3.0 archetype did not exist when I attempted it earlier. Thanks for
pointing it out.

The default package generated by the archetype command seems to be working
fine when uploading it but the moment I add in my code and dependencies it
goes back the issue I am seeing :


My app package looks like app.zip  ( I removed all of the application.class
for brevity sake ) . Please note that I deleted all the jars under /lib
folder and the contents of the /app folder in the apa pacakge.

I have also attached a copy of my pom.xml file.

Could you please advise if you something amiss right away in the structure
below ? It may be noted that I retained the classes of the  original simple
project generated by the archetype and even they stopped showing with the
launch button the moment I added in all of the application dependencies in
maven.

Regards,
Ananth

On Fri, Apr 29, 2016 at 12:01 PM, David Yan <david@datatorrent.com> wrote:

> Hi Ananth,
>
> I just tried with 3.3.0-incubating with the following and it's working:
>
> $ mvn archetype:generate  -DarchetypeGroupId=org.apache.apex
>  -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating  -DgroupId=com.example
> -Dpackage=com.example.mydtapp -DartifactId=mydtapp  -Dversion=1.0-SNAPSHOT
>
> Let me know whether this works for you or not.
>
> The system uses the MANIFEST.MF file to get the basic meta information on
> the app package and it looks at all the jars in the app directory and
> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>
> David
>
>
>
> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Regarding as to why I did not use the archetype, I was building the app
>> for 3.3.0 version and I was not able to specify an archetype with the
>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>> version 3.3.0 ? I was not sure if the archetype version should match the
>> exact apex engine version . My presumption was archetype does not matter if
>> the packaging is done right. So i went ahead and created a simple maven
>> project.
>>
>> The maven assembly plugin used the appPackage.xml and the related
>> constructs to build the exact file layout of the app as mentioned in the
>> output above.
>>
>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>
>> This brings me to a question. How does the engine know which is the main
>> class that is to be used for launching the Streaming App. I do have a class
>> that extends the StreaminApplication that is annotated with
>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>> do not see a "launch" icon next to it. All I see is the top level packaged
>> name in the web console but nothing beneath it for me to trigger a launch.
>>
>> Regards,
>> Ananth
>>
>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <david@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> The MANIFEST.MF file is important, and it's generated automatically if
>>> the archetype is used. For example, the MANFEST.MF file from the Pi demo in
>>> Malhar has this content:
>>>
>>> Manifest-Version: 1.0
>>> Archiver-Version: Plexus Archiver
>>> Built-By: david
>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>> Class-Path: lib/*.jar
>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>> Created-By: Apache Maven 3.3.9
>>> Build-Jdk: 1.8.0_92
>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>  value of Pi. This is a starting point to understand how Apex works.
>>> DT-App-Package-Group-Id: org.apache.apex
>>> DT-App-Package-Name: pi-demo
>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>
>>> Is there a reason why the archetype is not used in your project?
>>>
>>> David
>>>
>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> Please find the unzip result at the end of this email. The jar is in
>>>> the app directory inside the apa.
>>>> As to why I did not use the archetype, I was building the app for 3.3.0
>>>> version and I was not able to specify an archetype with the version 3.3.0
>>>> when I attempted it. My presumption was archetype does not matter if the
>>>> packaging is done right. So i went ahead and created a simple maven
>>>> project.
>>>>
>>>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>>>> below ) . I also pasted the contents of the properties.xml file at the end
>>>> of this email for your reference.
>>>>
>>>> Archive:  cassandra-events-1.2-SNAPSHOT.apa
>>>>     testing: META-INF/                OK
>>>>     testing: META-INF/MANIFEST.MF     OK
>>>>     testing: app/                     OK
>>>>     testing: app/cassandra-events-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/                     OK
>>>>     testing: lib/activation-1.1.jar   OK
>>>>     testing: lib/activemq-client-5.8.0.jar   OK
>>>>     testing: lib/antlr-2.7.7.jar      OK
>>>>     testing: lib/aopalliance-1.0.jar   OK
>>>>     testing: lib/app-commons-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/aspectjrt-1.8.7.jar   OK
>>>>     testing: lib/btf-1.2.jar          OK
>>>>     testing: lib/cassandra-driver-core-2.1.8.jar   OK
>>>>     testing: lib/commons-beanutils-1.8.3.jar   OK
>>>>     testing: lib/commons-compiler-2.7.8.jar   OK
>>>>     testing: lib/commons-lang3-3.1.jar   OK
>>>>     testing: lib/commons-logging-1.1.3.jar   OK
>>>>     testing: lib/curator-client-2.7.0.jar   OK
>>>>     testing: lib/curator-framework-2.7.0.jar   OK
>>>>     testing: lib/dom4j-1.6.1.jar      OK
>>>>     testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/dwh-kafka-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar   OK
>>>>     testing: lib/evo-inflector-1.2.1.jar   OK
>>>>     testing: lib/fastutil-6.6.4.jar   OK
>>>>     testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar   OK
>>>>     testing: lib/geronimo-jms_1.1_spec-1.1.1.jar   OK
>>>>     testing: lib/guava-14.0.1.jar     OK
>>>>     testing: lib/hawtbuf-1.9.jar      OK
>>>>     testing: lib/hibernate-commons-annotations-5.0.0.Final.jar   OK
>>>>     testing: lib/hibernate-core-5.0.4.Final.jar   OK
>>>>     testing: lib/hibernate-entitymanager-5.0.4.Final.jar   OK
>>>>     testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar   OK
>>>>     testing: lib/HikariCP-2.4.3.jar   OK
>>>>     testing: lib/jackson-annotations-2.6.4.jar   OK
>>>>     testing: lib/jackson-core-2.6.4.jar   OK
>>>>     testing: lib/jackson-coreutils-1.6.jar   OK
>>>>     testing: lib/jackson-databind-2.6.4.jar   OK
>>>>     testing: lib/jandex-2.0.0.CR1.jar   OK
>>>>     testing: lib/javassist-3.18.1-GA.jar   OK
>>>>     testing: lib/javax.mail-1.5.0.jar   OK
>>>>     testing: lib/jboss-logging-3.3.0.Final.jar   OK
>>>>     testing: lib/jcl-over-slf4j-1.7.13.jar   OK
>>>>     testing: lib/jersey-client-1.9.jar   OK
>>>>     testing: lib/jersey-core-1.9.jar   OK
>>>>     testing: lib/jetty-continuation-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-http-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-io-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-security-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-server-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-servlet-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-util-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jetty-websocket-8.1.10.v20130312.jar   OK
>>>>     testing: lib/jline-2.11.jar       OK
>>>>     testing: lib/jms-api-1.1-rev-1.jar   OK
>>>>     testing: lib/joda-time-2.7.jar    OK
>>>>     testing: lib/json-patch-1.7.jar   OK
>>>>     testing: lib/jsr305-2.0.1.jar     OK
>>>>     testing: lib/jta-1.1.jar          OK
>>>>     testing: lib/kafka-clients-0.9.0.0.jar   OK
>>>>     testing: lib/log4j-1.2.17.jar     OK
>>>>     testing: lib/logback-classic-1.1.5.jar   OK
>>>>     testing: lib/logback-core-1.1.5.jar   OK
>>>>     testing: lib/lz4-1.2.0.jar        OK
>>>>     testing: lib/malhar-contrib-3.3.1-incubating.jar   OK
>>>>     testing: lib/malhar-kafka-3.3.1-incubating.jar   OK
>>>>     testing: lib/malhar-library-3.3.1-incubating.jar   OK
>>>>     testing: lib/metrics-core-3.0.2.jar   OK
>>>>     testing: lib/msg-simple-1.1.jar   OK
>>>>     testing: lib/mysql-connector-java-5.1.36.jar   OK
>>>>     testing: lib/named-regexp-0.2.3.jar   OK
>>>>     testing: lib/netty-3.6.2.Final.jar   OK
>>>>     testing: lib/netty-buffer-4.0.27.Final.jar   OK
>>>>     testing: lib/netty-codec-4.0.27.Final.jar   OK
>>>>     testing: lib/netty-common-4.0.27.Final.jar   OK
>>>>     testing: lib/netty-handler-4.0.27.Final.jar   OK
>>>>     testing: lib/netty-transport-4.0.27.Final.jar   OK
>>>>     testing: lib/opencsv-2.3.jar      OK
>>>>     testing: lib/slf4j-api-1.7.10.jar   OK
>>>>     testing: lib/snappy-java-1.1.1.7.jar   OK
>>>>     testing: lib/spring-aop-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-beans-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-context-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-core-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-data-commons-1.11.4.RELEASE.jar   OK
>>>>     testing: lib/spring-data-jpa-1.9.4.RELEASE.jar   OK
>>>>     testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar   OK
>>>>     testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar   OK
>>>>     testing: lib/spring-expression-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-hateoas-0.19.0.RELEASE.jar   OK
>>>>     testing: lib/spring-jdbc-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-orm-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-plugin-core-1.2.0.RELEASE.jar   OK
>>>>     testing: lib/spring-tx-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-web-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/spring-webmvc-4.1.9.RELEASE.jar   OK
>>>>     testing: lib/super-csv-2.3.1.jar   OK
>>>>     testing: lib/super-csv-joda-2.3.1.jar   OK
>>>>     testing: lib/xml-apis-1.0.b2.jar   OK
>>>>     testing: lib/zookeeper-3.4.6.jar   OK
>>>>     testing: conf/                    OK
>>>>     testing: conf/properties.xml      OK
>>>> No errors detected in compressed data of
>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>
>>>> Contents of the properties.xml file under the conf folder is here:
>>>>
>>>> <?xml version="1.0"?>
>>>> <configuration>
>>>>     <property>
>>>>         <name>dt.application.CED.class</name>
>>>>         <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>         <description>An alias for the application</description>
>>>>     </property>
>>>>     <property>
>>>>         <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>         <value>512</value>
>>>>     </property>
>>>>
>>>>     <property>
>>>>         <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>         <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>     </property>
>>>>
>>>>
>>>>     <property>
>>>>         <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>         <value>20</value>
>>>>     </property>
>>>>
>>>> </configuration>
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <david@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>> Make sure the jar containing the application classes should be in the
>>>>> app directory inside the apa.
>>>>>
>>>>> Can I know why the apex-app-archetype is not used in your apa project?
>>>>>
>>>>> David
>>>>>
>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello All,
>>>>>>
>>>>>> I am having the following issue with my application. I have a maven
>>>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>>>
>>>>>> My maven packaging is ensuring that the final "apa" extension package
>>>>>> has the right folder structure if exploded on a disk.
>>>>>>
>>>>>> However I am not able to locate the Application name and the "launch"
>>>>>> button next to it after uploading the "apa" package via the browser.
>>>>>>
>>>>>> I tried looking at the docs to see how to specify a class as an App
>>>>>> that can be launched from a packaged application but could not find
much.
>>>>>> Could anyone of you please point me to the right way of associating
a class
>>>>>> as an application that will get enabled with a launch button in the
web ui
>>>>>> ? The class XYZ has the annotation called @ApplicationAnnotation
on top of
>>>>>> it.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message