apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shubham Pathak <shub...@datatorrent.com>
Subject Re: Malhar library does not build on adding additional dependency
Date Fri, 08 Jan 2016 12:43:18 GMT
Hi,

I observed that adding json-schema-validator dependency in malhar-lib is
leading to couple of dependency conflicts.
Just to summarize,
1. If i go with version 2.0.1 of json-schema-validator, malhar does not
build. There are some compilation errors and to fix those i had to add
dependency of jsr. Malhar builds successfully with no compilation errors
but  DTFileTest.seekDTFile test fails.
2. I tried to go with latest version of json-schema-validator. With this i
don't need to add dependency of jsr, malhar-iibrary builds successfully but
JSON parser operator fails at runtime because of guava library conflicts.

 Today i  moved JSON parser in malhar-contrib and i did not face any issues
during compilation / runtime.
 So i would be moving JSON parser to malhar-contrib.

Thanks,
Shubham

On Thu, Jan 7, 2016 at 11:12 AM, Shubham Pathak <shubham@datatorrent.com>
wrote:

> Hi,
>
> JSON parser operator that is already existing in malhar-lib needs to be
> enhanced. That enhancement needs dependency of json-schema-validator . So
> we would need to add that dependency into malhar-lib.
>
>
> Thanks,
> Shubham
>
> On Thu, Jan 7, 2016 at 10:55 AM, Thomas Weise <thomas@datatorrent.com>
> wrote:
>
>> Please address the dependency issue.
>>
>> On Wed, Jan 6, 2016 at 8:56 PM, Shubham Pathak <shubham@datatorrent.com>
>> wrote:
>>
>> > Hi Chandni,
>> >
>> > This is  enhancement to existing JSON Parser ( currently in malhar lib)
>> to
>> > take in schema for validations ( similar to  XML / CSV parsers ).
>> >
>> >
>> > Thanks,
>> > Shubham
>> >
>> > On Thu, Jan 7, 2016 at 12:35 AM, Chandni Singh <chandni@datatorrent.com
>> >
>> > wrote:
>> >
>> > > Hey Shubham,
>> > >
>> > > Can the enhancement (extension) which adds a new dependency go into
>> > > contrib?
>> > >
>> > > Thanks,
>> > > Chandni
>> > >
>> > > On Wed, Jan 6, 2016 at 12:37 AM, Shubham Pathak <
>> shubham@datatorrent.com
>> > >
>> > > wrote:
>> > >
>> > > > Chinmay,
>> > > >
>> > > > I tried to build malhar-library with the latest version of
>> > > > json-schema-validator and it builds successfully without me having
>> to
>> > add
>> > > > dependency for jsr.
>> > > >
>> > > > However, when i try to run the test, it throws a runtime error
>> > > > [image: Inline image 1]
>> > > >
>> > > >
>> > > >
>> > > > Issue is that guava 11.0 is being considered instead of 16.01.
>> > > >
>> > > > One solution to this can be add <exclusion> for guava in apex
common
>> > but
>> > > > that seems risky as it might lead to run time exceptions.
>> > > >
>> > > > For now I am  going ahead with the approach i mentioned in my
>> previous
>> > > > mail.
>> > > >
>> > > >
>> > > > Thomas,
>> > > > No new operator is being added. Existing operator is being enhanced
>> and
>> > > > that requires a dependency to be added.
>> > > >
>> > > >
>> > > > Thanks,
>> > > > Shubham
>> > > >
>> > > > On Tue, Jan 5, 2016 at 11:25 PM, Thomas Weise <
>> thomas@datatorrent.com>
>> > > > wrote:
>> > > >
>> > > >> There was a discussion a while ago on why we don't want to add
>> > > >> dependencies
>> > > >> to malhar-library unless they are broadly applicable to many
>> > operators.
>> > > >> Please review that discussion.
>> > > >>
>> > > >> As for adding dependencies in general, please analyze the situation
>> > > >> carefully before moving ahead.. You will want to look at licenses,
>> > > >> possible
>> > > >> conflicts with engine/Hadoop, transitive dependencies, functional
>> > > overlap
>> > > >> with existing dependencies etc. Also state why the dependency
is
>> > needed.
>> > > >>
>> > > >> Thomas
>> > > >>
>> > > >>
>> > > >> On Tue, Jan 5, 2016 at 8:27 AM, Chinmay Kolhatkar <
>> > > >> chinmay@datatorrent.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Hey Shubham,
>> > > >> >
>> > > >> > FileSplitterInput was already present and devel-3 compiles
fine
>> > > without
>> > > >> > jsr305 dependency (for Nullable).
>> > > >> > It seems that adding json related dependency is affecting
>> somewhere
>> > > >> else.
>> > > >> >
>> > > >> > Can you please check if all the unit tests are passing after
>> adding
>> > > >> > findbug's jsr305 dependency?
>> > > >> >
>> > > >> > Also, There seems to be quite some versions of
>> json-schema-validator
>> > > >> after
>> > > >> > 2.0.1, would it make sense to use latest version as dependency?
>> > > >> >
>> > > >> > Thanks,
>> > > >> > Chinmay.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > ~ Chinmay.
>> > > >> >
>> > > >> > On Tue, Jan 5, 2016 at 5:29 PM, Shubham Pathak <
>> > > shubham@datatorrent.com
>> > > >> >
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Fixed by adding dependency of jsr . Now the pom looks
like this
>> > > >> > >
>> > > >> > > <dependency>
>> > > >> > >       <groupId>com.github.fge</groupId>
>> > > >> > >       <artifactId>json-schema-validator</artifactId>
>> > > >> > >       <version>2.0.1</version>
>> > > >> > >     </dependency>
>> > > >> > >     <dependency>
>> > > >> > > <groupId>com.google.code.findbugs</groupId>
>> > > >> > > <artifactId>jsr305</artifactId>
>> > > >> > > <version>1.3.9</version>
>> > > >> > > <optional>true</optional>
>> > > >> > > </dependency>
>> > > >> > >
>> > > >> > > Thanks,
>> > > >> > > Shubham
>> > > >> > >
>> > > >> > >
>> > > >> > > On Tue, Jan 5, 2016 at 2:35 PM, Shubham Pathak <
>> > > >> shubham@datatorrent.com>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > Hi,
>> > > >> > > >
>> > > >> > > > I am working on an operator that needs following
dependency
>> to
>> > be
>> > > >> added
>> > > >> > > in
>> > > >> > > > Malhar lib
>> > > >> > > > <dependency>
>> > > >> > > >       <groupId>com.github.fge</groupId>
>> > > >> > > >       <artifactId>json-schema-validator</artifactId>
>> > > >> > > >       <version>2.0.1</version>
>> > > >> > > >     </dependency>
>> > > >> > > >
>> > > >> > > > But when i add this, and try to build malhar library,
i get
>> > > >> following
>> > > >> > > error
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> /code/projects/Apache/incubator-apex-malhar/library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java:[36,24]
>> > > >> > > > cannot find symbol
>> > > >> > > > [ERROR] symbol:   class Nullable
>> > > >> > > > [ERROR] location: package javax.annotation
>> > > >> > > >
>> > > >> > > > On further investigation i noticed, when i add
the above
>> > > dependency,
>> > > >> > > > jsr305 jar doesn't show up in dependency hierarchy.
>> > > >> > > >
>> > > >> > > > If i remove the above dependency, jsr305 shows
up.
>> > > >> > > >
>> > > >> > > > Can anyone please help ?
>> > > >> > > >
>> > > >> > > > Thanks,
>> > > >> > > > Shubham
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message