spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: Working Formula for Hive 0.13?
Date Mon, 28 Jul 2014 16:55:14 GMT
It would be great if the hive team can fix that issue. If not, we'll
have to continue forking our own version of Hive to change the way it
publishes artifacts.

- Patrick

On Mon, Jul 28, 2014 at 9:34 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> Talked with Owen offline. He confirmed that as of 0.13, hive-exec is still
> uber jar.
>
> Right now I am facing the following error building against Hive 0.13.1 :
>
> [ERROR] Failed to execute goal on project spark-hive_2.10: Could not
> resolve dependencies for project
> org.apache.spark:spark-hive_2.10:jar:1.1.0-SNAPSHOT: The following
> artifacts could not be resolved:
> org.spark-project.hive:hive-metastore:jar:0.13.1,
> org.spark-project.hive:hive-exec:jar:0.13.1,
> org.spark-project.hive:hive-serde:jar:0.13.1: Failure to find
> org.spark-project.hive:hive-metastore:jar:0.13.1 in
> http://repo.maven.apache.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of maven-repo
> has elapsed or updates are forced -> [Help 1]
>
> Some hint would be appreciated.
>
> Cheers
>
>
> On Mon, Jul 28, 2014 at 9:15 AM, Sean Owen <sowen@cloudera.com> wrote:
>
>> Yes, it is published. As of previous versions, at least, hive-exec
>> included all of its dependencies *in its artifact*, making it unusable
>> as-is because it contained copies of dependencies that clash with
>> versions present in other artifacts, and can't be managed with Maven
>> mechanisms.
>>
>> I am not sure why hive-exec was not published normally, with just its
>> own classes. That's why it was copied, into an artifact with just
>> hive-exec code.
>>
>> You could do the same thing for hive-exec 0.13.1.
>> Or maybe someone knows that it's published more 'normally' now.
>> I don't think hive-metastore is related to this question?
>>
>> I am no expert on the Hive artifacts, just remembering what the issue
>> was initially in case it helps you get to a similar solution.
>>
>> On Mon, Jul 28, 2014 at 4:47 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > hive-exec (as of 0.13.1) is published here:
>> >
>> http://search.maven.org/#artifactdetails%7Corg.apache.hive%7Chive-exec%7C0.13.1%7Cjar
>> >
>> > Should a JIRA be opened so that dependency on hive-metastore can be
>> > replaced by dependency on hive-exec ?
>> >
>> > Cheers
>> >
>> >
>> > On Mon, Jul 28, 2014 at 8:26 AM, Sean Owen <sowen@cloudera.com> wrote:
>> >
>> >> The reason for org.spark-project.hive is that Spark relies on
>> >> hive-exec, but the Hive project does not publish this artifact by
>> >> itself, only with all its dependencies as an uber jar. Maybe that's
>> >> been improved. If so, you need to point at the new hive-exec and
>> >> perhaps sort out its dependencies manually in your build.
>> >>
>> >> On Mon, Jul 28, 2014 at 4:01 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >> > I found 0.13.1 artifacts in maven:
>> >> >
>> >>
>> http://search.maven.org/#artifactdetails%7Corg.apache.hive%7Chive-metastore%7C0.13.1%7Cjar
>> >> >
>> >> > However, Spark uses groupId of org.spark-project.hive, not
>> >> org.apache.hive
>> >> >
>> >> > Can someone tell me how it is supposed to work ?
>> >> >
>> >> > Cheers
>> >> >
>> >> >
>> >> > On Mon, Jul 28, 2014 at 7:44 AM, Steve Nunez <snunez@hortonworks.com>
>> >> wrote:
>> >> >
>> >> >> I saw a note earlier, perhaps on the user list, that at least one
>> >> person is
>> >> >> using Hive 0.13. Anyone got a working build configuration for this
>> >> version
>> >> >> of Hive?
>> >> >>
>> >> >> Regards,
>> >> >> - Steve
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> CONFIDENTIALITY NOTICE
>> >> >> NOTICE: This message is intended for the use of the individual
or
>> >> entity to
>> >> >> which it is addressed and may contain information that is
>> confidential,
>> >> >> privileged and exempt from disclosure under applicable law. If
the
>> >> reader
>> >> >> of this message is not the intended recipient, you are hereby
>> notified
>> >> that
>> >> >> any printing, copying, dissemination, distribution, disclosure
or
>> >> >> forwarding of this communication is strictly prohibited. If you
have
>> >> >> received this communication in error, please contact the sender
>> >> immediately
>> >> >> and delete it from your system. Thank You.
>> >> >>
>> >>
>>

Mime
View raw message