hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <>
Subject Re: Hive in IntelliJ
Date Thu, 31 May 2012 17:14:08 GMT
I agree, even if we had a decent ant script most IDEs could deal with
it, but maven would be better.

I understand why Hive's ant build infrastructure is the way it is.
generating thrift, building a hive-exec.jar fat jar, getting hadoop in
the old days before it was in maven, templating engine to generate .q
files and diff them against .out files.

But I still think we should gut it. Most of it's complexity is self
imposed because common is its own subant, and cli is its own subant,
serde is its own subant. Then each subant usually has to hack up the
classpath somehow to work, or hack up something else to test.

And then support other hacks like this. Which get replicated to every subant.

    <fileset dir="${hadoop.root}">
      <!-- below is for 0.23 onwards -->
      <include name="share/hadoop/common/lib/*.jar" />
      <exclude name="share/hadoop/common/lib/hadoop-mapreduce-*.jar" />
      <exclude name="share/hadoop/common/lib/hadoop-yarn-*.jar" />

Not all of it is hives fault, because one build of hive works with
like 40 version of hadoop, and a majority of the hackery is dealing
with rather large upstream changes.

On the other hand maven is terrible at dealing with any of the outside
of the box stuff hive needs to do. Each ugly ant hack hive has to do
to deal with hadoop 20-20s-> would like have to be
written as a maven plugin which would not really leave us in a better
state. Ugly-one-off ants vs ugly one off maven plugins.

The way I look at it 33% of the problem is upstream changes and 33% is
hive desire to fork hadoop to run hive, and 33% is the every component
is a subant thing.

On 5/31/12, Owen O'Malley <> wrote:
> On Thu, May 31, 2012 at 9:35 AM, Edward Capriolo
> <>wrote:
>> Hive is in maven
> I meant that Hive is built with Ant. Intellij has better support for
> importing projects built by Maven. Hive's jars are published in Maven
> central, but that is a different thing.
> -- Owen

View raw message