hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/IvyPrimer" by KayKay
Date Sun, 10 Jan 2010 03:14:14 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hbase/IvyPrimer" page has been changed by KayKay.


New page:
HBase (trunk, 0.21.0) uses Ivy (http://ant.apache.org/ivy/) as the dependency manager to resolve
dependencies ( There has been an interest to eventually migrate to maven. Interested people
can contribute to the same ). 

 * Ivy Internals

Ivy is a plugin that fits nicely with Ant and hence needs no separate software , other than
Ant itself. (See Ant page for more details)

The build looks for the presence of the ivy library in ${basedir}/ivy/ivy-2.1.0.jar , and
downloads the same if it is not present. 
 * Ivy Cache 

Ivy looks up the ivy.xml in the current directory of ${basedir} , as well as the base directory
of contrib modules  ( stargate / transactional ). 

By default- it applies transitive dependency for the modules, to make sure we pull the entire
dependency tree along with a given module as well. 

The default settings in trunk, enables dependencies to be  pulled from the list of pre-defined
repositories (See ${basedir}/ivy/ivysettings.xml ). 
The order of the dependency resolvers specifically come in ivy/ivysettings.xml as follows:

    <chain name="default" dual="true">
      <resolver ref="maven2"/>

      <resolver ref="java-net" />

      <resolver ref="google-code" />

      <resolver ref="codehaus" />

      <resolver ref="apache-snapshot" />

These resolvers should be sufficient at this point . Should there be need to add more - continue
to maintain apache-snapshot as the last resolver in the order , because that being a development

After contains the URL-s it copies the libraries(currently, only jars) to  $HOME/.ivy2/cache
, the *first time* if it does not exist . For subsequent operations of invocation of ivy ,
it copies the jar files from the ivy cache to the appropriate directory where we want to be,
without incurring network cost then. 

 * FAQ:
 ** Where did the jars that were present in lib/*.jar go ? 
   They should now be present across - 



 ** I just updated my trunk. Could not see them. 

Do a 'ant compile' in the command line. As one of the dependent targets ( ivy-retrieve-common
and ivy-retrieve-test ) - the jars should be pulled to the directory structure mentioned above.

 ** Something is weird. I don't like this automagical dependency pull. How do I start all
over ? 

$ ant clean 
$ rm -fR ~/.ivy2/cache 
$ ant compile 

This would clear the ivy cache and start all over again. This is needed only in the worst
case and not for every build. 

Usually this happens when the publisher republishes the binary in the repository making the
client cache out of sync, with the server.  This does not happen. 

 ** I do not need to connect to the internet from the build server. What are the options now.

* Set up a directory structure to a repository listed ( Say , http://repo2.maven.org/maven2/
* For all the projects (And the versions) that you need , create the directory structure exactly
the same as those in the repository. 
* Modify ivy/ivysettings.xml to add a file/directory based resolver at the beginning. 
* In the chain of resolvers, add the new resolver right on top before everything else ( as
the order matters). 

So - now we are good with the build able to complete from our internally controlled repository
as opposed to being exposed to the outside world. 

View raw message