ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francisco Peredo <franciscoper...@tabasco.gob.mx>
Subject Re: private repository MANIFEST.MF guessing the right repository structure
Date Tue, 24 Mar 2009 22:00:19 GMT

Hi!
Thanks a lot! Interesting approach.... sadly I do not have perl in my
computer right now... I'll
download it at my house and bring it with me tomorrow...

I really think this should be a part of Ivy, it would really help to make
the transition from "everything copied in to WEB-INF/lib" in to the
centralized .jar repository approach. Maybe I'll create a JIRA issue...

Thanks
Regards,
Francisco Jose Peredo



Kirby Files wrote:
> 
> Francisco,
> 
> Francisco Peredo wrote on 03/24/2009 02:56 PM:
>> We download stuff at our houses, and then copy them in usb portable
>> storage
>> and bring them to work.
>>
> [...]
>> So, we want to upload our WEB-INF\lib .jar files (for hibernate, seam,
>> spring, commons, etc) in to a shared repository to a server inside our
>> intranet...
>>
> [...]
>> I guess that to do that we need to use the task ivy:publish... Am I
>> right?
>> But since we have hundreds of files, we would like to be able to do it in
>> single shot, something like
>>
>> <ivy:publis-as-repository source-dir="WEB-INF\lib\">
> 
> Well, ivy:install makes more sense to me, as you don't have an ivy.xml 
> file for each module.
> 
> ivy:install is good for moving things from one repo to another. The 
> source repo could be the ibiblio default repo, the IvyRoundup packager 
> resolver, or a local filesystem.
> 
> You could take another approach, and just use your home bandwidth to 
> install all of your required jars (if you knew their org/module/rev) 
> to a local filesystem resolver on a USB stick, using ibiblio as the 
> source repo.
> 
>>
>> Is it possible to automate this with Ivy? or do I have to write my own
>> ivy:publis-as-repository ant task?
> 
> Well, it's probably not elegant, but when faced with the same problem, 
> I wrote the following perl script, parse_jars.pl, which will use ant 
> with a default target like this:
> 
>    <target name="installjar"
>            depends="init-ivy"
>            description="--> install jar to shared repo">
>    <ivy:install
> 	settingsRef="masergy.settings"
> 	organisation="${org}"
> 	module="${module}"
> 	revision="${rev}"
> 	type="jar"
> 	from="${localjar.resolver}"
> 	to="${to.resolver}" />
>    </target>
> 
> My ivysettings.xml includes:
> 
>      <filesystem name="localjars">
>        <artifact pattern="${install.path}/[artifact]-[revision].[ext]" />
>        <artifact pattern="${install.path}/[artifact].[ext]" />
>      </filesystem>
> 
> The script will attempt to divine the correct org/module using 4 
> techniques: (1) the ozacc Maven search API; (2) pattern-matching on 
> the module name; (3) the top-level of the jar packages; (4) if all 
> else fails. just ask the user.
> 
> Note the script requires all jars to follow the naming convention 
> module-ver.jar. Too few jars have relevant and consistent metadata 
> within their MANIFEST to do otherwise.
> 
> Thanks,
> ---
> Kirby Files
> Software Architect
> Masergy Communications
> kfiles@masergy.com
> 
> 8< ------------------------ parse_jars.pl ---------------------- >8
> #!/usr/bin/perl
> # Looks for jars in the incoming directory, attemps to parse the
> # needed ivy info from the filename, and if successful, publishes
> # the jar to the Masergy ivrep repository.
> 
> # If using linux with package mgmt, dependencies are:
> # perl-libwww-perl
> # perl-XML-Xpath
> # perl-Archive-Zip
> use LWP::UserAgent;
> use XML::XPath;
> use XML::XPath::XMLParser;
> use Archive::Zip;
> use Cwd;
> use strict qw(vars);
> 
> use vars qw($indir $ant $rep $pre $post);
> 
> $indir = shift(@ARGV);
> if ($indir !~ m#^/#) {
>    my $cwd = getcwd();
>    $indir =~ s/^/$cwd\//;
>    print "CWD: $indir\n";
> }
> $ant = "/usr/local/ant/bin/ant";
> $rep = "newjars";
> $pre="<ivy:install settingsRef=\"masergy.settings\" organisation=\"";
> $post=" from=\"\${from.resolver}\" to=\"\${to.resolver}\" />";
> 
> open (DIR, "ls -1 $indir|") || die "could not open $indir\n";
> while (<DIR>) {
>    my ($org, $module, $jar, $ver);
>    next unless /jar/;
>    chomp;
>    my $file = $indir . "/" . $_;
>    $org=$_;
>    $module=$_;
>    $jar=$_;
>    $ver=$_;
>    $module=~s/^(.*)\.jar/$1/;
>    $module=~s/([^.]*)-\d.*/$1/;
>    $ver=~s/^[^.]*-([\d.]+.*)\.jar/$1/;
>    $ver="unknown" unless $ver =~ /\d/;
>    ## Figure out the org
>    ## Ask the user, if there's ambiguity
>    $org = &maven_search_org( $module );
>    if (-1 == $org) {
>      $org = &guess_org($module);
>      print "Org guessed from module name:\n  $org\n";
>      $org = &classes_org($file);
>      print "Org guessed from jar package naming:\n  $org\n"
>        unless (-1 == $org);
>      do {
>        $org = &get_user_org();
>      } while ($org =~ /^$/);
>    }
>    if ($ver eq "unknown" || ($module !~ /^\w+$/) || $org !~ /\w+/) {
>      print "Skipping jar $jar:\n";
>      print "  $org / $module / $ver\n";
>      next;
>    } else {
>      &publish_jar($jar, $org, $module, $ver);
>    }
>    #&write_ivy_conf($jar, $org, $module, $ver);
> }
> 
> ## Use ivy::install to publish the jar via sftp to ivyrep
> sub publish_jar
> {
>    my ($jar, $org, $module, $ver) = @_;
>    print "Uploading $org / $module / $jar ... ";
>    my $resp = `$ant -Dorg=$org -Dmodule=$module -Drev=$ver 
> -Dinstall.path=$indir 2>&1`;
>    if ($resp =~ /failed/i) {
>      print STDERR "Error uploading $jar:\n", $resp;
>      print "\n";
>    } else {
>      print "Done.\n";
>    }
> }
> 
> ## Find a directory in the jar to propose as the org
> ## searches for any dirs with two path elements
> sub classes_org
> {
>    my ($file) = @_;
>    my $somezip = Archive::Zip->new();
>    unless ( $somezip->read( $file ) == AZ_OK ) {
>         die "Error reading jarfile $file";
>    }
>    my @members = $somezip->members();
>    my @dirnames = ();
>    foreach (@members) {
>      if ($_->isDirectory()) {
>        my $fname = $_->fileName();
>        if ($fname =~ /^\w+\/\w+\/$/) {
>          $fname =~ s/^(\w+)\/(\w+)\/$/$1.$2/;
>          return $fname;
>        }
>      }
>    }
>    return -1;
> }
> 
> ## Ask the user for the best org name
> sub get_user_org
> {
>    print "Enter preffered org:> ";
>    my $line = <STDIN>;
>    chomp($line);
>    return $line;
> }
> 
> ## Make a guess about the org name from the module name
> sub guess_org
> {
>    my ($module) = @_;
>    my $org = $module;
>      $org=~s/spring.*/springframework/;
>      $org=~s/hiber.*/hibernate/;
>      $org=~s/standard.*/taglibs/;
>      $org=~s/jstl.*/taglibs/;
>      $org=~s/persistence.*/sun/;
>      $org=~s/jta.*/sun/;
>      $org=~s/servlet-api.*/tomcat/;
>      $org=~s/ojdbc.*/oracle/;
>      $org=~s/cglib.*/cglib/;
>      $org=~s/javassist.*/jboss/;
>      $org=~s/([^.]*)-\d.*jar/$1/;
>      $org=~s/(.*)\.jar/$1/;
>    return $org;
> }
> 
> ## Search the maven repo for a module matching this jar
> ## If found, return the module's organization
> ## if none, or more than one, are found, ask the user
> sub maven_search_org
> {
>    my ($module) = @_;
> 
>    my $ua = LWP::UserAgent->new;
>    $ua->agent('Maven Repo Search');
> 
>    my $service= 
> 'http://maven.ozacc.com/search?format=xml&type=jar&keyword=';
>    my $url=URI->new($service . $module);
>    my $req=HTTP::Request->new;
>    $req->method('GET');
>    $req->uri($url);
>    my $resp = $ua->request($req)->content;
> 
>    my $org = &parse_resp( $module, $resp );
>    return $org;
> }
> 
> ## Parse the XML response from the Maven search service
> sub parse_resp
> {
>    my ($module, $data) = @_;
> 
>    my $xp = XML::XPath->new(xml => $data);
> 
>    my $nodeset = $xp->find('//groupId'); # find all paragraphs
> 
>    # Get rid of duplicates
>    my %uniq=();
>    foreach my $node ($nodeset->get_nodelist) {
>       $uniq{ $xp->getNodeText( $node ) } = 1;
>    }
>    my @keys = (sort keys %uniq);
> 
>    if ($#keys == 0) {
>      return ($keys[0]);
>    } elsif ($#keys == -1) {
>      print "Found no module matching $module in Maven repo\n";
>      return -1;
>    } else {
>      print "Found multiple possible orgs for $module in Maven repo:\n";
>      foreach (@keys) {
>        print "  $_\n";
>      }
>      return -1;
>    }
> }
> 
> ## (obsolete) Write ant task and ivy.xml conf lines to create repo
> sub write_ivy_conf
> {
>    my ($jar, $org, $module, $ver) = @_;
>    my $line = $pre . $org . "\" module=\"" . $module . "\" 
> revision=\"" . $ver . "\"". $post . "\n";
>    system("mkdir -p $rep/$org/$module/jars");
>    link "$indir/$jar", "$rep/$org/$module/jars/$jar";
>    print $line;
>    print "<dependency org=\"" . $org . "\" name=\"" . $module . "\" 
> rev=\"" . $ver . "\"/>\n";
> }
> 
> 

-- 
View this message in context: http://www.nabble.com/private-repository-MANIFEST.MF-guessing-the-right-repository-structure-tp22687302p22690777.html
Sent from the ivy-user mailing list archive at Nabble.com.


Mime
View raw message