archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nicolas de loof" <nico...@apache.org>
Subject Re: wrong metadatas
Date Wed, 16 Jul 2008 08:25:00 GMT
I made some tests and this works nicely for me.

Thanks a lot !

2008/7/14 Maria Odea Ching <oching@apache.org>:

> Additional fixes in -r676516 and -r676526..
> Summary of the fixes:
>
>   1. generate checksums for the generated merged metadata
>   2. put the merged metadata & its checksums in the first repo where the
>   metadata was found. the filename for the merged metadata is
>   maven-metadata-${repoGroupId}.xml to handle instances when a repository
>   belongs to multiple groups.
>   3. merge the metadata if and only if all the following conditions are
>   met:
>      - the request was made to a repository group url
>      - the requested resource is a metadata file or a checksum file of a
>      metadata
>      - the requested metadata file is at the project level, not version
>      level
>   4. added test case when metadata checksums are requested after the
>   metadata file (simulate Maven requests)
>
> Thanks,
> Deng
>
> On Sun, Jul 13, 2008 at 10:08 PM, Maria Odea Ching <oching@apache.org>
> wrote:
>
> > Hi Nico,
> >
> > Could you verify the fix i committed in trunk -r676322 if it works for
> you?
> > I tested it out  using ranges and I was able to get the artifacts from
> the
> > different repos.
> >
> > Thanks,
> > Deng
> >
> >
> > On Fri, Jul 11, 2008 at 6:59 PM, Maria Odea Ching <oching@apache.org>
> > wrote:
> >
> >> On Fri, Jul 11, 2008 at 6:35 PM, nicolas de loof <nicolas@apache.org>
> >> wrote:
> >>
> >>> I've looked at MetaDataTools class.It merges the metadatas from
> multiple
> >>> repositories in a target file. This could be refactored into another
> >>> method
> >>> to return the ArchivaRepositoryMetadata objet, so that it can be used
> for
> >>> repo grouping.
> >>
> >>
> >> I'm using the RepositoryMetadataMerge.merge(..) to merge the metadata
> from
> >> each repository into
> >> a main metadata file, so I don't need to refactor anything. I have still
> >> yet to test it though. I'll commit it in svn asap
> >> so you could also take a look :)
> >>
> >> The "*write to file*" process should also be skipped when there is no
> >>> concrete metadata (availableVersions or Plugins) to avoid creating
> >>> "empty"
> >>> metadatas in managed repo for requested artifacts that aren't
> available.
> >>>
> >>> Could this be a security issue with archiva ? requesting a fake
> artifact
> >>> metadata creates a new File. Requesting thousand of them with a random
> >>> name
> >>> will create thousand files. This could be the basis of a "brute force"
> >>> deny
> >>> of service attack, with file system becoming full. Or maybe I'm a
> >>> paranoid ?
> >>
> >>
> >> This is indeed alarming! I'll try replicating this and send my results
> >> back..
> >>
> >>
> >>>
> >>> Nicolas.
> >>
> >>
> >> Thanks,
> >> Deng
> >>
> >>
> >>>
> >>> 2008/7/10 Maria Odea Ching <oching@apache.org>:
> >>>
> >>> > Okay, thanks Nico.. I think I can do the merging stuff. Will be back
> >>> online
> >>> > later when I get home..
> >>> >
> >>> > -Deng
> >>> >
> >>> > On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <nicolas@apache.org
> >
> >>> > wrote:
> >>> >
> >>> > > 2nd option may be simplier but not really nice, let's consider
this
> >>> use
> >>> > > case
> >>> > > :
> >>> > >
> >>> > > A user has an issue with a maven plugin. As a nice guy, he creates
> an
> >>> > > issue,
> >>> > > attach a patch, but to get quick fix he deploys a snapshot to
its
> >>> > corporate
> >>> > > repo.
> >>> > >
> >>> > > To get it using a group repository ("public" proxying public repos
> +
> >>> > > "corporate"), we need to merge the metadatas from both
> repositories,
> >>> so
> >>> > > that
> >>> > > the custom snapshot will get listed in metadatas + all available
> >>> version
> >>> > on
> >>> > > public snapshot repositories.
> >>> > >
> >>> > > From my understanding, this will require some "*list available
> >>> files*"
> >>> > > logic
> >>> > > + "*merge metadatas*" in ArchivaDavResource.createResource(),
and
> not
> >>> > just
> >>> > > "return resource;"
> >>> > >
> >>> > > I've commited the "*list available files*" part that was trivial.
> >>> > >
> >>> > > I'm not used with Dav (jackRabit) API and metadata file format,
so
> >>> any
> >>> > help
> >>> > > for the "*merge metadatas*"  would be welcome (TODO at line 263).
> >>> This
> >>> > > would
> >>> > > require to create an "in memory" virtual resource for the merged
> >>> > metadata.
> >>> > >
> >>> > > Nico.
> >>> > >
> >>> > >
> >>> > >
> >>> > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> >>> > >
> >>> > > > Oh ok, sorry I thought it was a blank file returned. I haven't
> >>> tried
> >>> > > > replicating it yet, was just looking through the codes.
> >>> > > > I guess either of the 2 options would work.. I'm still finishing
> up
> >>> > some
> >>> > > > things in the search, I'll see what I can do for 872 later
> tonight.
> >>> > > > Unless of course you want to work on it? :)
> >>> > > >
> >>> > > > Thanks,
> >>> > > > Deng
> >>> > > >
> >>> > > > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <
> >>> nicolas@apache.org>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > This mean the First metadata file found in on repo of
the group
> >>> is
> >>> > > > returned
> >>> > > > > !
> >>> > > > >
> >>> > > > > This is what happen : I get a metadat file with no <version>
> >>> included
> >>> > > > (not
> >>> > > > > an blank file, but an XML construct with just root element)
:
> >>> this is
> >>> > > the
> >>> > > > > content of my first repo in the group, that did not
retrieve
> the
> >>> > > expected
> >>> > > > > artifact.
> >>> > > > >
> >>> > > > > I made a simple test : place "release" repo before "corporate"
> in
> >>> the
> >>> > > > > group,
> >>> > > > > and I the get the expected metadata.
> >>> > > > >
> >>> > > > > My conclusion is we have two options :
> >>> > > > >
> >>> > > > > - support metadata merge from repositories in the group
(not
> just
> >>> > > "first
> >>> > > > > win")
> >>> > > > > - don't create metadata in managed repo if no artifact
was
> found
> >>> from
> >>> > > > > proxies.
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> >>> > > > >
> >>> > > > > > The ArchivaVirtualDavResource is for the dav browse
> >>> (directories),
> >>> > > not
> >>> > > > > for
> >>> > > > > > artifact requests..
> >>> > > > > >
> >>> > > > > > Hmm, the virtual repos work this way:
> >>> > > > > > - client requests for an artifact
> >>> > > > > > - RepoServlet handles the request and goes to
> >>> > > ArchivaDavSessionProvider
> >>> > > > > > - in ArchivaDavSessionProvider, the request is
checked if it
> is
> >>> a
> >>> > > repo
> >>> > > > > > group
> >>> > > > > > url or a regular repo url.
> >>> > > > > >   If it is, then loop through the repos under the
group. The
> >>> first
> >>> > > > > > requested artifact found among the repos
> >>> > > > > >   is returned (proxied or already existing). The
metadata
> >>> > > update/merge
> >>> > > > > > happens when the artifact is
> >>> > > > > >   fetched from the proxies (but this is on a per
repository
> >>> level,
> >>> > > > > there's
> >>> > > > > > no metadata of metadata files
> >>> > > > > >   from each repo being merged at the repo group
level)
> >>> > > > > >
> >>> > > > > > The same process goes if the request made is for
a metadata
> >>> file. I
> >>> > > > don't
> >>> > > > > > know why a blank metadata file was returned.. :(
> >>> > > > > >
> >>> > > > > > Thanks,
> >>> > > > > > Deng
> >>> > > > > >
> >>> > > > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof
<
> >>> > nicolas@apache.org
> >>> > > >
> >>> > > > > > wrote:
> >>> > > > > >
> >>> > > > > > > I just started investigating on this. I can't
find where
> the
> >>> > > > metadatas
> >>> > > > > > are
> >>> > > > > > > merged when a repository group is set. From
my
> understanding,
> >>> > > > > > > ArchivaVirtualDavResource is used with a List<File>
of
> >>> metadatas
> >>> > > from
> >>> > > > > > each
> >>> > > > > > > repo in the group. But I don't find WHERE
the metadata
> files
> >>> are
> >>> > > > merged
> >>> > > > > > > into
> >>> > > > > > > a single XML content...
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > 2008/7/10 Maria Odea Ching <oching@exist.com>:
> >>> > > > > > >
> >>> > > > > > > > Ok, thanks Nico, Dan :)
> >>> > > > > > > > I'll look into this further.. I guess
we could include
> this
> >>> in
> >>> > > > 1.1.1
> >>> > > > > > > which
> >>> > > > > > > > is scheduled to be released right after
1.1 to fix
> another
> >>> > > blocker
> >>> > > > in
> >>> > > > > > > 1.1.
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > > Deng
> >>> > > > > > > >
> >>> > > > > > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas
de loof <
> >>> > > > nicolas@apache.org
> >>> > > > > >
> >>> > > > > > > > wrote:
> >>> > > > > > > >
> >>> > > > > > > > > MRM-872 created for this, with the
concrete case I fall
> >>> into.
> >>> > > > > > > > >
> >>> > > > > > > > > If confirmed, this is a blocker
for repository group
> use
> >>> > > > > > > > >
> >>> > > > > > > > > 2008/7/10 Dan Tran <dantran@gmail.com>:
> >>> > > > > > > > >
> >>> > > > > > > > > > sounds like  a blocking bug?
would love to see this
> >>> fixed
> >>> > in
> >>> > > > 1.1
> >>> > > > > > :-)
> >>> > > > > > > > > >
> >>> > > > > > > > > > On Wed, Jul 9, 2008 at 7:11
PM, Maria Odea Ching <
> >>> > > > > > oching@apache.org>
> >>> > > > > > > > > > wrote:
> >>> > > > > > > > > > > Hi Nico,
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > That looks like a new
bug :( The metadata should be
> >>> just
> >>> > > the
> >>> > > > > same
> >>> > > > > > > as
> >>> > > > > > > > > the
> >>> > > > > > > > > > > regular metadata of a
managed repo (not belonging
> to
> >>> a
> >>> > > group)
> >>> > > > > as
> >>> > > > > > it
> >>> > > > > > > > > uses
> >>> > > > > > > > > > the
> >>> > > > > > > > > > > same code for webdav &
proxying. Could you file a
> >>> jira
> >>> > for
> >>> > > > > this?
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > Thanks,
> >>> > > > > > > > > > > Deng
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > On Wed, Jul 9, 2008 at
8:45 PM, nicolas de loof <
> >>> > > > > > > nicolas@apache.org>
> >>> > > > > > > > > > wrote:
> >>> > > > > > > > > > >
> >>> > > > > > > > > > >> Hy,
> >>> > > > > > > > > > >>
> >>> > > > > > > > > > >> I get strange behaviour
wen using repository group
> :
> >>> > > > > > > > > > >>
> >>> > > > > > > > > > >> From a fresh maven
installation (empty local
> repo),
> >>> with
> >>> > > > > > settings
> >>> > > > > > > > set
> >>> > > > > > > > > to
> >>> > > > > > > > > > >> mirror central to
my archiva repository group URL,
> >>> the
> >>> > > > eclipse
> >>> > > > > > 2.5
> >>> > > > > > > > > > plugin
> >>> > > > > > > > > > >> can't find the required
> >>> > > > > org.eclipse.core:resources:[3.1.0,4.0.0)
> >>> > > > > > > > > > >>
> >>> > > > > > > > > > >> The maven-metadata.xml
returned by the group URL
> is
> >>> > emty.
> >>> > > > > > > requesting
> >>> > > > > > > > > > >> metadatas from individual
managed repos in the
> group
> >>> are
> >>> > > > fine.
> >>> > > > > > Is
> >>> > > > > > > > this
> >>> > > > > > > > > a
> >>> > > > > > > > > > >> known limitation ?
> >>> > > > > > > > > > >>
> >>> > > > > > > > > > >> Nico.
> >>> > > > > > > > > > >>
> >>> > > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message