archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nicolas de loof" <nico...@apache.org>
Subject Re: wrong metadatas
Date Wed, 16 Jul 2008 08:48:51 GMT
Ready to get a release for me.

2008/7/16 Emmanuel Venisse <emmanuel.venisse@gmail.com>:

> If it's ok, it is possible to restart the release or do you want to add
> more
> things in 1.1?
>
> Emmanuel
>
> On Wed, Jul 16, 2008 at 10:25 AM, nicolas de loof <nicolas@apache.org>
> wrote:
>
> > I made some tests and this works nicely for me.
> >
> > Thanks a lot !
> >
> > 2008/7/14 Maria Odea Ching <oching@apache.org>:
> >
> > > Additional fixes in -r676516 and -r676526..
> > > Summary of the fixes:
> > >
> > >   1. generate checksums for the generated merged metadata
> > >   2. put the merged metadata & its checksums in the first repo where
> the
> > >   metadata was found. the filename for the merged metadata is
> > >   maven-metadata-${repoGroupId}.xml to handle instances when a
> repository
> > >   belongs to multiple groups.
> > >   3. merge the metadata if and only if all the following conditions are
> > >   met:
> > >      - the request was made to a repository group url
> > >      - the requested resource is a metadata file or a checksum file of
> a
> > >      metadata
> > >      - the requested metadata file is at the project level, not version
> > >      level
> > >   4. added test case when metadata checksums are requested after the
> > >   metadata file (simulate Maven requests)
> > >
> > > Thanks,
> > > Deng
> > >
> > > On Sun, Jul 13, 2008 at 10:08 PM, Maria Odea Ching <oching@apache.org>
> > > wrote:
> > >
> > > > Hi Nico,
> > > >
> > > > Could you verify the fix i committed in trunk -r676322 if it works
> for
> > > you?
> > > > I tested it out  using ranges and I was able to get the artifacts
> from
> > > the
> > > > different repos.
> > > >
> > > > Thanks,
> > > > Deng
> > > >
> > > >
> > > > On Fri, Jul 11, 2008 at 6:59 PM, Maria Odea Ching <oching@apache.org
> >
> > > > wrote:
> > > >
> > > >> On Fri, Jul 11, 2008 at 6:35 PM, nicolas de loof <
> nicolas@apache.org>
> > > >> wrote:
> > > >>
> > > >>> I've looked at MetaDataTools class.It merges the metadatas from
> > > multiple
> > > >>> repositories in a target file. This could be refactored into
> another
> > > >>> method
> > > >>> to return the ArchivaRepositoryMetadata objet, so that it can
be
> used
> > > for
> > > >>> repo grouping.
> > > >>
> > > >>
> > > >> I'm using the RepositoryMetadataMerge.merge(..) to merge the
> metadata
> > > from
> > > >> each repository into
> > > >> a main metadata file, so I don't need to refactor anything. I have
> > still
> > > >> yet to test it though. I'll commit it in svn asap
> > > >> so you could also take a look :)
> > > >>
> > > >> The "*write to file*" process should also be skipped when there is
> no
> > > >>> concrete metadata (availableVersions or Plugins) to avoid creating
> > > >>> "empty"
> > > >>> metadatas in managed repo for requested artifacts that aren't
> > > available.
> > > >>>
> > > >>> Could this be a security issue with archiva ? requesting a fake
> > > artifact
> > > >>> metadata creates a new File. Requesting thousand of them with
a
> > random
> > > >>> name
> > > >>> will create thousand files. This could be the basis of a "brute
> > force"
> > > >>> deny
> > > >>> of service attack, with file system becoming full. Or maybe I'm
a
> > > >>> paranoid ?
> > > >>
> > > >>
> > > >> This is indeed alarming! I'll try replicating this and send my
> results
> > > >> back..
> > > >>
> > > >>
> > > >>>
> > > >>> Nicolas.
> > > >>
> > > >>
> > > >> Thanks,
> > > >> Deng
> > > >>
> > > >>
> > > >>>
> > > >>> 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > > >>>
> > > >>> > Okay, thanks Nico.. I think I can do the merging stuff. Will
be
> > back
> > > >>> online
> > > >>> > later when I get home..
> > > >>> >
> > > >>> > -Deng
> > > >>> >
> > > >>> > On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <
> > nicolas@apache.org
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > 2nd option may be simplier but not really nice, let's
consider
> > this
> > > >>> use
> > > >>> > > case
> > > >>> > > :
> > > >>> > >
> > > >>> > > A user has an issue with a maven plugin. As a nice guy,
he
> > creates
> > > an
> > > >>> > > issue,
> > > >>> > > attach a patch, but to get quick fix he deploys a snapshot
to
> its
> > > >>> > corporate
> > > >>> > > repo.
> > > >>> > >
> > > >>> > > To get it using a group repository ("public" proxying
public
> > repos
> > > +
> > > >>> > > "corporate"), we need to merge the metadatas from both
> > > repositories,
> > > >>> so
> > > >>> > > that
> > > >>> > > the custom snapshot will get listed in metadatas + all
> available
> > > >>> version
> > > >>> > on
> > > >>> > > public snapshot repositories.
> > > >>> > >
> > > >>> > > From my understanding, this will require some "*list
available
> > > >>> files*"
> > > >>> > > logic
> > > >>> > > + "*merge metadatas*" in ArchivaDavResource.createResource(),
> and
> > > not
> > > >>> > just
> > > >>> > > "return resource;"
> > > >>> > >
> > > >>> > > I've commited the "*list available files*" part that
was
> trivial.
> > > >>> > >
> > > >>> > > I'm not used with Dav (jackRabit) API and metadata file
format,
> > so
> > > >>> any
> > > >>> > help
> > > >>> > > for the "*merge metadatas*"  would be welcome (TODO
at line
> 263).
> > > >>> This
> > > >>> > > would
> > > >>> > > require to create an "in memory" virtual resource for
the
> merged
> > > >>> > metadata.
> > > >>> > >
> > > >>> > > Nico.
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > > >>> > >
> > > >>> > > > Oh ok, sorry I thought it was a blank file returned.
I
> haven't
> > > >>> tried
> > > >>> > > > replicating it yet, was just looking through the
codes.
> > > >>> > > > I guess either of the 2 options would work.. I'm
still
> > finishing
> > > up
> > > >>> > some
> > > >>> > > > things in the search, I'll see what I can do for
872 later
> > > tonight.
> > > >>> > > > Unless of course you want to work on it? :)
> > > >>> > > >
> > > >>> > > > Thanks,
> > > >>> > > > Deng
> > > >>> > > >
> > > >>> > > > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof
<
> > > >>> nicolas@apache.org>
> > > >>> > > > wrote:
> > > >>> > > >
> > > >>> > > > > This mean the First metadata file found in
on repo of the
> > group
> > > >>> is
> > > >>> > > > returned
> > > >>> > > > > !
> > > >>> > > > >
> > > >>> > > > > This is what happen : I get a metadat file
with no
> <version>
> > > >>> included
> > > >>> > > > (not
> > > >>> > > > > an blank file, but an XML construct with just
root element)
> :
> > > >>> this is
> > > >>> > > the
> > > >>> > > > > content of my first repo in the group, that
did not
> retrieve
> > > the
> > > >>> > > expected
> > > >>> > > > > artifact.
> > > >>> > > > >
> > > >>> > > > > I made a simple test : place "release" repo
before
> > "corporate"
> > > in
> > > >>> the
> > > >>> > > > > group,
> > > >>> > > > > and I the get the expected metadata.
> > > >>> > > > >
> > > >>> > > > > My conclusion is we have two options :
> > > >>> > > > >
> > > >>> > > > > - support metadata merge from repositories
in the group
> (not
> > > just
> > > >>> > > "first
> > > >>> > > > > win")
> > > >>> > > > > - don't create metadata in managed repo if
no artifact was
> > > found
> > > >>> from
> > > >>> > > > > proxies.
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > > >>> > > > >
> > > >>> > > > > > The ArchivaVirtualDavResource is for
the dav browse
> > > >>> (directories),
> > > >>> > > not
> > > >>> > > > > for
> > > >>> > > > > > artifact requests..
> > > >>> > > > > >
> > > >>> > > > > > Hmm, the virtual repos work this way:
> > > >>> > > > > > - client requests for an artifact
> > > >>> > > > > > - RepoServlet handles the request and
goes to
> > > >>> > > ArchivaDavSessionProvider
> > > >>> > > > > > - in ArchivaDavSessionProvider, the request
is checked if
> > it
> > > is
> > > >>> a
> > > >>> > > repo
> > > >>> > > > > > group
> > > >>> > > > > > url or a regular repo url.
> > > >>> > > > > >   If it is, then loop through the repos
under the group.
> > The
> > > >>> first
> > > >>> > > > > > requested artifact found among the repos
> > > >>> > > > > >   is returned (proxied or already existing).
The metadata
> > > >>> > > update/merge
> > > >>> > > > > > happens when the artifact is
> > > >>> > > > > >   fetched from the proxies (but this
is on a per
> repository
> > > >>> level,
> > > >>> > > > > there's
> > > >>> > > > > > no metadata of metadata files
> > > >>> > > > > >   from each repo being merged at the
repo group level)
> > > >>> > > > > >
> > > >>> > > > > > The same process goes if the request
made is for a
> metadata
> > > >>> file. I
> > > >>> > > > don't
> > > >>> > > > > > know why a blank metadata file was returned..
:(
> > > >>> > > > > >
> > > >>> > > > > > Thanks,
> > > >>> > > > > > Deng
> > > >>> > > > > >
> > > >>> > > > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas
de loof <
> > > >>> > nicolas@apache.org
> > > >>> > > >
> > > >>> > > > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > I just started investigating on
this. I can't find
> where
> > > the
> > > >>> > > > metadatas
> > > >>> > > > > > are
> > > >>> > > > > > > merged when a repository group is
set. From my
> > > understanding,
> > > >>> > > > > > > ArchivaVirtualDavResource is used
with a List<File> of
> > > >>> metadatas
> > > >>> > > from
> > > >>> > > > > > each
> > > >>> > > > > > > repo in the group. But I don't find
WHERE the metadata
> > > files
> > > >>> are
> > > >>> > > > merged
> > > >>> > > > > > > into
> > > >>> > > > > > > a single XML content...
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > > 2008/7/10 Maria Odea Ching <oching@exist.com>:
> > > >>> > > > > > >
> > > >>> > > > > > > > Ok, thanks Nico, Dan :)
> > > >>> > > > > > > > I'll look into this further..
I guess we could
> include
> > > this
> > > >>> in
> > > >>> > > > 1.1.1
> > > >>> > > > > > > which
> > > >>> > > > > > > > is scheduled to be released
right after 1.1 to fix
> > > another
> > > >>> > > blocker
> > > >>> > > > in
> > > >>> > > > > > > 1.1.
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > > Deng
> > > >>> > > > > > > >
> > > >>> > > > > > > > On Thu, Jul 10, 2008 at 2:37
PM, nicolas de loof <
> > > >>> > > > nicolas@apache.org
> > > >>> > > > > >
> > > >>> > > > > > > > wrote:
> > > >>> > > > > > > >
> > > >>> > > > > > > > > MRM-872 created for this,
with the concrete case I
> > fall
> > > >>> into.
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > If confirmed, this is
a blocker for repository
> group
> > > use
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > 2008/7/10 Dan Tran <dantran@gmail.com>:
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > > sounds like  a blocking
bug? would love to see
> this
> > > >>> fixed
> > > >>> > in
> > > >>> > > > 1.1
> > > >>> > > > > > :-)
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > On Wed, Jul 9, 2008
at 7:11 PM, Maria Odea Ching
> <
> > > >>> > > > > > oching@apache.org>
> > > >>> > > > > > > > > > wrote:
> > > >>> > > > > > > > > > > Hi Nico,
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > That looks like
a new bug :( The metadata
> should
> > be
> > > >>> just
> > > >>> > > the
> > > >>> > > > > same
> > > >>> > > > > > > as
> > > >>> > > > > > > > > the
> > > >>> > > > > > > > > > > regular metadata
of a managed repo (not
> belonging
> > > to
> > > >>> a
> > > >>> > > group)
> > > >>> > > > > as
> > > >>> > > > > > it
> > > >>> > > > > > > > > uses
> > > >>> > > > > > > > > > the
> > > >>> > > > > > > > > > > same code for
webdav & proxying. Could you file
> a
> > > >>> jira
> > > >>> > for
> > > >>> > > > > this?
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Thanks,
> > > >>> > > > > > > > > > > Deng
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > On Wed, Jul
9, 2008 at 8:45 PM, nicolas de loof
> <
> > > >>> > > > > > > nicolas@apache.org>
> > > >>> > > > > > > > > > wrote:
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > >> Hy,
> > > >>> > > > > > > > > > >>
> > > >>> > > > > > > > > > >> I get strange
behaviour wen using repository
> > group
> > > :
> > > >>> > > > > > > > > > >>
> > > >>> > > > > > > > > > >> From a fresh
maven installation (empty local
> > > repo),
> > > >>> with
> > > >>> > > > > > settings
> > > >>> > > > > > > > set
> > > >>> > > > > > > > > to
> > > >>> > > > > > > > > > >> mirror central
to my archiva repository group
> > URL,
> > > >>> the
> > > >>> > > > eclipse
> > > >>> > > > > > 2.5
> > > >>> > > > > > > > > > plugin
> > > >>> > > > > > > > > > >> can't find
the required
> > > >>> > > > > org.eclipse.core:resources:[3.1.0,4.0.0)
> > > >>> > > > > > > > > > >>
> > > >>> > > > > > > > > > >> The maven-metadata.xml
returned by the group
> URL
> > > is
> > > >>> > emty.
> > > >>> > > > > > > requesting
> > > >>> > > > > > > > > > >> metadatas
from individual managed repos in the
> > > group
> > > >>> are
> > > >>> > > > fine.
> > > >>> > > > > > Is
> > > >>> > > > > > > > this
> > > >>> > > > > > > > > a
> > > >>> > > > > > > > > > >> known limitation
?
> > > >>> > > > > > > > > > >>
> > > >>> > > > > > > > > > >> Nico.
> > > >>> > > > > > > > > > >>
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message