archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Venisse" <emmanuel.veni...@gmail.com>
Subject Re: wrong metadatas
Date Wed, 16 Jul 2008 08:47:18 GMT
If it's ok, it is possible to restart the release or do you want to add more
things in 1.1?

Emmanuel

On Wed, Jul 16, 2008 at 10:25 AM, nicolas de loof <nicolas@apache.org>
wrote:

> I made some tests and this works nicely for me.
>
> Thanks a lot !
>
> 2008/7/14 Maria Odea Ching <oching@apache.org>:
>
> > Additional fixes in -r676516 and -r676526..
> > Summary of the fixes:
> >
> >   1. generate checksums for the generated merged metadata
> >   2. put the merged metadata & its checksums in the first repo where the
> >   metadata was found. the filename for the merged metadata is
> >   maven-metadata-${repoGroupId}.xml to handle instances when a repository
> >   belongs to multiple groups.
> >   3. merge the metadata if and only if all the following conditions are
> >   met:
> >      - the request was made to a repository group url
> >      - the requested resource is a metadata file or a checksum file of a
> >      metadata
> >      - the requested metadata file is at the project level, not version
> >      level
> >   4. added test case when metadata checksums are requested after the
> >   metadata file (simulate Maven requests)
> >
> > Thanks,
> > Deng
> >
> > On Sun, Jul 13, 2008 at 10:08 PM, Maria Odea Ching <oching@apache.org>
> > wrote:
> >
> > > Hi Nico,
> > >
> > > Could you verify the fix i committed in trunk -r676322 if it works for
> > you?
> > > I tested it out  using ranges and I was able to get the artifacts from
> > the
> > > different repos.
> > >
> > > Thanks,
> > > Deng
> > >
> > >
> > > On Fri, Jul 11, 2008 at 6:59 PM, Maria Odea Ching <oching@apache.org>
> > > wrote:
> > >
> > >> On Fri, Jul 11, 2008 at 6:35 PM, nicolas de loof <nicolas@apache.org>
> > >> wrote:
> > >>
> > >>> I've looked at MetaDataTools class.It merges the metadatas from
> > multiple
> > >>> repositories in a target file. This could be refactored into another
> > >>> method
> > >>> to return the ArchivaRepositoryMetadata objet, so that it can be used
> > for
> > >>> repo grouping.
> > >>
> > >>
> > >> I'm using the RepositoryMetadataMerge.merge(..) to merge the metadata
> > from
> > >> each repository into
> > >> a main metadata file, so I don't need to refactor anything. I have
> still
> > >> yet to test it though. I'll commit it in svn asap
> > >> so you could also take a look :)
> > >>
> > >> The "*write to file*" process should also be skipped when there is no
> > >>> concrete metadata (availableVersions or Plugins) to avoid creating
> > >>> "empty"
> > >>> metadatas in managed repo for requested artifacts that aren't
> > available.
> > >>>
> > >>> Could this be a security issue with archiva ? requesting a fake
> > artifact
> > >>> metadata creates a new File. Requesting thousand of them with a
> random
> > >>> name
> > >>> will create thousand files. This could be the basis of a "brute
> force"
> > >>> deny
> > >>> of service attack, with file system becoming full. Or maybe I'm a
> > >>> paranoid ?
> > >>
> > >>
> > >> This is indeed alarming! I'll try replicating this and send my results
> > >> back..
> > >>
> > >>
> > >>>
> > >>> Nicolas.
> > >>
> > >>
> > >> Thanks,
> > >> Deng
> > >>
> > >>
> > >>>
> > >>> 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > >>>
> > >>> > Okay, thanks Nico.. I think I can do the merging stuff. Will be
> back
> > >>> online
> > >>> > later when I get home..
> > >>> >
> > >>> > -Deng
> > >>> >
> > >>> > On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <
> nicolas@apache.org
> > >
> > >>> > wrote:
> > >>> >
> > >>> > > 2nd option may be simplier but not really nice, let's consider
> this
> > >>> use
> > >>> > > case
> > >>> > > :
> > >>> > >
> > >>> > > A user has an issue with a maven plugin. As a nice guy, he
> creates
> > an
> > >>> > > issue,
> > >>> > > attach a patch, but to get quick fix he deploys a snapshot
to its
> > >>> > corporate
> > >>> > > repo.
> > >>> > >
> > >>> > > To get it using a group repository ("public" proxying public
> repos
> > +
> > >>> > > "corporate"), we need to merge the metadatas from both
> > repositories,
> > >>> so
> > >>> > > that
> > >>> > > the custom snapshot will get listed in metadatas + all available
> > >>> version
> > >>> > on
> > >>> > > public snapshot repositories.
> > >>> > >
> > >>> > > From my understanding, this will require some "*list available
> > >>> files*"
> > >>> > > logic
> > >>> > > + "*merge metadatas*" in ArchivaDavResource.createResource(),
and
> > not
> > >>> > just
> > >>> > > "return resource;"
> > >>> > >
> > >>> > > I've commited the "*list available files*" part that was
trivial.
> > >>> > >
> > >>> > > I'm not used with Dav (jackRabit) API and metadata file format,
> so
> > >>> any
> > >>> > help
> > >>> > > for the "*merge metadatas*"  would be welcome (TODO at line
263).
> > >>> This
> > >>> > > would
> > >>> > > require to create an "in memory" virtual resource for the
merged
> > >>> > metadata.
> > >>> > >
> > >>> > > Nico.
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > >>> > >
> > >>> > > > Oh ok, sorry I thought it was a blank file returned.
I haven't
> > >>> tried
> > >>> > > > replicating it yet, was just looking through the codes.
> > >>> > > > I guess either of the 2 options would work.. I'm still
> finishing
> > up
> > >>> > some
> > >>> > > > things in the search, I'll see what I can do for 872
later
> > tonight.
> > >>> > > > Unless of course you want to work on it? :)
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > > Deng
> > >>> > > >
> > >>> > > > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <
> > >>> nicolas@apache.org>
> > >>> > > > wrote:
> > >>> > > >
> > >>> > > > > This mean the First metadata file found in on repo
of the
> group
> > >>> is
> > >>> > > > returned
> > >>> > > > > !
> > >>> > > > >
> > >>> > > > > This is what happen : I get a metadat file with
no <version>
> > >>> included
> > >>> > > > (not
> > >>> > > > > an blank file, but an XML construct with just root
element) :
> > >>> this is
> > >>> > > the
> > >>> > > > > content of my first repo in the group, that did
not retrieve
> > the
> > >>> > > expected
> > >>> > > > > artifact.
> > >>> > > > >
> > >>> > > > > I made a simple test : place "release" repo before
> "corporate"
> > in
> > >>> the
> > >>> > > > > group,
> > >>> > > > > and I the get the expected metadata.
> > >>> > > > >
> > >>> > > > > My conclusion is we have two options :
> > >>> > > > >
> > >>> > > > > - support metadata merge from repositories in the
group (not
> > just
> > >>> > > "first
> > >>> > > > > win")
> > >>> > > > > - don't create metadata in managed repo if no artifact
was
> > found
> > >>> from
> > >>> > > > > proxies.
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > 2008/7/10 Maria Odea Ching <oching@apache.org>:
> > >>> > > > >
> > >>> > > > > > The ArchivaVirtualDavResource is for the dav
browse
> > >>> (directories),
> > >>> > > not
> > >>> > > > > for
> > >>> > > > > > artifact requests..
> > >>> > > > > >
> > >>> > > > > > Hmm, the virtual repos work this way:
> > >>> > > > > > - client requests for an artifact
> > >>> > > > > > - RepoServlet handles the request and goes
to
> > >>> > > ArchivaDavSessionProvider
> > >>> > > > > > - in ArchivaDavSessionProvider, the request
is checked if
> it
> > is
> > >>> a
> > >>> > > repo
> > >>> > > > > > group
> > >>> > > > > > url or a regular repo url.
> > >>> > > > > >   If it is, then loop through the repos under
the group.
> The
> > >>> first
> > >>> > > > > > requested artifact found among the repos
> > >>> > > > > >   is returned (proxied or already existing).
The metadata
> > >>> > > update/merge
> > >>> > > > > > happens when the artifact is
> > >>> > > > > >   fetched from the proxies (but this is on
a per repository
> > >>> level,
> > >>> > > > > there's
> > >>> > > > > > no metadata of metadata files
> > >>> > > > > >   from each repo being merged at the repo
group level)
> > >>> > > > > >
> > >>> > > > > > The same process goes if the request made
is for a metadata
> > >>> file. I
> > >>> > > > don't
> > >>> > > > > > know why a blank metadata file was returned..
:(
> > >>> > > > > >
> > >>> > > > > > Thanks,
> > >>> > > > > > Deng
> > >>> > > > > >
> > >>> > > > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de
loof <
> > >>> > nicolas@apache.org
> > >>> > > >
> > >>> > > > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I just started investigating on this.
I can't find where
> > the
> > >>> > > > metadatas
> > >>> > > > > > are
> > >>> > > > > > > merged when a repository group is set.
From my
> > understanding,
> > >>> > > > > > > ArchivaVirtualDavResource is used with
a List<File> of
> > >>> metadatas
> > >>> > > from
> > >>> > > > > > each
> > >>> > > > > > > repo in the group. But I don't find WHERE
the metadata
> > files
> > >>> are
> > >>> > > > merged
> > >>> > > > > > > into
> > >>> > > > > > > a single XML content...
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > > 2008/7/10 Maria Odea Ching <oching@exist.com>:
> > >>> > > > > > >
> > >>> > > > > > > > Ok, thanks Nico, Dan :)
> > >>> > > > > > > > I'll look into this further.. I
guess we could include
> > this
> > >>> in
> > >>> > > > 1.1.1
> > >>> > > > > > > which
> > >>> > > > > > > > is scheduled to be released right
after 1.1 to fix
> > another
> > >>> > > blocker
> > >>> > > > in
> > >>> > > > > > > 1.1.
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > > Deng
> > >>> > > > > > > >
> > >>> > > > > > > > On Thu, Jul 10, 2008 at 2:37 PM,
nicolas de loof <
> > >>> > > > nicolas@apache.org
> > >>> > > > > >
> > >>> > > > > > > > wrote:
> > >>> > > > > > > >
> > >>> > > > > > > > > MRM-872 created for this, with
the concrete case I
> fall
> > >>> into.
> > >>> > > > > > > > >
> > >>> > > > > > > > > If confirmed, this is a blocker
for repository group
> > use
> > >>> > > > > > > > >
> > >>> > > > > > > > > 2008/7/10 Dan Tran <dantran@gmail.com>:
> > >>> > > > > > > > >
> > >>> > > > > > > > > > sounds like  a blocking
bug? would love to see this
> > >>> fixed
> > >>> > in
> > >>> > > > 1.1
> > >>> > > > > > :-)
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > On Wed, Jul 9, 2008 at
7:11 PM, Maria Odea Ching <
> > >>> > > > > > oching@apache.org>
> > >>> > > > > > > > > > wrote:
> > >>> > > > > > > > > > > Hi Nico,
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > That looks like a
new bug :( The metadata should
> be
> > >>> just
> > >>> > > the
> > >>> > > > > same
> > >>> > > > > > > as
> > >>> > > > > > > > > the
> > >>> > > > > > > > > > > regular metadata
of a managed repo (not belonging
> > to
> > >>> a
> > >>> > > group)
> > >>> > > > > as
> > >>> > > > > > it
> > >>> > > > > > > > > uses
> > >>> > > > > > > > > > the
> > >>> > > > > > > > > > > same code for webdav
& proxying. Could you file a
> > >>> jira
> > >>> > for
> > >>> > > > > this?
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > Thanks,
> > >>> > > > > > > > > > > Deng
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > On Wed, Jul 9, 2008
at 8:45 PM, nicolas de loof <
> > >>> > > > > > > nicolas@apache.org>
> > >>> > > > > > > > > > wrote:
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > >> Hy,
> > >>> > > > > > > > > > >>
> > >>> > > > > > > > > > >> I get strange
behaviour wen using repository
> group
> > :
> > >>> > > > > > > > > > >>
> > >>> > > > > > > > > > >> From a fresh
maven installation (empty local
> > repo),
> > >>> with
> > >>> > > > > > settings
> > >>> > > > > > > > set
> > >>> > > > > > > > > to
> > >>> > > > > > > > > > >> mirror central
to my archiva repository group
> URL,
> > >>> the
> > >>> > > > eclipse
> > >>> > > > > > 2.5
> > >>> > > > > > > > > > plugin
> > >>> > > > > > > > > > >> can't find the
required
> > >>> > > > > org.eclipse.core:resources:[3.1.0,4.0.0)
> > >>> > > > > > > > > > >>
> > >>> > > > > > > > > > >> The maven-metadata.xml
returned by the group URL
> > is
> > >>> > emty.
> > >>> > > > > > > requesting
> > >>> > > > > > > > > > >> metadatas from
individual managed repos in the
> > group
> > >>> are
> > >>> > > > fine.
> > >>> > > > > > Is
> > >>> > > > > > > > this
> > >>> > > > > > > > > a
> > >>> > > > > > > > > > >> known limitation
?
> > >>> > > > > > > > > > >>
> > >>> > > > > > > > > > >> Nico.
> > >>> > > > > > > > > > >>
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message