archiva-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff <predato...@gmail.com>
Subject Re: Index help...
Date Mon, 04 Feb 2013 15:47:30 GMT
I don't *yet* have a need, but our user base is growing and so is our CI/CD
pipeline (number of GO and Jenkins build agents/slaves) and I wanted to
plan ahead.  I am trying to find the document that talked about keeping
indexes local while allowing a shared repo, but I have been unsuccessful
thus far.  Maybe I dreamed it up.  :-)

Thanks for your help.


On Mon, Feb 4, 2013 at 2:56 AM, Olivier Lamy <olamy@apache.org> wrote:

> complicated to have scabality when serving files :-(
> we don't (yet) any sync possible (that's something we start discussing
> on dev@ with maybe using new Apache project called helix).
> With your setup each repository will be independent.
> Do you have so many users to need that ?
>
> 2013/2/2 Jeff <predatorvi@gmail.com>:
> > I will do that.
> >
> > However, how do I configure the repository to support the previously
> > mentioned horizontal scaling with a shared repository?
> >
> >
> > On Sat, Feb 2, 2013 at 10:26 AM, Olivier Lamy <olamy@apache.org> wrote:
> >
> >> our default is .indexer.
> >> Oh I believe using an absolute path for the index location doesn't
> >> work and it's not available via http(s).
> >> Can you simply change the index path to .index or leave it blank
> >> (default is .indexer)  ?
> >> As it that will be available as this one
> >>
> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer
> >>
> >>
> >>
> >> 2013/2/2 Jeff <predatorvi@gmail.com>:
> >> > Also, isn't the default index directory supposed to be .index?  That's
> >> what
> >> > it seems to be on Maven Central.
> >> >
> >> > http://repo.maven.apache.org/maven2/.index/
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > On Fri, Feb 1, 2013 at 3:28 PM, Jeff <predatorvi@gmail.com> wrote:
> >> >
> >> >> If I attempt to go to a similar URL for any of our repositories
> there is
> >> >> no .indexer or .index path such as:
> >> >>
> >> >> http://archiva/repository/dna-releases/.indexer
> >> >> http://archiva/repository/dna-snapshots/.index
> >> >> http://archiva/repository/dnabackend-releases/.indexer
> >> >> http://archiva/repository/dnabackend-snapshots/.index
> >> >>
> >> >> Is this because I moved them?  If so, I am happy to put them back to
> the
> >> >> normal location, but then what would change to allow multiple archiva
> >> >> tomcat servers to use the same 'data' repositories?
> >> >>
> >> >>
> >> >>
> >> >> On Fri, Feb 1, 2013 at 2:10 PM, Olivier Lamy <olamy@apache.org>
> wrote:
> >> >>
> >> >>> 2013/2/1 Jeff <predatorvi@gmail.com>:
> >> >>> > I'm not sure what you mean by "why you don't access to indexes
via
> >> >>> http"...
> >> >>> Something like
> >> >>>
> >>
> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer/
> >> >>>
> >> >>> it's available for all repositories (and for group too since 1.4-M2
> in
> >> >>> path lie archiva/repository/${grouprepositoryId}/.indexer/
> >> >>>
> >> >>> Don't know exactly how it works in IDE. But they use remote indexes
> >> >>> available via http(s) and not file system directories.
> >> >>> ?
> >> >>>
> >> >>> >
> >> >>> > When I setup Archiva running in Tomcat 6 under CentOS 6.2
Linux, I
> >> have
> >> >>> the
> >> >>> > folder structure below:
> >> >>> >
> >> >>> > /archiva
> >> >>> >    /conf
> >> >>> >    /data
> >> >>> >    /derbydb
> >> >>> >    /indexes
> >> >>> >    /logs
> >> >>> >
> >> >>> > I think the only thing that is not standard is the 'indexes'
> >> directory.
> >> >>>  I
> >> >>> > was trying to setup Archiva and anticipate possibly needing
to
> scale
> >> >>> > horizontally by adding multiple front end servers behind our
load
> >> >>> balancer.
> >> >>> >   I read somewhere (I thought) that the indexes are specific
to
> each
> >> >>> front
> >> >>> > end server, but that the repository data could be shared,
so I
> >> mounted
> >> >>> the
> >> >>> > /archiva/data folder as a remote NFS mount.  The rest of the
> folders
> >> >>> above
> >> >>> > are located physically on the archiva server.
> >> >>> >
> >> >>> > For each repository, the index folder is set to something
like:
> >> >>> >
> >> >>> > /archiva/indexes/team1-releases
> >> >>> > /archiva/indexes/team1-snapshots
> >> >>> >
> >> >>> > But my understanding is that these indexes are the internal
> Archiva
> >> >>> > indexes, not the publicly consumable indexes.  Nowhere in
the
> >> >>> > /archiva/data/repositories/team1-releases or other repositories
> are
> >> >>> there
> >> >>> > any .index folders.
> >> >>> >
> >> >>> > I also have a single repository group that proxies all internal
> and
> >> >>> > externally proxied repositories under a single virtual repository
> and
> >> >>> there
> >> >>> > are no indexes consolidated under that repository group either
> which
> >> is
> >> >>> > what I thought was added in 1.4-M2 (the ability to consolidate
> >> indexes).
> >> >>> >
> >> >>> > Inside of /archiva/indexes, each repository index does have
a
> bunch
> >> of
> >> >>> > files that look like:
> >> >>> >
> >> >>> > _7.fdt
> >> >>> > _7.fdx
> >> >>> > ...
> >> >>> > nexux-maven-reposotory-index.6948.gz
> >> >>> > nexux-maven-reposotory-index.6949.gz
> >> >>> > nexux-maven-reposotory-index.6950.gz
> >> >>> > nexux-maven-reposotory-index.6951.gz
> >> >>> > ...
> >> >>> >
> >> >>> > So something is generating these (I assume correctly).
> >> >>> >
> >> >>> > Also in each repository, the setting named "Skip Packed Index
> >> creation"
> >> >>> is
> >> >>> > NOT checked.
> >> >>>
> >> >>> Those nexux-maven-reposotory-index.xxx.gz files is incremental
> >> >>> generation. see a file called
> >> >>> nexus-maven-repository-index-packer.properties which contains some
> >> >>> metadatas on incremental generation.
> >> >>> >
> >> >>> > Hopefully that clarifies things and we can work this out and
get
> it
> >> >>> > addressed as needed for 1.4-M4 :-).
> >> >>> >
> >> >>> >
> >> >>> > On Fri, Feb 1, 2013 at 9:51 AM, Olivier Lamy <olamy@apache.org>
> >> wrote:
> >> >>> >
> >> >>> >> maybe there is an issue and setup indexes to a dedicated
location
> >> >>> doesn't
> >> >>> >> work.
> >> >>> >> Is there any indexes files in archiva installation ?
> >> >>> >>
> >> >>> >> BTW why you don't access to indexes via http ?
> >> >>> >>
> >> >>> >> 2013/2/1 Jeff <predatorvi@gmail.com>:
> >> >>> >> > Yep, multiple times.  The server has been up and
running for a
> >> >>> number of
> >> >>> >> > months.
> >> >>> >> > On Feb 1, 2013 2:33 AM, "Olivier Lamy" <olamy@apache.org>
> wrote:
> >> >>> >> >
> >> >>> >> >> Did you try a full scan ?
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> 2013/1/31 Jeff <predatorvi@gmail.com>:
> >> >>> >> >> > I'm currently using 1.4-M3 and have configured
my Archiva
> >> indexes
> >> >>> to
> >> >>> >> >> point
> >> >>> >> >> > to a dedicated index folder located on the
local host and
> put
> >> the
> >> >>> data
> >> >>> >> >> > folder on a shared NFS mount.
> >> >>> >> >> >
> >> >>> >> >> > However, the .index folder for local repos
is not getting
> >> created
> >> >>> and
> >> >>> >> >> thus
> >> >>> >> >> > my IDE's can only see versions of what I
have in my local
> .m2
> >> >>> >> repository.
> >> >>> >> >> >
> >> >>> >> >> > I don't see anything in the logs suggesting
that it is
> failing.
> >> >>> >> >> >
> >> >>> >> >> > Thoughts?
> >> >>> >> >> >
> >> >>> >> >> > --
> >> >>> >> >> > Jeff Vincent
> >> >>> >> >> > predatorvi@gmail.com
> >> >>> >> >> > See my LinkedIn profile at:
> >> >>> >> >> > http://www.linkedin.com/in/rjeffreyvincent
> >> >>> >> >> > I ♥ DropBox <http://db.tt/9O6LfBX>
!!
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> --
> >> >>> >> >> Olivier Lamy
> >> >>> >> >> Talend: http://coders.talend.com
> >> >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>> >> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Olivier Lamy
> >> >>> >> Talend: http://coders.talend.com
> >> >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>> >>
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Jeff Vincent
> >> >>> > predatorvi@gmail.com
> >> >>> > See my LinkedIn profile at:
> >> >>> > http://www.linkedin.com/in/rjeffreyvincent
> >> >>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Olivier Lamy
> >> >>> Talend: http://coders.talend.com
> >> >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Jeff Vincent
> >> >> predatorvi@gmail.com
> >> >> See my LinkedIn profile at:
> >> >> http://www.linkedin.com/in/rjeffreyvincent
> >> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Jeff Vincent
> >> > predatorvi@gmail.com
> >> > See my LinkedIn profile at:
> >> > http://www.linkedin.com/in/rjeffreyvincent
> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> Talend: http://coders.talend.com
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
> >
> >
> >
> > --
> > Jeff Vincent
> > predatorvi@gmail.com
> > See my LinkedIn profile at:
> > http://www.linkedin.com/in/rjeffreyvincent
> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>
>
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>



-- 
Jeff Vincent
predatorvi@gmail.com
See my LinkedIn profile at:
http://www.linkedin.com/in/rjeffreyvincent
I ♥ DropBox <http://db.tt/9O6LfBX> !!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message