archiva-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Lamy <ol...@apache.org>
Subject Re: Index help...
Date Wed, 06 Feb 2013 17:40:12 GMT
2013/2/6 Olivier Lamy <olamy@apache.org>:
> The issue I have is that will break backward compat.
> Create an issue in jira to see others reactions.
> Note: http://archiva.apache.org/docs/1.4-M4-SNAPSHOT/userguide/virtual-repositories.html
> section "Merged index".
> I know people using that (including my current company).
> BTW I agree that will be more following a "de facto" standard.
NOTE we can that make configurable :-)
>
> 2013/2/6 Jeff <predatorvi@gmail.com>:
>> Also, would there be value in Archiva changing the default to .index
>> instead of .indexer since .index seems to be the standard used for index
>> downloads by clients?
>>
>>
>> On Wed, Feb 6, 2013 at 8:54 AM, Jeff <predatorvi@gmail.com> wrote:
>>
>>> Thanks Olivier!  I will set an absolute directory.
>>>
>>> On a side note as per previous discussion, I have an NFS mount for the
>>> archiva data folder that is symlinked.
>>>
>>> NFS: /mnt/archiva-remote
>>>
>>> ARCHIVA:  /archiva/data --> /mnt/archiva-remote
>>>
>>> When I set my repository folder via the Archiva UI to something like:
>>>
>>> /archiva/data/repositories/cms-releases
>>>
>>> After I save and then refresh the UI, the value for the repository gets
>>> changed to:
>>>
>>> /mnt/archiva-remote/repositories/cms-releases
>>>
>>> While this still works, it makes it difficult to move or relocate the
>>> share. Recently we had to create a bigger share (we now have 4TB) on
>>> another system to which we mounted at a different mount point, to which we
>>> copied the data files from the old share and then switched the symlink, but
>>> it broke the repositories because the directories were pointing to the old
>>> NFS mount.  Something to look at anyway.
>>>
>>> When is 1.4-M4 scheduled?
>>>
>>>
>>> On Wed, Feb 6, 2013 at 4:15 AM, Olivier Lamy <olamy@apache.org> wrote:
>>>
>>>> in trunk.
>>>> if blank .indexer directory is created in the repository directory.
>>>> I agree "Index Directory" is a bit confusing option.
>>>> As this doesn't work really if you configure something outside of the
>>>> top of your repository directory.
>>>> Furthermore I have noticed changing the value it's not dynamic (need a
>>>> restart).
>>>> So I will try to fix that.
>>>> Currently what you can do is adding a value with a directory inside
>>>> your repository directory.
>>>> If repository use /home/archiva/repo/mygreat-repo
>>>> use /home/archiva/repo/mygreat-repo/.indexer
>>>> I will try to fix that in trunk (or at least improve documentation ! )
>>>> 2013/2/6 Olivier Lamy <olamy@apache.org>:
>>>> > Hi,
>>>> > Apologize for delay.
>>>> >
>>>> > 2013/2/4 Jeff <predatorvi@gmail.com>:
>>>> >> I'm now trying to switch up the index directories on my repositories
>>>> and am
>>>> >> banging my head.  I've simply removed the "Index Directory" setting on
>>>> my
>>>> >> repos and I see the .indexer folder getting created.  Which is good.
>>>> >>
>>>> >> However, according to
>>>> >> http://archiva.apache.org/docs/1.4-M3/adminguide/repositories.html, it
>>>> >> states:
>>>> >>
>>>> >> *Another directory named .index is also created at the root of the
>>>> >> repository directory. This is not configurable though as it contains
>>>> the
>>>> >> packaged/bundled index which is consumed by different consumers of the
>>>> >> index such as M2Eclipse. *
>>>> > doh documentation is wrong (we use .indexer)
>>>> >>
>>>> >>
>>>> >> However, it is NOT getting created.  I've restarted tomcat, I've
>>>> forced a
>>>> >> full index scan on every repository as well as a Directory scan.
>>>> >>
>>>> >> The above statement also lead me to believe that regardless of the
>>>> location
>>>> >> of ".indexer" that ".index" would still get created at the root but it
>>>> >> isn't.
>>>> >>
>>>> >> On one repository I tried setting "Index Directory" to ".index", but I
>>>> get
>>>> >> an error that it can't create "/usr/share/tomcat6/.index".
>>>> > A bit weird.
>>>> > What is -Dappserver.base value ?
>>>> > BTW I agree there are some possible with relative path (I fixed some in
>>>> trunk).
>>>> > But while studying your case, I think this need more
>>>> attention/debugging.
>>>> > I'm on it. I will keep you informed as soon I have fixed (or at least
>>>> > understood the issue :-)) that in source code.
>>>> >>
>>>> >> Thanks for any additional direction...
>>>> >>
>>>> >>
>>>> >> On Mon, Feb 4, 2013 at 8:47 AM, Jeff <predatorvi@gmail.com> wrote:
>>>> >>
>>>> >>> I don't *yet* have a need, but our user base is growing and so is our
>>>> >>> CI/CD pipeline (number of GO and Jenkins build agents/slaves) and I
>>>> wanted
>>>> >>> to plan ahead.  I am trying to find the document that talked about
>>>> keeping
>>>> >>> indexes local while allowing a shared repo, but I have been
>>>> unsuccessful
>>>> >>> thus far.  Maybe I dreamed it up.  :-)
>>>> >>>
>>>> >>> Thanks for your help.
>>>> >>>
>>>> >>>
>>>> >>> On Mon, Feb 4, 2013 at 2:56 AM, Olivier Lamy <olamy@apache.org>
>>>> wrote:
>>>> >>>
>>>> >>>> complicated to have scabality when serving files :-(
>>>> >>>> we don't (yet) any sync possible (that's something we start
>>>> discussing
>>>> >>>> on dev@ with maybe using new Apache project called helix).
>>>> >>>> With your setup each repository will be independent.
>>>> >>>> Do you have so many users to need that ?
>>>> >>>>
>>>> >>>> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>>> >>>> > I will do that.
>>>> >>>> >
>>>> >>>> > However, how do I configure the repository to support the
>>>> previously
>>>> >>>> > mentioned horizontal scaling with a shared repository?
>>>> >>>> >
>>>> >>>> >
>>>> >>>> > On Sat, Feb 2, 2013 at 10:26 AM, Olivier Lamy <olamy@apache.org>
>>>> wrote:
>>>> >>>> >
>>>> >>>> >> our default is .indexer.
>>>> >>>> >> Oh I believe using an absolute path for the index location doesn't
>>>> >>>> >> work and it's not available via http(s).
>>>> >>>> >> Can you simply change the index path to .index or leave it blank
>>>> >>>> >> (default is .indexer)  ?
>>>> >>>> >> As it that will be available as this one
>>>> >>>> >>
>>>> >>>>
>>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> > Also, isn't the default index directory supposed to be .index?
>>>> >>>>  That's
>>>> >>>> >> what
>>>> >>>> >> > it seems to be on Maven Central.
>>>> >>>> >> >
>>>> >>>> >> > http://repo.maven.apache.org/maven2/.index/
>>>> >>>> >> >
>>>> >>>> >> > Thanks.
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> > On Fri, Feb 1, 2013 at 3:28 PM, Jeff <predatorvi@gmail.com>
>>>> wrote:
>>>> >>>> >> >
>>>> >>>> >> >> If I attempt to go to a similar URL for any of our repositories
>>>> >>>> there is
>>>> >>>> >> >> no .indexer or .index path such as:
>>>> >>>> >> >>
>>>> >>>> >> >> http://archiva/repository/dna-releases/.indexer
>>>> >>>> >> >> http://archiva/repository/dna-snapshots/.index
>>>> >>>> >> >> http://archiva/repository/dnabackend-releases/.indexer
>>>> >>>> >> >> http://archiva/repository/dnabackend-snapshots/.index
>>>> >>>> >> >>
>>>> >>>> >> >> Is this because I moved them?  If so, I am happy to put them
>>>> back
>>>> >>>> to the
>>>> >>>> >> >> normal location, but then what would change to allow multiple
>>>> >>>> archiva
>>>> >>>> >> >> tomcat servers to use the same 'data' repositories?
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >> On Fri, Feb 1, 2013 at 2:10 PM, Olivier Lamy <olamy@apache.org
>>>> >
>>>> >>>> wrote:
>>>> >>>> >> >>
>>>> >>>> >> >>> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> > I'm not sure what you mean by "why you don't access to
>>>> indexes
>>>> >>>> via
>>>> >>>> >> >>> http"...
>>>> >>>> >> >>> Something like
>>>> >>>> >> >>>
>>>> >>>> >>
>>>> >>>>
>>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer/
>>>> >>>> >> >>>
>>>> >>>> >> >>> it's available for all repositories (and for group too since
>>>> >>>> 1.4-M2 in
>>>> >>>> >> >>> path lie archiva/repository/${grouprepositoryId}/.indexer/
>>>> >>>> >> >>>
>>>> >>>> >> >>> Don't know exactly how it works in IDE. But they use remote
>>>> indexes
>>>> >>>> >> >>> available via http(s) and not file system directories.
>>>> >>>> >> >>> ?
>>>> >>>> >> >>>
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > When I setup Archiva running in Tomcat 6 under CentOS 6.2
>>>> Linux,
>>>> >>>> I
>>>> >>>> >> have
>>>> >>>> >> >>> the
>>>> >>>> >> >>> > folder structure below:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > /archiva
>>>> >>>> >> >>> >    /conf
>>>> >>>> >> >>> >    /data
>>>> >>>> >> >>> >    /derbydb
>>>> >>>> >> >>> >    /indexes
>>>> >>>> >> >>> >    /logs
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > I think the only thing that is not standard is the 'indexes'
>>>> >>>> >> directory.
>>>> >>>> >> >>>  I
>>>> >>>> >> >>> > was trying to setup Archiva and anticipate possibly needing
>>>> to
>>>> >>>> scale
>>>> >>>> >> >>> > horizontally by adding multiple front end servers behind
>>>> our load
>>>> >>>> >> >>> balancer.
>>>> >>>> >> >>> >   I read somewhere (I thought) that the indexes are
>>>> specific to
>>>> >>>> each
>>>> >>>> >> >>> front
>>>> >>>> >> >>> > end server, but that the repository data could be shared,
>>>> so I
>>>> >>>> >> mounted
>>>> >>>> >> >>> the
>>>> >>>> >> >>> > /archiva/data folder as a remote NFS mount.  The rest of the
>>>> >>>> folders
>>>> >>>> >> >>> above
>>>> >>>> >> >>> > are located physically on the archiva server.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > For each repository, the index folder is set to something
>>>> like:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > /archiva/indexes/team1-releases
>>>> >>>> >> >>> > /archiva/indexes/team1-snapshots
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > But my understanding is that these indexes are the internal
>>>> >>>> Archiva
>>>> >>>> >> >>> > indexes, not the publicly consumable indexes.  Nowhere in
>>>> the
>>>> >>>> >> >>> > /archiva/data/repositories/team1-releases or other
>>>> repositories
>>>> >>>> are
>>>> >>>> >> >>> there
>>>> >>>> >> >>> > any .index folders.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > I also have a single repository group that proxies all
>>>> internal
>>>> >>>> and
>>>> >>>> >> >>> > externally proxied repositories under a single virtual
>>>> >>>> repository and
>>>> >>>> >> >>> there
>>>> >>>> >> >>> > are no indexes consolidated under that repository group
>>>> either
>>>> >>>> which
>>>> >>>> >> is
>>>> >>>> >> >>> > what I thought was added in 1.4-M2 (the ability to
>>>> consolidate
>>>> >>>> >> indexes).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Inside of /archiva/indexes, each repository index does have
>>>> a
>>>> >>>> bunch
>>>> >>>> >> of
>>>> >>>> >> >>> > files that look like:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > _7.fdt
>>>> >>>> >> >>> > _7.fdx
>>>> >>>> >> >>> > ...
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6948.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6949.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6950.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6951.gz
>>>> >>>> >> >>> > ...
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > So something is generating these (I assume correctly).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Also in each repository, the setting named "Skip Packed
>>>> Index
>>>> >>>> >> creation"
>>>> >>>> >> >>> is
>>>> >>>> >> >>> > NOT checked.
>>>> >>>> >> >>>
>>>> >>>> >> >>> Those nexux-maven-reposotory-index.xxx.gz files is incremental
>>>> >>>> >> >>> generation. see a file called
>>>> >>>> >> >>> nexus-maven-repository-index-packer.properties which contains
>>>> some
>>>> >>>> >> >>> metadatas on incremental generation.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Hopefully that clarifies things and we can work this out
>>>> and get
>>>> >>>> it
>>>> >>>> >> >>> > addressed as needed for 1.4-M4 :-).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > On Fri, Feb 1, 2013 at 9:51 AM, Olivier Lamy <
>>>> olamy@apache.org>
>>>> >>>> >> wrote:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >> maybe there is an issue and setup indexes to a dedicated
>>>> >>>> location
>>>> >>>> >> >>> doesn't
>>>> >>>> >> >>> >> work.
>>>> >>>> >> >>> >> Is there any indexes files in archiva installation ?
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> BTW why you don't access to indexes via http ?
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> >> > Yep, multiple times.  The server has been up and running
>>>> for a
>>>> >>>> >> >>> number of
>>>> >>>> >> >>> >> > months.
>>>> >>>> >> >>> >> > On Feb 1, 2013 2:33 AM, "Olivier Lamy" <olamy@apache.org
>>>> >
>>>> >>>> wrote:
>>>> >>>> >> >>> >> >
>>>> >>>> >> >>> >> >> Did you try a full scan ?
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >> 2013/1/31 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> >> >> > I'm currently using 1.4-M3 and have configured my
>>>> Archiva
>>>> >>>> >> indexes
>>>> >>>> >> >>> to
>>>> >>>> >> >>> >> >> point
>>>> >>>> >> >>> >> >> > to a dedicated index folder located on the local host
>>>> and
>>>> >>>> put
>>>> >>>> >> the
>>>> >>>> >> >>> data
>>>> >>>> >> >>> >> >> > folder on a shared NFS mount.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > However, the .index folder for local repos is not
>>>> getting
>>>> >>>> >> created
>>>> >>>> >> >>> and
>>>> >>>> >> >>> >> >> thus
>>>> >>>> >> >>> >> >> > my IDE's can only see versions of what I have in my
>>>> local
>>>> >>>> .m2
>>>> >>>> >> >>> >> repository.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > I don't see anything in the logs suggesting that it is
>>>> >>>> failing.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > Thoughts?
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > --
>>>> >>>> >> >>> >> >> > Jeff Vincent
>>>> >>>> >> >>> >> >> > predatorvi@gmail.com
>>>> >>>> >> >>> >> >> > See my LinkedIn profile at:
>>>> >>>> >> >>> >> >> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >>> >> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >> --
>>>> >>>> >> >>> >> >> Olivier Lamy
>>>> >>>> >> >>> >> >> Talend: http://coders.talend.com
>>>> >>>> >> >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> --
>>>> >>>> >> >>> >> Olivier Lamy
>>>> >>>> >> >>> >> Talend: http://coders.talend.com
>>>> >>>> >> >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > --
>>>> >>>> >> >>> > Jeff Vincent
>>>> >>>> >> >>> > predatorvi@gmail.com
>>>> >>>> >> >>> > See my LinkedIn profile at:
>>>> >>>> >> >>> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>>
>>>> >>>> >> >>>
>>>> >>>> >> >>>
>>>> >>>> >> >>> --
>>>> >>>> >> >>> Olivier Lamy
>>>> >>>> >> >>> Talend: http://coders.talend.com
>>>> >>>> >> >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >> --
>>>> >>>> >> >> Jeff Vincent
>>>> >>>> >> >> predatorvi@gmail.com
>>>> >>>> >> >> See my LinkedIn profile at:
>>>> >>>> >> >> http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> > --
>>>> >>>> >> > Jeff Vincent
>>>> >>>> >> > predatorvi@gmail.com
>>>> >>>> >> > See my LinkedIn profile at:
>>>> >>>> >> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >> --
>>>> >>>> >> Olivier Lamy
>>>> >>>> >> Talend: http://coders.talend.com
>>>> >>>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >>
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >
>>>> >>>> > --
>>>> >>>> > Jeff Vincent
>>>> >>>> > predatorvi@gmail.com
>>>> >>>> > See my LinkedIn profile at:
>>>> >>>> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Olivier Lamy
>>>> >>>> Talend: http://coders.talend.com
>>>> >>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> Jeff Vincent
>>>> >>> predatorvi@gmail.com
>>>> >>> See my LinkedIn profile at:
>>>> >>> http://www.linkedin.com/in/rjeffreyvincent
>>>> >>> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Jeff Vincent
>>>> >> predatorvi@gmail.com
>>>> >> See my LinkedIn profile at:
>>>> >> http://www.linkedin.com/in/rjeffreyvincent
>>>> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Olivier Lamy
>>>> > Talend: http://coders.talend.com
>>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>
>>>>
>>>>
>>>> --
>>>> Olivier Lamy
>>>> Talend: http://coders.talend.com
>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>
>>>> 2013/2/6 Olivier Lamy <olamy@apache.org>:
>>>> > Hi,
>>>> > Apologize for delay.
>>>> >
>>>> > 2013/2/4 Jeff <predatorvi@gmail.com>:
>>>> >> I'm now trying to switch up the index directories on my repositories
>>>> and am
>>>> >> banging my head.  I've simply removed the "Index Directory" setting on
>>>> my
>>>> >> repos and I see the .indexer folder getting created.  Which is good.
>>>> >>
>>>> >> However, according to
>>>> >> http://archiva.apache.org/docs/1.4-M3/adminguide/repositories.html, it
>>>> >> states:
>>>> >>
>>>> >> *Another directory named .index is also created at the root of the
>>>> >> repository directory. This is not configurable though as it contains
>>>> the
>>>> >> packaged/bundled index which is consumed by different consumers of the
>>>> >> index such as M2Eclipse. *
>>>> > doh documentation is wrong (we use .indexer)
>>>> >>
>>>> >>
>>>> >> However, it is NOT getting created.  I've restarted tomcat, I've
>>>> forced a
>>>> >> full index scan on every repository as well as a Directory scan.
>>>> >>
>>>> >> The above statement also lead me to believe that regardless of the
>>>> location
>>>> >> of ".indexer" that ".index" would still get created at the root but it
>>>> >> isn't.
>>>> >>
>>>> >> On one repository I tried setting "Index Directory" to ".index", but I
>>>> get
>>>> >> an error that it can't create "/usr/share/tomcat6/.index".
>>>> > A bit weird.
>>>> > What is -Dappserver.base value ?
>>>> > BTW I agree there are some possible with relative path (I fixed some in
>>>> trunk).
>>>> > But while studying your case, I think this need more
>>>> attention/debugging.
>>>> > I'm on it. I will keep you informed as soon I have fixed (or at least
>>>> > understood the issue :-)) that in source code.
>>>> >>
>>>> >> Thanks for any additional direction...
>>>> >>
>>>> >>
>>>> >> On Mon, Feb 4, 2013 at 8:47 AM, Jeff <predatorvi@gmail.com> wrote:
>>>> >>
>>>> >>> I don't *yet* have a need, but our user base is growing and so is our
>>>> >>> CI/CD pipeline (number of GO and Jenkins build agents/slaves) and I
>>>> wanted
>>>> >>> to plan ahead.  I am trying to find the document that talked about
>>>> keeping
>>>> >>> indexes local while allowing a shared repo, but I have been
>>>> unsuccessful
>>>> >>> thus far.  Maybe I dreamed it up.  :-)
>>>> >>>
>>>> >>> Thanks for your help.
>>>> >>>
>>>> >>>
>>>> >>> On Mon, Feb 4, 2013 at 2:56 AM, Olivier Lamy <olamy@apache.org>
>>>> wrote:
>>>> >>>
>>>> >>>> complicated to have scabality when serving files :-(
>>>> >>>> we don't (yet) any sync possible (that's something we start
>>>> discussing
>>>> >>>> on dev@ with maybe using new Apache project called helix).
>>>> >>>> With your setup each repository will be independent.
>>>> >>>> Do you have so many users to need that ?
>>>> >>>>
>>>> >>>> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>>> >>>> > I will do that.
>>>> >>>> >
>>>> >>>> > However, how do I configure the repository to support the
>>>> previously
>>>> >>>> > mentioned horizontal scaling with a shared repository?
>>>> >>>> >
>>>> >>>> >
>>>> >>>> > On Sat, Feb 2, 2013 at 10:26 AM, Olivier Lamy <olamy@apache.org>
>>>> wrote:
>>>> >>>> >
>>>> >>>> >> our default is .indexer.
>>>> >>>> >> Oh I believe using an absolute path for the index location doesn't
>>>> >>>> >> work and it's not available via http(s).
>>>> >>>> >> Can you simply change the index path to .index or leave it blank
>>>> >>>> >> (default is .indexer)  ?
>>>> >>>> >> As it that will be available as this one
>>>> >>>> >>
>>>> >>>>
>>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> > Also, isn't the default index directory supposed to be .index?
>>>> >>>>  That's
>>>> >>>> >> what
>>>> >>>> >> > it seems to be on Maven Central.
>>>> >>>> >> >
>>>> >>>> >> > http://repo.maven.apache.org/maven2/.index/
>>>> >>>> >> >
>>>> >>>> >> > Thanks.
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> > On Fri, Feb 1, 2013 at 3:28 PM, Jeff <predatorvi@gmail.com>
>>>> wrote:
>>>> >>>> >> >
>>>> >>>> >> >> If I attempt to go to a similar URL for any of our repositories
>>>> >>>> there is
>>>> >>>> >> >> no .indexer or .index path such as:
>>>> >>>> >> >>
>>>> >>>> >> >> http://archiva/repository/dna-releases/.indexer
>>>> >>>> >> >> http://archiva/repository/dna-snapshots/.index
>>>> >>>> >> >> http://archiva/repository/dnabackend-releases/.indexer
>>>> >>>> >> >> http://archiva/repository/dnabackend-snapshots/.index
>>>> >>>> >> >>
>>>> >>>> >> >> Is this because I moved them?  If so, I am happy to put them
>>>> back
>>>> >>>> to the
>>>> >>>> >> >> normal location, but then what would change to allow multiple
>>>> >>>> archiva
>>>> >>>> >> >> tomcat servers to use the same 'data' repositories?
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >> On Fri, Feb 1, 2013 at 2:10 PM, Olivier Lamy <olamy@apache.org
>>>> >
>>>> >>>> wrote:
>>>> >>>> >> >>
>>>> >>>> >> >>> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> > I'm not sure what you mean by "why you don't access to
>>>> indexes
>>>> >>>> via
>>>> >>>> >> >>> http"...
>>>> >>>> >> >>> Something like
>>>> >>>> >> >>>
>>>> >>>> >>
>>>> >>>>
>>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer/
>>>> >>>> >> >>>
>>>> >>>> >> >>> it's available for all repositories (and for group too since
>>>> >>>> 1.4-M2 in
>>>> >>>> >> >>> path lie archiva/repository/${grouprepositoryId}/.indexer/
>>>> >>>> >> >>>
>>>> >>>> >> >>> Don't know exactly how it works in IDE. But they use remote
>>>> indexes
>>>> >>>> >> >>> available via http(s) and not file system directories.
>>>> >>>> >> >>> ?
>>>> >>>> >> >>>
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > When I setup Archiva running in Tomcat 6 under CentOS 6.2
>>>> Linux,
>>>> >>>> I
>>>> >>>> >> have
>>>> >>>> >> >>> the
>>>> >>>> >> >>> > folder structure below:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > /archiva
>>>> >>>> >> >>> >    /conf
>>>> >>>> >> >>> >    /data
>>>> >>>> >> >>> >    /derbydb
>>>> >>>> >> >>> >    /indexes
>>>> >>>> >> >>> >    /logs
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > I think the only thing that is not standard is the 'indexes'
>>>> >>>> >> directory.
>>>> >>>> >> >>>  I
>>>> >>>> >> >>> > was trying to setup Archiva and anticipate possibly needing
>>>> to
>>>> >>>> scale
>>>> >>>> >> >>> > horizontally by adding multiple front end servers behind
>>>> our load
>>>> >>>> >> >>> balancer.
>>>> >>>> >> >>> >   I read somewhere (I thought) that the indexes are
>>>> specific to
>>>> >>>> each
>>>> >>>> >> >>> front
>>>> >>>> >> >>> > end server, but that the repository data could be shared,
>>>> so I
>>>> >>>> >> mounted
>>>> >>>> >> >>> the
>>>> >>>> >> >>> > /archiva/data folder as a remote NFS mount.  The rest of the
>>>> >>>> folders
>>>> >>>> >> >>> above
>>>> >>>> >> >>> > are located physically on the archiva server.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > For each repository, the index folder is set to something
>>>> like:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > /archiva/indexes/team1-releases
>>>> >>>> >> >>> > /archiva/indexes/team1-snapshots
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > But my understanding is that these indexes are the internal
>>>> >>>> Archiva
>>>> >>>> >> >>> > indexes, not the publicly consumable indexes.  Nowhere in
>>>> the
>>>> >>>> >> >>> > /archiva/data/repositories/team1-releases or other
>>>> repositories
>>>> >>>> are
>>>> >>>> >> >>> there
>>>> >>>> >> >>> > any .index folders.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > I also have a single repository group that proxies all
>>>> internal
>>>> >>>> and
>>>> >>>> >> >>> > externally proxied repositories under a single virtual
>>>> >>>> repository and
>>>> >>>> >> >>> there
>>>> >>>> >> >>> > are no indexes consolidated under that repository group
>>>> either
>>>> >>>> which
>>>> >>>> >> is
>>>> >>>> >> >>> > what I thought was added in 1.4-M2 (the ability to
>>>> consolidate
>>>> >>>> >> indexes).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Inside of /archiva/indexes, each repository index does have
>>>> a
>>>> >>>> bunch
>>>> >>>> >> of
>>>> >>>> >> >>> > files that look like:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > _7.fdt
>>>> >>>> >> >>> > _7.fdx
>>>> >>>> >> >>> > ...
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6948.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6949.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6950.gz
>>>> >>>> >> >>> > nexux-maven-reposotory-index.6951.gz
>>>> >>>> >> >>> > ...
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > So something is generating these (I assume correctly).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Also in each repository, the setting named "Skip Packed
>>>> Index
>>>> >>>> >> creation"
>>>> >>>> >> >>> is
>>>> >>>> >> >>> > NOT checked.
>>>> >>>> >> >>>
>>>> >>>> >> >>> Those nexux-maven-reposotory-index.xxx.gz files is incremental
>>>> >>>> >> >>> generation. see a file called
>>>> >>>> >> >>> nexus-maven-repository-index-packer.properties which contains
>>>> some
>>>> >>>> >> >>> metadatas on incremental generation.
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > Hopefully that clarifies things and we can work this out
>>>> and get
>>>> >>>> it
>>>> >>>> >> >>> > addressed as needed for 1.4-M4 :-).
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > On Fri, Feb 1, 2013 at 9:51 AM, Olivier Lamy <
>>>> olamy@apache.org>
>>>> >>>> >> wrote:
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >> maybe there is an issue and setup indexes to a dedicated
>>>> >>>> location
>>>> >>>> >> >>> doesn't
>>>> >>>> >> >>> >> work.
>>>> >>>> >> >>> >> Is there any indexes files in archiva installation ?
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> BTW why you don't access to indexes via http ?
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> >> > Yep, multiple times.  The server has been up and running
>>>> for a
>>>> >>>> >> >>> number of
>>>> >>>> >> >>> >> > months.
>>>> >>>> >> >>> >> > On Feb 1, 2013 2:33 AM, "Olivier Lamy" <olamy@apache.org
>>>> >
>>>> >>>> wrote:
>>>> >>>> >> >>> >> >
>>>> >>>> >> >>> >> >> Did you try a full scan ?
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >> 2013/1/31 Jeff <predatorvi@gmail.com>:
>>>> >>>> >> >>> >> >> > I'm currently using 1.4-M3 and have configured my
>>>> Archiva
>>>> >>>> >> indexes
>>>> >>>> >> >>> to
>>>> >>>> >> >>> >> >> point
>>>> >>>> >> >>> >> >> > to a dedicated index folder located on the local host
>>>> and
>>>> >>>> put
>>>> >>>> >> the
>>>> >>>> >> >>> data
>>>> >>>> >> >>> >> >> > folder on a shared NFS mount.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > However, the .index folder for local repos is not
>>>> getting
>>>> >>>> >> created
>>>> >>>> >> >>> and
>>>> >>>> >> >>> >> >> thus
>>>> >>>> >> >>> >> >> > my IDE's can only see versions of what I have in my
>>>> local
>>>> >>>> .m2
>>>> >>>> >> >>> >> repository.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > I don't see anything in the logs suggesting that it is
>>>> >>>> failing.
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > Thoughts?
>>>> >>>> >> >>> >> >> >
>>>> >>>> >> >>> >> >> > --
>>>> >>>> >> >>> >> >> > Jeff Vincent
>>>> >>>> >> >>> >> >> > predatorvi@gmail.com
>>>> >>>> >> >>> >> >> > See my LinkedIn profile at:
>>>> >>>> >> >>> >> >> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >>> >> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >> >> --
>>>> >>>> >> >>> >> >> Olivier Lamy
>>>> >>>> >> >>> >> >> Talend: http://coders.talend.com
>>>> >>>> >> >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>> >> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >> --
>>>> >>>> >> >>> >> Olivier Lamy
>>>> >>>> >> >>> >> Talend: http://coders.talend.com
>>>> >>>> >> >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>> >>
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> >
>>>> >>>> >> >>> > --
>>>> >>>> >> >>> > Jeff Vincent
>>>> >>>> >> >>> > predatorvi@gmail.com
>>>> >>>> >> >>> > See my LinkedIn profile at:
>>>> >>>> >> >>> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>>
>>>> >>>> >> >>>
>>>> >>>> >> >>>
>>>> >>>> >> >>> --
>>>> >>>> >> >>> Olivier Lamy
>>>> >>>> >> >>> Talend: http://coders.talend.com
>>>> >>>> >> >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >> >>>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >>
>>>> >>>> >> >> --
>>>> >>>> >> >> Jeff Vincent
>>>> >>>> >> >> predatorvi@gmail.com
>>>> >>>> >> >> See my LinkedIn profile at:
>>>> >>>> >> >> http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >> >>
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> >
>>>> >>>> >> > --
>>>> >>>> >> > Jeff Vincent
>>>> >>>> >> > predatorvi@gmail.com
>>>> >>>> >> > See my LinkedIn profile at:
>>>> >>>> >> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >>
>>>> >>>> >> --
>>>> >>>> >> Olivier Lamy
>>>> >>>> >> Talend: http://coders.talend.com
>>>> >>>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>> >>
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >
>>>> >>>> > --
>>>> >>>> > Jeff Vincent
>>>> >>>> > predatorvi@gmail.com
>>>> >>>> > See my LinkedIn profile at:
>>>> >>>> > http://www.linkedin.com/in/rjeffreyvincent
>>>> >>>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Olivier Lamy
>>>> >>>> Talend: http://coders.talend.com
>>>> >>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> Jeff Vincent
>>>> >>> predatorvi@gmail.com
>>>> >>> See my LinkedIn profile at:
>>>> >>> http://www.linkedin.com/in/rjeffreyvincent
>>>> >>> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Jeff Vincent
>>>> >> predatorvi@gmail.com
>>>> >> See my LinkedIn profile at:
>>>> >> http://www.linkedin.com/in/rjeffreyvincent
>>>> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Olivier Lamy
>>>> > Talend: http://coders.talend.com
>>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>
>>>>
>>>>
>>>> --
>>>> Olivier Lamy
>>>> Talend: http://coders.talend.com
>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>
>>>
>>>
>>>
>>> --
>>> Jeff Vincent
>>> predatorvi@gmail.com
>>> See my LinkedIn profile at:
>>> http://www.linkedin.com/in/rjeffreyvincent
>>> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>
>>
>>
>>
>> --
>> Jeff Vincent
>> predatorvi@gmail.com
>> See my LinkedIn profile at:
>> http://www.linkedin.com/in/rjeffreyvincent
>> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>
>
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy



-- 
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy

Mime
View raw message