archiva-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Lamy <ol...@apache.org>
Subject Re: Index help...
Date Wed, 06 Feb 2013 10:24:33 GMT
Hi,
Apologize for delay.

2013/2/4 Jeff <predatorvi@gmail.com>:
> I'm now trying to switch up the index directories on my repositories and am
> banging my head.  I've simply removed the "Index Directory" setting on my
> repos and I see the .indexer folder getting created.  Which is good.
>
> However, according to
> http://archiva.apache.org/docs/1.4-M3/adminguide/repositories.html, it
> states:
>
> *Another directory named .index is also created at the root of the
> repository directory. This is not configurable though as it contains the
> packaged/bundled index which is consumed by different consumers of the
> index such as M2Eclipse. *
doh documentation is wrong (we use .indexer)
>
>
> However, it is NOT getting created.  I've restarted tomcat, I've forced a
> full index scan on every repository as well as a Directory scan.
>
> The above statement also lead me to believe that regardless of the location
> of ".indexer" that ".index" would still get created at the root but it
> isn't.
>
> On one repository I tried setting "Index Directory" to ".index", but I get
> an error that it can't create "/usr/share/tomcat6/.index".
A bit weird.
What is -Dappserver.base value ?
BTW I agree there are some possible with relative path (I fixed some in trunk).
But while studying your case, I think this need more attention/debugging.
I'm on it. I will keep you informed as soon I have fixed (or at least
understood the issue :-)) that in source code.
>
> Thanks for any additional direction...
>
>
> On Mon, Feb 4, 2013 at 8:47 AM, Jeff <predatorvi@gmail.com> wrote:
>
>> I don't *yet* have a need, but our user base is growing and so is our
>> CI/CD pipeline (number of GO and Jenkins build agents/slaves) and I wanted
>> to plan ahead.  I am trying to find the document that talked about keeping
>> indexes local while allowing a shared repo, but I have been unsuccessful
>> thus far.  Maybe I dreamed it up.  :-)
>>
>> Thanks for your help.
>>
>>
>> On Mon, Feb 4, 2013 at 2:56 AM, Olivier Lamy <olamy@apache.org> wrote:
>>
>>> complicated to have scabality when serving files :-(
>>> we don't (yet) any sync possible (that's something we start discussing
>>> on dev@ with maybe using new Apache project called helix).
>>> With your setup each repository will be independent.
>>> Do you have so many users to need that ?
>>>
>>> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>> > I will do that.
>>> >
>>> > However, how do I configure the repository to support the previously
>>> > mentioned horizontal scaling with a shared repository?
>>> >
>>> >
>>> > On Sat, Feb 2, 2013 at 10:26 AM, Olivier Lamy <olamy@apache.org> wrote:
>>> >
>>> >> our default is .indexer.
>>> >> Oh I believe using an absolute path for the index location doesn't
>>> >> work and it's not available via http(s).
>>> >> Can you simply change the index path to .index or leave it blank
>>> >> (default is .indexer)  ?
>>> >> As it that will be available as this one
>>> >>
>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer
>>> >>
>>> >>
>>> >>
>>> >> 2013/2/2 Jeff <predatorvi@gmail.com>:
>>> >> > Also, isn't the default index directory supposed to be .index?
>>>  That's
>>> >> what
>>> >> > it seems to be on Maven Central.
>>> >> >
>>> >> > http://repo.maven.apache.org/maven2/.index/
>>> >> >
>>> >> > Thanks.
>>> >> >
>>> >> >
>>> >> > On Fri, Feb 1, 2013 at 3:28 PM, Jeff <predatorvi@gmail.com>
wrote:
>>> >> >
>>> >> >> If I attempt to go to a similar URL for any of our repositories
>>> there is
>>> >> >> no .indexer or .index path such as:
>>> >> >>
>>> >> >> http://archiva/repository/dna-releases/.indexer
>>> >> >> http://archiva/repository/dna-snapshots/.index
>>> >> >> http://archiva/repository/dnabackend-releases/.indexer
>>> >> >> http://archiva/repository/dnabackend-snapshots/.index
>>> >> >>
>>> >> >> Is this because I moved them?  If so, I am happy to put them
back
>>> to the
>>> >> >> normal location, but then what would change to allow multiple
>>> archiva
>>> >> >> tomcat servers to use the same 'data' repositories?
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Fri, Feb 1, 2013 at 2:10 PM, Olivier Lamy <olamy@apache.org>
>>> wrote:
>>> >> >>
>>> >> >>> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>> >> >>> > I'm not sure what you mean by "why you don't access
to indexes
>>> via
>>> >> >>> http"...
>>> >> >>> Something like
>>> >> >>>
>>> >>
>>> https://archiva-repository.apache.org/archiva/repository/snapshots/.indexer/
>>> >> >>>
>>> >> >>> it's available for all repositories (and for group too
since
>>> 1.4-M2 in
>>> >> >>> path lie archiva/repository/${grouprepositoryId}/.indexer/
>>> >> >>>
>>> >> >>> Don't know exactly how it works in IDE. But they use remote
indexes
>>> >> >>> available via http(s) and not file system directories.
>>> >> >>> ?
>>> >> >>>
>>> >> >>> >
>>> >> >>> > When I setup Archiva running in Tomcat 6 under CentOS
6.2 Linux,
>>> I
>>> >> have
>>> >> >>> the
>>> >> >>> > folder structure below:
>>> >> >>> >
>>> >> >>> > /archiva
>>> >> >>> >    /conf
>>> >> >>> >    /data
>>> >> >>> >    /derbydb
>>> >> >>> >    /indexes
>>> >> >>> >    /logs
>>> >> >>> >
>>> >> >>> > I think the only thing that is not standard is the
'indexes'
>>> >> directory.
>>> >> >>>  I
>>> >> >>> > was trying to setup Archiva and anticipate possibly
needing to
>>> scale
>>> >> >>> > horizontally by adding multiple front end servers
behind our load
>>> >> >>> balancer.
>>> >> >>> >   I read somewhere (I thought) that the indexes are
specific to
>>> each
>>> >> >>> front
>>> >> >>> > end server, but that the repository data could be
shared, so I
>>> >> mounted
>>> >> >>> the
>>> >> >>> > /archiva/data folder as a remote NFS mount.  The rest
of the
>>> folders
>>> >> >>> above
>>> >> >>> > are located physically on the archiva server.
>>> >> >>> >
>>> >> >>> > For each repository, the index folder is set to something
like:
>>> >> >>> >
>>> >> >>> > /archiva/indexes/team1-releases
>>> >> >>> > /archiva/indexes/team1-snapshots
>>> >> >>> >
>>> >> >>> > But my understanding is that these indexes are the
internal
>>> Archiva
>>> >> >>> > indexes, not the publicly consumable indexes.  Nowhere
in the
>>> >> >>> > /archiva/data/repositories/team1-releases or other
repositories
>>> are
>>> >> >>> there
>>> >> >>> > any .index folders.
>>> >> >>> >
>>> >> >>> > I also have a single repository group that proxies
all internal
>>> and
>>> >> >>> > externally proxied repositories under a single virtual
>>> repository and
>>> >> >>> there
>>> >> >>> > are no indexes consolidated under that repository
group either
>>> which
>>> >> is
>>> >> >>> > what I thought was added in 1.4-M2 (the ability to
consolidate
>>> >> indexes).
>>> >> >>> >
>>> >> >>> > Inside of /archiva/indexes, each repository index
does have a
>>> bunch
>>> >> of
>>> >> >>> > files that look like:
>>> >> >>> >
>>> >> >>> > _7.fdt
>>> >> >>> > _7.fdx
>>> >> >>> > ...
>>> >> >>> > nexux-maven-reposotory-index.6948.gz
>>> >> >>> > nexux-maven-reposotory-index.6949.gz
>>> >> >>> > nexux-maven-reposotory-index.6950.gz
>>> >> >>> > nexux-maven-reposotory-index.6951.gz
>>> >> >>> > ...
>>> >> >>> >
>>> >> >>> > So something is generating these (I assume correctly).
>>> >> >>> >
>>> >> >>> > Also in each repository, the setting named "Skip Packed
Index
>>> >> creation"
>>> >> >>> is
>>> >> >>> > NOT checked.
>>> >> >>>
>>> >> >>> Those nexux-maven-reposotory-index.xxx.gz files is incremental
>>> >> >>> generation. see a file called
>>> >> >>> nexus-maven-repository-index-packer.properties which contains
some
>>> >> >>> metadatas on incremental generation.
>>> >> >>> >
>>> >> >>> > Hopefully that clarifies things and we can work this
out and get
>>> it
>>> >> >>> > addressed as needed for 1.4-M4 :-).
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Fri, Feb 1, 2013 at 9:51 AM, Olivier Lamy <olamy@apache.org>
>>> >> wrote:
>>> >> >>> >
>>> >> >>> >> maybe there is an issue and setup indexes to a
dedicated
>>> location
>>> >> >>> doesn't
>>> >> >>> >> work.
>>> >> >>> >> Is there any indexes files in archiva installation
?
>>> >> >>> >>
>>> >> >>> >> BTW why you don't access to indexes via http ?
>>> >> >>> >>
>>> >> >>> >> 2013/2/1 Jeff <predatorvi@gmail.com>:
>>> >> >>> >> > Yep, multiple times.  The server has been
up and running for a
>>> >> >>> number of
>>> >> >>> >> > months.
>>> >> >>> >> > On Feb 1, 2013 2:33 AM, "Olivier Lamy" <olamy@apache.org>
>>> wrote:
>>> >> >>> >> >
>>> >> >>> >> >> Did you try a full scan ?
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >> 2013/1/31 Jeff <predatorvi@gmail.com>:
>>> >> >>> >> >> > I'm currently using 1.4-M3 and have
configured my Archiva
>>> >> indexes
>>> >> >>> to
>>> >> >>> >> >> point
>>> >> >>> >> >> > to a dedicated index folder located
on the local host and
>>> put
>>> >> the
>>> >> >>> data
>>> >> >>> >> >> > folder on a shared NFS mount.
>>> >> >>> >> >> >
>>> >> >>> >> >> > However, the .index folder for local
repos is not getting
>>> >> created
>>> >> >>> and
>>> >> >>> >> >> thus
>>> >> >>> >> >> > my IDE's can only see versions of
what I have in my local
>>> .m2
>>> >> >>> >> repository.
>>> >> >>> >> >> >
>>> >> >>> >> >> > I don't see anything in the logs
suggesting that it is
>>> failing.
>>> >> >>> >> >> >
>>> >> >>> >> >> > Thoughts?
>>> >> >>> >> >> >
>>> >> >>> >> >> > --
>>> >> >>> >> >> > Jeff Vincent
>>> >> >>> >> >> > predatorvi@gmail.com
>>> >> >>> >> >> > See my LinkedIn profile at:
>>> >> >>> >> >> > http://www.linkedin.com/in/rjeffreyvincent
>>> >> >>> >> >> > I ♥ DropBox <http://db.tt/9O6LfBX>
!!
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >> --
>>> >> >>> >> >> Olivier Lamy
>>> >> >>> >> >> Talend: http://coders.talend.com
>>> >> >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >>> >> >>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> --
>>> >> >>> >> Olivier Lamy
>>> >> >>> >> Talend: http://coders.talend.com
>>> >> >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >>> >>
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > --
>>> >> >>> > Jeff Vincent
>>> >> >>> > predatorvi@gmail.com
>>> >> >>> > See my LinkedIn profile at:
>>> >> >>> > http://www.linkedin.com/in/rjeffreyvincent
>>> >> >>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> --
>>> >> >>> Olivier Lamy
>>> >> >>> Talend: http://coders.talend.com
>>> >> >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Jeff Vincent
>>> >> >> predatorvi@gmail.com
>>> >> >> See my LinkedIn profile at:
>>> >> >> http://www.linkedin.com/in/rjeffreyvincent
>>> >> >> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Jeff Vincent
>>> >> > predatorvi@gmail.com
>>> >> > See my LinkedIn profile at:
>>> >> > http://www.linkedin.com/in/rjeffreyvincent
>>> >> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Olivier Lamy
>>> >> Talend: http://coders.talend.com
>>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Jeff Vincent
>>> > predatorvi@gmail.com
>>> > See my LinkedIn profile at:
>>> > http://www.linkedin.com/in/rjeffreyvincent
>>> > I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>>
>>>
>>>
>>> --
>>> Olivier Lamy
>>> Talend: http://coders.talend.com
>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>
>>
>>
>>
>> --
>> Jeff Vincent
>> predatorvi@gmail.com
>> See my LinkedIn profile at:
>> http://www.linkedin.com/in/rjeffreyvincent
>> I ♥ DropBox <http://db.tt/9O6LfBX> !!
>>
>
>
>
> --
> Jeff Vincent
> predatorvi@gmail.com
> See my LinkedIn profile at:
> http://www.linkedin.com/in/rjeffreyvincent
> I ♥ DropBox <http://db.tt/9O6LfBX> !!



--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy

Mime
View raw message