archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Sanchez <car...@apache.org>
Subject Re: Searching pom elements and generic metadata
Date Thu, 20 Nov 2014 10:09:44 GMT
I have added these REST methods with JCR implementations

artifactsByProjectVersionMetadata/{key}/{value}
artifactsByMetadata/{key}/{value}
artifactsByProperty/{key}/{value}
searchArtifacts/{text}
searchArtifacts/{key}/{text}


Docs here
https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search

Any suggestions on how to run these types of queries in Cassandra?
https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764



On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <carlos@csanchez.org> wrote:

> I have added search methods for any property, including facets
>
> restServices/archivaServices/browseService/searchArtifacts/key/text
> restServices/archivaServices/browseService/searchArtifacts/text
>
> that search in JCR
>
> SELECT * FROM [archiva:projectVersion] AS projectVersion
>   LEFT OUTER JOIN [archiva:artifact] AS artifact ON ISCHILDNODE(artifact,
> projectVersion)
>   LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
> projectVersion)
>   WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
> $value))
>     AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>
> But seems that performance is bad for a "normal" sized repo. Any
> suggestions?
> I'll try with subselects and tuning jcr, I would imagine that it should
> work decently and there's something wrong with archiva.
>
>
>
> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <olamy@apache.org> wrote:
>
>> On 7 November 2014 07:42, Carlos Sanchez <carlos@apache.org> wrote:
>> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
>> > passing the repository
>> >
>> >
>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>> >
>> > The problem I hit with not passing the repo is that
>> > AbstractTestService.buildArtifacts needs it. Will look later.
>> >
>> > About adding the other pom fields to the search index, from what I saw I
>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in
>> the
>> > maven-indexer project, and then archiva would just use that?
>>
>> In such case you will have to modify the lucene model in the maven
>> indexer project ( can certainly breaks a lot :-) ).
>>
>> >
>> > Are the fields indexed already searchable somewhere in archiva with
>> field
>> > selection? e.g. g:myGroupId or v:1.0 ?
>>
>> Not associated at this level but that's already in our repository.
>>
>>
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <olamy@apache.org> wrote:
>> >
>> >> Hi,
>> >> I would make repositoryId as a query param so will make it optional.
>> >> If empty/null use the users repositories?
>> >>
>> >> On 6 November 2014 06:22, Carlos Sanchez <carlos@apache.org> wrote:
>> >> > Thanks, got an initial implementation of getArtifactsByMetadata at
>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>> >> >
>> >> > I added the repositoryId param, like other methods, but I'm not sure
>> if
>> >> > that would be limiting the feature, searching in all repos would be
>> more
>> >> > interesting.
>> >> >
>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <olamy@apache.org>
>> wrote:
>> >> >
>> >> >> basic Cassandra requests tru Hector client won't work.
>> >> >> You will have to write some manual cql queries.
>> >> >>
>> >> >> On 5 November 2014 22:07, Carlos Sanchez <carlos@csanchez.org>
>> wrote:
>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <olamy@apache.org>
>> >> wrote:
>> >> >> >
>> >> >> >> I would add that in the interface MetadataRepository,
add a test
>> in
>> >> >> >> the tck and implements it for all providers :-)
>> >> >> >> Maybe having 2 methods:
>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value)
( maybe
>> a Map
>> >> >> >> for multiple value pair? will you test the full value
or a
>> matching?)
>> >> >> >>
>> >> >> >
>> >> >> > is the matching syntax for cassandra an jcr different?
>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to
do
>> >> matching
>> >> >> is
>> >> >> > it going to work only in jcr?
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) (
same
>> >> questions )
>> >> >> >>
>> >> >> >> To start I won't go with a Map and exact matching :-)
(I reckon
>> pom
>> >> >> >> content can be a pain with Cassandra impl)
>> >> >> >>
>> >> >> >> WDYT?
>> >> >> >>
>> >> >> >> Olivier
>> >> >> >>
>> >> >> >>
>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <carlos@apache.org>
>> wrote:
>> >> >> >> > Thanks for the pointers, so if I add a method to
>> >> JcrMetadataRepository
>> >> >> >> >
>> >> >> >> > getArtifactsByProperty(key, value)
>> >> >> >> >
>> >> >> >> > similar to getArtifactsByDateRange, writing the correct
JCR
>> query,
>> >> I
>> >> >> >> could
>> >> >> >> > access all that metadata?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > In which of the rest services would it make sense
to add such a
>> >> >> search?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
>> olamy@apache.org>
>> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> Hi,
>> >> >> >> >> The current SearchService search in maven indexer
available
>> datas.
>> >> >> >> >> All the data you want to search are not available
here but in
>> the
>> >> >> >> >> MetadataRepository ( so jcr, file or the new
Cassandra).
>> >> >> >> >> And currently there is no search methods for
that!
>> >> >> >> >> So really good news to see you as a volunteer
to implement
>> this!
>> >> :-)
>> >> >> >> >> Let me know if you need any pointers.
>> >> >> >> >>
>> >> >> >> >> Cheers
>> >> >> >> >> Olivier
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <carlos@apache.org>
>> >> wrote:
>> >> >> >> >> > Hi,
>> >> >> >> >> >
>> >> >> >> >> > Two things I wasn't able to do and was going
to implement
>> >> (unless I
>> >> >> >> >> missed
>> >> >> >> >> > something):
>> >> >> >> >> >
>> >> >> >> >> > * search for generic metadata
>> >> >> >> >> > * search for pom contents
>> >> >> >> >> >
>> >> >> >> >> > The way the search queries are built in
SearchService only
>> looks
>> >> >> for
>> >> >> >> the
>> >> >> >> >> > typical fields (artifactId, groupId,...),
so that needs to
>> be
>> >> >> changed,
>> >> >> >> >> but
>> >> >> >> >> > I don't know if the pom and generic metadata
fields are even
>> >> >> indexed,
>> >> >> >> any
>> >> >> >> >> > pointers?
>> >> >> >> >> >
>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue"
and
>> >> something
>> >> >> >> like
>> >> >> >> >> > "developer:carlos*" through the rest api
and ui.
>> >> >> >> >> >
>> >> >> >> >> > Thanks
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Olivier Lamy
>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Olivier Lamy
>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Olivier Lamy
>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Olivier Lamy
>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message