archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Sanchez <car...@apache.org>
Subject Re: Searching pom elements and generic metadata
Date Mon, 24 Nov 2014 19:05:57 GMT
PR submitted as https://github.com/apache/archiva/pull/14

feedback welcome, there are some new methods there

On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <carlos@apache.org> wrote:

> I have added these REST methods with JCR implementations
>
> artifactsByProjectVersionMetadata/{key}/{value}
> artifactsByMetadata/{key}/{value}
> artifactsByProperty/{key}/{value}
> searchArtifacts/{text}
> searchArtifacts/{key}/{text}
>
>
> Docs here
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
>
> Any suggestions on how to run these types of queries in Cassandra?
>
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
>
>
>
> On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <carlos@csanchez.org>
> wrote:
>
>> I have added search methods for any property, including facets
>>
>> restServices/archivaServices/browseService/searchArtifacts/key/text
>> restServices/archivaServices/browseService/searchArtifacts/text
>>
>> that search in JCR
>>
>> SELECT * FROM [archiva:projectVersion] AS projectVersion
>>   LEFT OUTER JOIN [archiva:artifact] AS artifact ON ISCHILDNODE(artifact,
>> projectVersion)
>>   LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
>> projectVersion)
>>   WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
>> $value))
>>     AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>>
>> But seems that performance is bad for a "normal" sized repo. Any
>> suggestions?
>> I'll try with subselects and tuning jcr, I would imagine that it should
>> work decently and there's something wrong with archiva.
>>
>>
>>
>> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <olamy@apache.org> wrote:
>>
>>> On 7 November 2014 07:42, Carlos Sanchez <carlos@apache.org> wrote:
>>> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
>>> > passing the repository
>>> >
>>> >
>>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>>> >
>>> > The problem I hit with not passing the repo is that
>>> > AbstractTestService.buildArtifacts needs it. Will look later.
>>> >
>>> > About adding the other pom fields to the search index, from what I saw
>>> I
>>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in
>>> the
>>> > maven-indexer project, and then archiva would just use that?
>>>
>>> In such case you will have to modify the lucene model in the maven
>>> indexer project ( can certainly breaks a lot :-) ).
>>>
>>> >
>>> > Are the fields indexed already searchable somewhere in archiva with
>>> field
>>> > selection? e.g. g:myGroupId or v:1.0 ?
>>>
>>> Not associated at this level but that's already in our repository.
>>>
>>>
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <olamy@apache.org>
>>> wrote:
>>> >
>>> >> Hi,
>>> >> I would make repositoryId as a query param so will make it optional.
>>> >> If empty/null use the users repositories?
>>> >>
>>> >> On 6 November 2014 06:22, Carlos Sanchez <carlos@apache.org> wrote:
>>> >> > Thanks, got an initial implementation of getArtifactsByMetadata
at
>>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>>> >> >
>>> >> > I added the repositoryId param, like other methods, but I'm not
>>> sure if
>>> >> > that would be limiting the feature, searching in all repos would
be
>>> more
>>> >> > interesting.
>>> >> >
>>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <olamy@apache.org>
>>> wrote:
>>> >> >
>>> >> >> basic Cassandra requests tru Hector client won't work.
>>> >> >> You will have to write some manual cql queries.
>>> >> >>
>>> >> >> On 5 November 2014 22:07, Carlos Sanchez <carlos@csanchez.org>
>>> wrote:
>>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <olamy@apache.org>
>>> >> wrote:
>>> >> >> >
>>> >> >> >> I would add that in the interface MetadataRepository,
add a
>>> test in
>>> >> >> >> the tck and implements it for all providers :-)
>>> >> >> >> Maybe having 2 methods:
>>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value)
( maybe
>>> a Map
>>> >> >> >> for multiple value pair? will you test the full value
or a
>>> matching?)
>>> >> >> >>
>>> >> >> >
>>> >> >> > is the matching syntax for cassandra an jcr different?
>>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex-
to do
>>> >> matching
>>> >> >> is
>>> >> >> > it going to work only in jcr?
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >> * pom contents: getArtifactsByProperty( key, value
) ( same
>>> >> questions )
>>> >> >> >>
>>> >> >> >> To start I won't go with a Map and exact matching
:-) (I reckon
>>> pom
>>> >> >> >> content can be a pain with Cassandra impl)
>>> >> >> >>
>>> >> >> >> WDYT?
>>> >> >> >>
>>> >> >> >> Olivier
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <carlos@apache.org>
>>> wrote:
>>> >> >> >> > Thanks for the pointers, so if I add a method
to
>>> >> JcrMetadataRepository
>>> >> >> >> >
>>> >> >> >> > getArtifactsByProperty(key, value)
>>> >> >> >> >
>>> >> >> >> > similar to getArtifactsByDateRange, writing the
correct JCR
>>> query,
>>> >> I
>>> >> >> >> could
>>> >> >> >> > access all that metadata?
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > In which of the rest services would it make sense
to add such
>>> a
>>> >> >> search?
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy
<
>>> olamy@apache.org>
>>> >> >> wrote:
>>> >> >> >> >
>>> >> >> >> >> Hi,
>>> >> >> >> >> The current SearchService search in maven
indexer available
>>> datas.
>>> >> >> >> >> All the data you want to search are not available
here but
>>> in the
>>> >> >> >> >> MetadataRepository ( so jcr, file or the
new Cassandra).
>>> >> >> >> >> And currently there is no search methods
for that!
>>> >> >> >> >> So really good news to see you as a volunteer
to implement
>>> this!
>>> >> :-)
>>> >> >> >> >> Let me know if you need any pointers.
>>> >> >> >> >>
>>> >> >> >> >> Cheers
>>> >> >> >> >> Olivier
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez
<carlos@apache.org>
>>> >> wrote:
>>> >> >> >> >> > Hi,
>>> >> >> >> >> >
>>> >> >> >> >> > Two things I wasn't able to do and was
going to implement
>>> >> (unless I
>>> >> >> >> >> missed
>>> >> >> >> >> > something):
>>> >> >> >> >> >
>>> >> >> >> >> > * search for generic metadata
>>> >> >> >> >> > * search for pom contents
>>> >> >> >> >> >
>>> >> >> >> >> > The way the search queries are built
in SearchService only
>>> looks
>>> >> >> for
>>> >> >> >> the
>>> >> >> >> >> > typical fields (artifactId, groupId,...),
so that needs to
>>> be
>>> >> >> changed,
>>> >> >> >> >> but
>>> >> >> >> >> > I don't know if the pom and generic
metadata fields are
>>> even
>>> >> >> indexed,
>>> >> >> >> any
>>> >> >> >> >> > pointers?
>>> >> >> >> >> >
>>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue"
and
>>> >> something
>>> >> >> >> like
>>> >> >> >> >> > "developer:carlos*" through the rest
api and ui.
>>> >> >> >> >> >
>>> >> >> >> >> > Thanks
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> --
>>> >> >> >> >> Olivier Lamy
>>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Olivier Lamy
>>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Olivier Lamy
>>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Olivier Lamy
>>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >>
>>>
>>>
>>>
>>> --
>>> Olivier Lamy
>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message