From solr-user-return-142176-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Thu Jun 28 19:41:54 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id A944A180662 for ; Thu, 28 Jun 2018 19:41:53 +0200 (CEST) Received: (qmail 46006 invoked by uid 500); 28 Jun 2018 17:41:51 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 45994 invoked by uid 99); 28 Jun 2018 17:41:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jun 2018 17:41:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C2F76C765B for ; Thu, 28 Jun 2018 17:41:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.051 X-Spam-Level: X-Spam-Status: No, score=-0.051 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, KAM_LOTSOFHASH=0.25, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id qz9k3C6qpZv5 for ; Thu, 28 Jun 2018 17:41:49 +0000 (UTC) Received: from us-smtp-delivery-180.mimecast.com (us-smtp-delivery-180.mimecast.com [63.128.21.180]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C35905F23C for ; Thu, 28 Jun 2018 17:41:48 +0000 (UTC) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03lp0016.outbound.protection.outlook.com [207.46.163.16]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id us-mta-276-7-E5l3tRMGyQa9BzC-2chQ-1; Thu, 28 Jun 2018 13:41:40 -0400 Received: from MWHPR03MB3039.namprd03.prod.outlook.com (10.175.137.16) by MWHPR03MB2703.namprd03.prod.outlook.com (10.168.207.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.906.20; Thu, 28 Jun 2018 17:41:39 +0000 Received: from MWHPR03MB3039.namprd03.prod.outlook.com ([fe80::a589:5d42:a1a4:573d]) by MWHPR03MB3039.namprd03.prod.outlook.com ([fe80::a589:5d42:a1a4:573d%6]) with mapi id 15.20.0884.024; Thu, 28 Jun 2018 17:41:39 +0000 From: Harsh Verma To: "solr-user@lucene.apache.org" Subject: Custom FieldType not getting populated in the response Thread-Topic: Custom FieldType not getting populated in the response Thread-Index: AdQPB0NsyiigFkp+S+qElcMazWxybQ== Date: Thu, 28 Jun 2018 17:41:38 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [69.25.143.34] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;MWHPR03MB2703;7:bHj+ZseJFik52+2y7ZXFCSAvyqSBNauKtv+34cmiHuHhGU2RcG0x7kikM1JLLfnJ8AQ1UxZL1Iaasimhkj6SE0vhBrWf1JgSdB8PTz6ZrzcmJK72UW9Mymo1l5iY1asd5KfhuIz811NYvpcktYcj7uXa+PB7RL3PgP4FNarzPZq6x67Hu+suz5SDIHXzqm7jg6WteoPAUOSoZbhZk0AZkHuxveZ23US2hIY5ONdFB71DbgxCxOmttqWcfA+48INw;20:GQQbpHxFk9Y0R6FRojuuPYGKVYLQp6xkP+pBFIu1QtF8WAtiHwelthN/zG6GOeNhFvDFZi4tW2cJaJgn2OjDjPn06Dgkr/2UtWq0k02iPxTSKbk2lyv7mlbziHUjwjOjnbM6P6GNq0Ke2TlElmNr9Tak5uPOnJ+Lu+YDwDdfilM= x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: cbf93956-9745-40a2-1f2a-08d5dd1e6710 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600026)(711020)(2017052603328)(7153060)(7193020);SRVR:MWHPR03MB2703; x-ms-traffictypediagnostic: MWHPR03MB2703: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(68173958961439)(166708455590820)(21748063052155)(192278398808882); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3231254)(944501410)(52105095)(3002001)(149027)(150027)(6041310)(20161123564045)(20161123558120)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016);SRVR:MWHPR03MB2703;BCL:0;PCL:0;RULEID:;SRVR:MWHPR03MB2703; x-forefront-prvs: 0717E25089 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(136003)(376002)(366004)(346002)(39860400002)(69234005)(53754006)(54094003)(189003)(199004)(6916009)(6436002)(9326002)(790700001)(7696005)(575784001)(478600001)(3846002)(6116002)(8936002)(5660300001)(86362001)(966005)(5640700003)(2351001)(97736004)(25786009)(606006)(6306002)(66066001)(9686003)(68736007)(106356001)(105586002)(14454004)(2501003)(5250100002)(53936002)(74316002)(54896002)(7736002)(26005)(476003)(236005)(186003)(14444005)(256004)(2906002)(33656002)(99286004)(55016002)(486006)(81166006)(316002)(6506007)(102836004)(2900100001)(81156014)(8676002);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR03MB2703;H:MWHPR03MB3039.namprd03.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; x-microsoft-antispam-message-info: 8RKa+p+aJQwiDcOg5QzdefJrQQb7Gc80aQTee0HpdNRLXvtBUX3hXBrz1uFZAlDJ6lrlvQR/o4U+5U3fbRYBgPfBtzucIazuuhzpLkD9qNwSPN69R65tI4nAN7V/Yxc87HChUDHEpFYeZRhKUZQG/acJ7OZ/ZbwogQ2S3vQ7h9nVAAxnzlcDdEb49uezTlkkbmaZCaP3K16iPaC1lu2cAXwlxzlysEvfceKFE17nlqgTHOkYMZtW/yGE2FjRycwxj4puvRjre5JaGrPJbt//4btZnTFONaMN0xnk6ganwWepsQ+4am8hFA8ZrK7z0/A4BeazInOTpzHlNhaPlhFJWZDM7bZeiv6qX5Earwfv6Co= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: tableau.com X-MS-Exchange-CrossTenant-Network-Message-Id: cbf93956-9745-40a2-1f2a-08d5dd1e6710 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2018 17:41:38.8818 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5067385a-c584-42cc-8679-3fa2879f99be X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2703 X-MC-Unique: 7-E5l3tRMGyQa9BzC-2chQ-1 Content-Type: multipart/alternative; boundary="_000_MWHPR03MB30398C6A08F57DE4653ABFB9DA4F0MWHPR03MB3039namp_" --_000_MWHPR03MB30398C6A08F57DE4653ABFB9DA4F0MWHPR03MB3039namp_ Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Hi All, I am trying to fetch values (docValues to be exact) from external redis dur= ing query time using a custom written FieldType which I am loading in my so= lr instance. I notice that my custom FieldType is being initialized based o= n the schema when loading the core. I also notice that during query time, m= y custom field type code is called but the results back from solr do not po= pulate the custom field even if I explicitly request it. Also, I am asking = the results to be sorted by the custom field but results are not sorted. Solr version: 7.3.1 Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c= 6d9b3b6cc7eaaf0b49c Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242f= d7ba3d8d3a3bda209ac3 Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802= a6bbcdde9215a2fdaf5 Query that I am making: /solr/redis/select?fl=3Dhits&q=3D*:*&sort=3Dhits%20= asc Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c= 4c35a9f85016fe30c0 Another query with debug flag: solr/redis/select?debugQuery=3Don&fl=3D*,hit= s&q=3D*:*&sort=3Dhits%20desc Response: https://gist.github.com/vermaharsh/f7f74a65a5403ecec9310ceb81cb67= 4c Sort order should be - 10, 9, 8, 6, 5, 4, 3, 2, 1, 7; because, correspondin= g hits value from redis are - 11111, 900, 800, 600, 500, 400, 300, 200, 100= , 7. Can someone help me identify what am I missing? Thanks, Harsh Copy of discussion on #solr irc channel. I am trying to write a custom solr FieldType which fetches value fr= om external redis instance. I got Solr to load my custom jars and have defi= ned the schema with custom fieldType class. But at query time, I cannot get= values in the response. [08:38] I do not see any errors in the logs as well, so cannot tell= if something failed [08:38] here is the code for my custom FieldType - https://gist.git= hub.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c [08:39] this is my solrconfig.xml for my custom configset - https:/= /gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 [08:42] I basically placed the necessary jars under contrib/redis-f= ield [08:43] and my managed-schema - https://gist.github.com/vermaharsh/= 8a89195377802a6bbcdde9215a2fdaf5 [08:43] I am using solr version 7.3.1 [08:44] any idea why I am not getting the value for my custom field= type back? [08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FY= I, this should go in #solr -- this channel is for development of Solr itsel= f. [08:46] <@elyograg> the field is not marked as stored. I wonder if that mi= ght tell Solr that it shouldn't be returned at all. (I don't know whether = setting stored=3Dtrue might require something more in your code, though) [08:46] I am using it as docValue, but I can try stored. Though, as= you mentioned, not sure if that would need more to be implemented in the c= ode [08:47] I will try #solr channel for the question as well [08:47] <@elyograg> ah, I didn't scroll to the right enough to see that par= t. :) [08:48] <@elyograg> I wonder if you might need useDocValuesAsStored=3D"true= ". [08:49] I thought that is the default value [08:51] for completeness, this is the query that I am using - /solr= /redis/select?fl=3Dhits&q=3D*:*&sort=3Dhits%20asc [08:52] and response that I got back - https://gist.github.com/verm= aharsh/2f63282b10320c4c35a9f85016fe30c0 [08:53] =3D=3D dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has = joined #solr-dev [08:53] <@elyograg> this one's probably going to need to go to the mailing = list. with all the pastes you've mentioned here. [08:57] =3D=3D dataminion [~leahy@c-69-181-118-61.hsd1.ca.comcast.net] has = quit [Ping timeout: 264 seconds] [08:59] <@elyograg> you may be right about that being default. The code in= FieldType.java seems to support that. [09:00] <@elyograg> if (schemaVersion >=3D 1.6f) properties |=3D USE_DO= CVALUES_AS_STORED; [09:02] alright, thanks elyograg for looking into it. I will send t= his to mailing list as well. --_000_MWHPR03MB30398C6A08F57DE4653ABFB9DA4F0MWHPR03MB3039namp_--