lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason <hialo...@gmail.com>
Subject ValueSource with long BinaryField
Date Fri, 08 Feb 2019 06:28:05 GMT
Hi,
I'm suffering from very long search response.
Specifically, sort by ValueSource.
I stored 4,096 length double array using BinaryField per a doc.
In searching, I searched with sort function which is calculate a distance
between two double array in custom defined ValueSource.
But response time is too long.
I think that both reading binary values and decoding base64 are too long.
Is there a good solution to improve response time?

Custom ValueSource snippet is below.

public class VectorValueSource extends ValueSource {

    private final java.util.Base64.Decoder decoder =
java.util.Base64.getDecoder();

    public double distance(double[] qVector, double[] dVector) {
        double result = 0;
        for (int i = 0; i < qVector.length; i++) {
          double v = qVector[i] - dVector[i];
          result += v * v;
        }
        return Math.sqrt(result);
    }

    @Override
    public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
        final FieldInfo fieldInfo =
readerContext.reader().getFieldInfos().fieldInfo(field);
        
        if (fieldInfo != null && fieldInfo.getDocValuesType() ==
DocValuesType.BINARY) {
            final BinaryDocValues binaryValues =
DocValues.getBinary(readerContext.reader(), field);
            final Bits docsWithField =
DocValues.getDocsWithField(readerContext.reader(), field);
            
            return new FunctionValues() {

                @Override
                public double doubleVal(int doc) {
                    BytesRef target = null;
                    if (binaryValues.get(doc).length > 0) {
                        target = binaryValues.get(doc);
                       
VectorUtils.byte2Double(decoder.decode(target.bytes), dFeature, false);
                        return distance(qFeature, dFeature);
                    }
                    else {
                        return maxDistance;
                    }
                }
            };
        }
    }
}



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Mime
View raw message