lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bauer, Herbert S. (Scott)" <Bauer.Sc...@mayo.edu>
Subject Re: Serializing Queries
Date Tue, 22 Mar 2016 21:51:54 GMT
This seems to work for me.  Unfortunately I had to stick with the 2.2.1
version myself.  Thanks to our use of an old version of spring I had some
asm version clashes.  I did also make use of the custom serializers found
here:
https://github.com/magro/kryo-serializers.git

writing:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
        Output output = new Output(baos);
        Kryo kryo = new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
        UnmodifiableCollectionsSerializer.registerSerializers(kryo);
        kryo.register(Arrays.asList("").getClass(), new
ArraysAsListSerializer());
        kryo.register(BooleanClause.class);
        kryo.register(Query.class);
        kryo.register(Occur.class);
        kryo.register(ToParentBlockJoinQuery.class);
        kryo.register(QueryBitSetProducer.class);
        SynchronizedCollectionsSerializer.registerSerializers(kryo);
        kryo.writeClassAndObject(output, (List<BooleanClause>)
builder.build().clauses());

        output.close();
        String outputString =
Base64.encodeBase64String(baos.toByteArray());


Reading:

String inputString = (String) in.readObject();
        ByteArrayInputStream bais = new
ByteArrayInputStream(Base64.decodeBase64(inputString));
        Input input = new Input(bais);
        Kryo kryo = new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
        UnmodifiableCollectionsSerializer.registerSerializers(kryo);
        kryo.register(Arrays.asList("").getClass(), new
ArraysAsListSerializer());
        kryo.register(BooleanClause.class);
        kryo.register(Query.class);
        kryo.register(Occur.class);
        kryo.register(ToParentBlockJoinQuery.class);
        kryo.register(QueryBitSetProducer.class);
        SynchronizedCollectionsSerializer.registerSerializers(kryo);
        @SuppressWarnings("unchecked")
        List<BooleanClause> queryObject = (List<BooleanClause>)
kryo.readClassAndObject(input);
        input.close();
        builder = new BooleanQuery.Builder();
        for (BooleanClause clause : queryObject) {
            builder.add(clause);
        }


Base64 didn¹t line up perfectly either, but I found an apache commons
version that worked.  Not sure all the class registration was necessary.

I¹m really grateful for the code sample and direction.  This seems to be a
very effective tool for solving a lot of serialization problems that could
have been tough to solve.



On 3/18/16, 10:44 AM, "McKinley, James T" <james.mckinley@cengage.com>
wrote:

>We use Kryo to pass query objects between hosts:
>
>https://github.com/EsotericSoftware/kryo
>
>We initially had some trouble with it creating dynamic classes and
>running out of PermGen space but we got around that using an ObjectPool:
>
>http://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/p
>ool/impl/StackObjectPool.html
>
>I've not looked at the project recently, we're using version 2.21 and
>there's a 3.0.0 now, so they may have solved the issues we had and made
>things nicer, but here's how we're doing it with 2.21:
>
>To serialize:
>
>static private ObjectPool<Kryo> pool = new StackObjectPool<Kryo>(new
>KryoFactory(), 75, 75);
>
>ByteArrayOutputStream baos = new ByteArrayOutputStream();
>Output output = new Output(baos);
>kryo = pool.borrowObject();
>kryo.writeClassAndObject(output, query);
>pool.returnObject(kryo);
>output.close();
>String base64EncodedSerializedObject =
>Base64.encodeBytes(baos.toByteArray());
>
>Where query is a Lucene Query object (I've left out the error handling
>for brevity).
>
>To deserialize:
>
>ByteArrayInputStream bais = new
>ByteArrayInputStream(Base64.decode(encodedQuery));
>Input input = new Input(bais);
>kryo = pool.borrowObject();
>deserializedQueryObject = (Query) kryo.readClassAndObject(input);
>pool.returnObject(kryo);
>input.close();
>
>Hope that might help.
>
>Jim
>
>________________________________________
>From: Bauer, Herbert S. (Scott) <Bauer.Scott@mayo.edu>
>Sent: 18 March 2016 10:02
>To: java-user@lucene.apache.org
>Subject: Serializing Queries
>
>Has anyone in this group solved the problem of serializing complex
>boolean queries (Some of our clauses have span and other query types)?
>Our Java RMI depends upon being able to do so.   I have seen posts that
>say you can just parse the string representation but apparently that only
>works on simple query representations.     I¹m looking at the CoreParser
>and it¹s supporting xml parsing capabilities with an eye toward
>Marshalling the boolean query into a DOM object and unmarshalling it on
>the server side using some of the support implied by the CoreParser and
>related classes.  -scott
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message