Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Message-Id: <519743$2mk2ot@ironport10.mayo.edu>
Date: Tue, 22 Mar 2016 21:51:54 +0000
From: "Bauer, Herbert S. (Scott)" <Bauer.Scott@mayo.edu>
Subject: Re: Serializing Queries
In-reply-to: 
 <BY2PR0201MB0837F467F88AC9E888CDF5F4958C0@BY2PR0201MB0837.namprd02.prod.outlook.com>
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
Content-id: <4DEF702B20E21A4885D015CA2A8F6758@exchange.mayo.edu>
MIME-version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-language: en-US
Content-transfer-encoding: quoted-printable
Accept-Language: en-US
Thread-topic: Serializing Queries
Thread-index: AQHRgR7B4MzlSdfgbUaHVq7HuV5LCZ9fT3lfgAa4XQA=
References: <519743$2lopq3@ironport10.mayo.edu>
 <BY2PR0201MB0837F467F88AC9E888CDF5F4958C0@BY2PR0201MB0837.namprd02.prod.outlook.com>
User-Agent: Microsoft-MacOutlook/14.6.2.160219

This seems to work for me.  Unfortunately I had to stick with the 2.2.1
version myself.  Thanks to our use of an old version of spring I had some
asm version clashes.  I did also make use of the custom serializers found
here:
https://github.com/magro/kryo-serializers.git

writing:

ByteArrayOutputStream baos =3D new ByteArrayOutputStream();
        Output output =3D new Output(baos);
        Kryo kryo =3D new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
        UnmodifiableCollectionsSerializer.registerSerializers(kryo);
        kryo.register(Arrays.asList("").getClass(), new
ArraysAsListSerializer());
        kryo.register(BooleanClause.class);
        kryo.register(Query.class);
        kryo.register(Occur.class);
        kryo.register(ToParentBlockJoinQuery.class);
        kryo.register(QueryBitSetProducer.class);
        SynchronizedCollectionsSerializer.registerSerializers(kryo);
        kryo.writeClassAndObject(output, (List<BooleanClause>)
builder.build().clauses());

        output.close();
        String outputString =3D
Base64.encodeBase64String(baos.toByteArray());


Reading:

String inputString =3D (String) in.readObject();
        ByteArrayInputStream bais =3D new
ByteArrayInputStream(Base64.decodeBase64(inputString));
        Input input =3D new Input(bais);
        Kryo kryo =3D new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
        UnmodifiableCollectionsSerializer.registerSerializers(kryo);
        kryo.register(Arrays.asList("").getClass(), new
ArraysAsListSerializer());
        kryo.register(BooleanClause.class);
        kryo.register(Query.class);
        kryo.register(Occur.class);
        kryo.register(ToParentBlockJoinQuery.class);
        kryo.register(QueryBitSetProducer.class);
        SynchronizedCollectionsSerializer.registerSerializers(kryo);
        @SuppressWarnings("unchecked")
        List<BooleanClause> queryObject =3D (List<BooleanClause>)
kryo.readClassAndObject(input);
        input.close();
        builder =3D new BooleanQuery.Builder();
        for (BooleanClause clause : queryObject) {
            builder.add(clause);
        }


Base64 didn=B9t line up perfectly either, but I found an apache commons
version that worked.  Not sure all the class registration was necessary.

I=B9m really grateful for the code sample and direction.  This seems to be =
a
very effective tool for solving a lot of serialization problems that could
have been tough to solve.


On 3/18/16, 10:44 AM, "McKinley, James T" <james.mckinley@cengage.com>
wrote:

>We use Kryo to pass query objects between hosts:
>
>https://github.com/EsotericSoftware/kryo
>
>We initially had some trouble with it creating dynamic classes and
>running out of PermGen space but we got around that using an ObjectPool:
>
>http://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/p
>ool/impl/StackObjectPool.html
>
>I've not looked at the project recently, we're using version 2.21 and
>there's a 3.0.0 now, so they may have solved the issues we had and made
>things nicer, but here's how we're doing it with 2.21:
>
>To serialize:
>
>static private ObjectPool<Kryo> pool =3D new StackObjectPool<Kryo>(new
>KryoFactory(), 75, 75);
>
>ByteArrayOutputStream baos =3D new ByteArrayOutputStream();
>Output output =3D new Output(baos);
>kryo =3D pool.borrowObject();
>kryo.writeClassAndObject(output, query);
>pool.returnObject(kryo);
>output.close();
>String base64EncodedSerializedObject =3D
>Base64.encodeBytes(baos.toByteArray());
>
>Where query is a Lucene Query object (I've left out the error handling
>for brevity).
>
>To deserialize:
>
>ByteArrayInputStream bais =3D new
>ByteArrayInputStream(Base64.decode(encodedQuery));
>Input input =3D new Input(bais);
>kryo =3D pool.borrowObject();
>deserializedQueryObject =3D (Query) kryo.readClassAndObject(input);
>pool.returnObject(kryo);
>input.close();
>
>Hope that might help.
>
>Jim
>
>________________________________________
>From: Bauer, Herbert S. (Scott) <Bauer.Scott@mayo.edu>
>Sent: 18 March 2016 10:02
>To: java-user@lucene.apache.org
>Subject: Serializing Queries
>
>Has anyone in this group solved the problem of serializing complex
>boolean queries (Some of our clauses have span and other query types)?
>Our Java RMI depends upon being able to do so.   I have seen posts that
>say you can just parse the string representation but apparently that only
>works on simple query representations.     I=B9m looking at the CoreParser
>and it=B9s supporting xml parsing capabilities with an eye toward
>Marshalling the boolean query into a DOM object and unmarshalling it on
>the server side using some of the support implied by the CoreParser and
>related classes.  -scott
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org