gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Furkan KAMACI (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-431) reuseObjects is Not Used
Date Thu, 03 Sep 2015 09:26:45 GMT

    [ https://issues.apache.org/jira/browse/GORA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728742#comment-14728742

Furkan KAMACI commented on GORA-431:

These are from latest e-mails at mail list:

bq. Issue for general serialization mechanism for Map Reduce: https://issues.apache.org/jira/browse/HADOOP-1986
and here is the main usage of that parameter: https://avro.apache.org/docs/1.7.6/api/java/org/apache/avro/generic/GenericDatumReader.html#read(D,
org.apache.avro.io.Decoder) [~renato2099], I've checked Hadoop source codes and I see that
we have to find an appropriate way to pass that property. We do not instantiate that class
at Gora. What do you suggest?

bq. Thanks for taking the time and looking into this. I agree with you, there is something
fuzzy going on there. So our persistent classes extent org.apache.avro.specific.SpecificRecord
and use their specific DatumReader/Writer which in turn use the the methods that you are pointing
out. And yes, we should find a way to pass this into the datastores (which shouldn't be too
hard) but we should also double check if it makes sense to allow the user to set it or not.
I mean is it always better to have it set as <true> (smaller memory footprint)? or is
there any case when using the object between serialization that could cause wrong behaviour
if we were reusing the object? I don't know the actual effect on setting this to true or false.
Could you please create a simple test where we could see the different behaviour of setting
this to true or false? That would be great as it'd help everybody understand better what was
the reasoning for setting this parameter always true.

[~lewismc] I should create a test case for it and according to that test we can decide whether
to remove it or add as a config.

> reuseObjects is Not Used
> ------------------------
>                 Key: GORA-431
>                 URL: https://issues.apache.org/jira/browse/GORA-431
>             Project: Apache Gora
>          Issue Type: Bug
>          Components: gora-core
>            Reporter: Furkan KAMACI
>            Priority: Minor
>             Fix For: 0.6.1
> here is a method as:
> {code}
>     public static void setIOSerializations(Configuration conf, boolean reuseObjects)
> {code}
> at GoraMapReduceUtils.java
> However, reuseObjects is never used at that method. On the other hand it is always true
at PersistentDeserializer.deserialize()

This message was sent by Atlassian JIRA

View raw message