apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-1984) Operators that use Kryo directly would throw exception in local mode
Date Wed, 27 Jan 2016 17:40:39 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119859#comment-15119859
] 

ASF GitHub Bot commented on APEXMALHAR-1984:
--------------------------------------------

Github user siyuanh commented on a diff in the pull request:

    https://github.com/apache/incubator-apex-malhar/pull/178#discussion_r51019702
  
    --- Diff: contrib/src/main/java/com/datatorrent/contrib/couchbase/AbstractCouchBaseInputOperator.java
---
    @@ -149,15 +145,8 @@ public void partitioned(Map<Integer, Partition<AbstractCouchBaseInputOperator<T>
         int numPartitions = conf.getServers().size();
         List<String> list = conf.getServers();
         Collection<Partition<AbstractCouchBaseInputOperator<T>>> newPartitions
= Lists.newArrayListWithExpectedSize(numPartitions);
    -    Kryo kryo = new Kryo();
         for (int i = 0; i < numPartitions; i++) {
    -      ByteArrayOutputStream bos = new ByteArrayOutputStream();
    -      Output output = new Output(bos);
    -      kryo.writeObject(output, this);
    -      output.close();
    -      Input lInput = new Input(bos.toByteArray());
    -      @SuppressWarnings("unchecked")
    -      AbstractCouchBaseInputOperator<T> oper = kryo.readObject(lInput, this.getClass());
    +      AbstractCouchBaseInputOperator<T> oper = PartitionerUtils.cloneOperatorByKryo(this);
    --- End diff --
    
    I've created a threadlocal pool for kryo objects. I think it's even better idea to create
kryo on the fly.


> Operators that use Kryo directly would throw exception in local mode
> --------------------------------------------------------------------
>
>                 Key: APEXMALHAR-1984
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-1984
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: Siyuan Hua
>            Assignee: Siyuan Hua
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message