flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hogan <c...@greghogan.com>
Subject Re: Problem with Summerization
Date Wed, 28 Jun 2017 14:05:26 GMT
The stacktrace indicates this may be a bug that was fixed for Flink 1.2:

    [FLINK-4624] [gelly] Support null values in Graph Summarization

    * Bug was caused by serializers that cannot handle null values (e.g.
Long)
    * VertexGroupItem now uses Either<NullValue, VV> instead of VV
    * Generalized test cases
    * Added tests for vertex/edge values of type Long
    * Replaced Guava Lists.newArrayList() with new ArrayList<>()


On Wed, Jun 28, 2017 at 2:22 AM, <rostami@informatik.uni-leipzig.de> wrote:

> Hi,
>
> these are the outputs for a small test graph:
>
> gellyGraph.getVertices().print();
> (000000000000000000000003,9)
> (000000000000000000000007,9)
> (000000000000000000000005,5)
> (000000000000000000000001,9)
> (000000000000000000000006,9)
> (000000000000000000000000,9)
> (000000000000000000000004,9)
> (000000000000000000000002,2)
> (000000000000000000000008,8)
> (000000000000000000000009,9)
> (00000000000000000000000a,10)
>
>
> gellyGraph.getEdges().print();
> (000000000000000000000002,000000000000000000000001,(null))
> (000000000000000000000006,000000000000000000000003,(null))
> (000000000000000000000000,000000000000000000000001,(null))
> (000000000000000000000008,000000000000000000000003,(null))
> (000000000000000000000005,000000000000000000000006,(null))
> (000000000000000000000009,000000000000000000000000,(null))
> (000000000000000000000009,00000000000000000000000a,(null))
> (000000000000000000000005,00000000000000000000000a,(null))
> (000000000000000000000009,000000000000000000000004,(null))
> (000000000000000000000005,000000000000000000000006,(null))
> (000000000000000000000005,000000000000000000000007,(null))
> (000000000000000000000009,000000000000000000000000,(null))
> (000000000000000000000006,000000000000000000000007,(null))
> (000000000000000000000004,000000000000000000000007,(null))
> (000000000000000000000008,000000000000000000000007,(null))
> (000000000000000000000002,000000000000000000000000,(null))
> (000000000000000000000009,000000000000000000000001,(null))
> (000000000000000000000005,000000000000000000000003,(null))
> (000000000000000000000008,000000000000000000000006,(null))
> (000000000000000000000000,000000000000000000000004,(null))
> (000000000000000000000002,000000000000000000000004,(null))
> (000000000000000000000007,000000000000000000000004,(null))
> (000000000000000000000007,000000000000000000000006,(null))
> (000000000000000000000004,000000000000000000000000,(null))
>
> g.getVertices().print(); //correct results
> (000000000000000000000005,(5,1))
> (000000000000000000000002,(2,1))
> (000000000000000000000008,(8,1))
> (000000000000000000000004,(9,7))
> (00000000000000000000000a,(10,1))
>
> g.getEdges.print(); //error!!
>
> As I said, I also tested when edges have values other than null
> and the same problem appears.
>
> Regards,
> Ali
>
>
> Quoting Greg Hogan <code@greghogan.com>:
>
> Hi Ali,
>>
>> Could you print and include a gellyGraph which results in this error.
>>
>> Greg
>>
>>
>> On Jun 27, 2017, at 2:48 PM, rostami@informatik.uni-leipzig.de wrote:
>>>
>>> Dear All,
>>>
>>> I do not understand what the error in the following code can be?
>>>
>>> Graph<GradoopId, Long, NullValue> gellyGraph = ...
>>>
>>> Graph<GradoopId,Summarization.VertexValue<Long>,
>>> Summarization.EdgeValue<NullValue>> g =
>>>                            gellyGraph.run(new Summarization<GradoopId,
>>> Long, NullValue>());
>>>
>>> g.getVertices().print(); //this one works fine
>>> g.getEdges().print();    //this one gives the following error
>>>
>>> ...
>>> org.apache.flink.types.NullFieldException: Field 2 is null, but
>>> expected to hold a value.
>>>         at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.
>>> serialize(TupleSerializer.java:126)
>>>         at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.
>>> serialize(TupleSerializer.java:30)
>>>         at org.apache.flink.runtime.plugable.SerializationDelegate.
>>> write(SerializationDelegate.java:56)
>>>         at org.apache.flink.runtime.io.network.api.serialization.Spanni
>>> ngRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
>>>         at org.apache.flink.runtime.io.network.api.writer.RecordWriter.
>>> emit(RecordWriter.java:85)
>>>         at org.apache.flink.runtime.operators.shipping.OutputCollector.
>>> collect(OutputCollector.java:65)
>>>         at org.apache.flink.runtime.operators.util.metrics.CountingColl
>>> ector.collect(CountingCollector.java:35)
>>>         at org.apache.flink.graph.library.Summarization$VertexGroupRedu
>>> cer.reduce(Summarization.java:318)
>>>         at org.apache.flink.runtime.operators.GroupReduceDriver.run(
>>> GroupReduceDriver.java:131)
>>>         at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.
>>> java:486)
>>>         at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTas
>>> k.java:351)
>>>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
>>>         at java.lang.Thread.run(Thread.java:748)
>>>
>>> I also tried with different types for EdgeValue. The same error appears.
>>> When I look at the result vertices, everything looks fine.
>>> But the result edges has this problem.
>>>
>>> Any idea?
>>>
>>> Regards,
>>> Ali
>>>
>>
>
>
>

Mime
View raw message