flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasiliki Kalavri <vasilikikala...@gmail.com>
Subject Re: On (FLINK-1526) JIRA issue
Date Thu, 22 Sep 2016 09:40:48 GMT
Hi Olga,

when you use mapEdges() or mapVertices() with generics, Flink cannot
determine the type because of type erasure, like the exception says. That's
why we also provide methods that take the type information as a parameter.
You can use those to make the return type explicit. In your example, you
should do something like the following (line 41):

final TypeInformation<Long> longType = BasicTypeInfo.LONG_TYPE_INFO;
final TypeInformation<Double> doubleType = BasicTypeInfo.DOUBLE_TYPE_INFO;
Graph<Long, String, Tuple3<Double, Long, Long>> graphOut =
    graph.mapEdges(new InitializeEdges<Long, Double>(), new
TupleTypeInfo(longType, longType,
        new TupleTypeInfo<Tuple3<Double, Long, Long>>(doubleType,
longType,longType)));

Regarding the nested loops, I am almost sure that you will face problems if
you try to experiment with large datasets. I haven't looked into your code
yet, but according to the JIRA discussion, we've faced this problem before
and afaik, this is still an issue.

Cheers,
-Vasia.

On 22 September 2016 at 01:12, Olga Golovneva <melchaeva@gmail.com> wrote:

> Hi Vasia,
>
> I have uploaded these tests on github:
> https://github.com/OlgaGolovneva/MST/tree/master/tests
>
> I have also uploaded source code, but I'm still working on it:
> https://github.com/OlgaGolovneva/MST/tree/master/src
>
> ​>I think you cannot add attachments to the mailing list. Could you upload
> >your example somewhere and post a link here? I'm actually surprised that
> >the while-loop works without problems.
>
> I have run the program on several simple tests, and I was going to try
> large datasets in the next few days. Please, let me know if this approach
> is wrong.
>
> Thanks,
> Olga
>
> On Wed, Sep 21, 2016 at 4:55 PM, Vasiliki Kalavri <
> vasilikikalavri@gmail.com
> > wrote:
>
> > Hi Olga,
> >
> > On 21 September 2016 at 18:50, Olga Golovneva <melchaeva@gmail.com>
> wrote:
> >
> > > Hi devs,
> > >
> > > I was working on  (FLINK-1526) "Add Minimum Spanning Tree library
> method
> > > and example" issue. I've developed (Java) code that implements
> > distributed
> > > Boruvka's algorithm in Gelly library. I've run several tests and it
> seems
> > > to work fine, although I didn't test it on extremely large input graphs
> > > yet, and I'm also trying to optimize my code.
> > > Particularly, I have two main issues:
> > >
> > > 1. Nested loops.
> > > I have to use nested loops, and I do not see the way to avoid them. As
> > > they are currently not supported, I'm using Bulk Iterations inside a
> > > "classic" while loop. I've included in attachment simple example
> > > MyNestedIterationExample that shows this issue.
> > >
> >
> > ​I think you cannot add attachments to the mailing list. Could you upload
> > your example somewhere and post a link here? I'm actually surprised that
> > the while-loop works without problems.
> >
> >
> > >
> > > 2. For some reason I cannot create class that works with types with
> > > generic variables in Tuple2(or Tuple3), thus my code does not support
> > > generic types. I also included simple example MyTuple3Example. Here is
> > the
> > > Exception I get:
> > > "Exception in thread "main" org.apache.flink.api.common.functions.
> > InvalidTypesException:
> > > Type of TypeVariable 'EV' in 'class org.apache.flink.graph.
> > > examples.MyTuple3Example$InitializeEdges' could not be determined.
> This
> > > is most likely a type erasure problem. The type extraction currently
> > > supports types with generic variables only in cases where all variables
> > in
> > > the return type can be deduced from the input type(s)."
> > >
> >
> > ​Can you upload this example and link to it too?
> >
> > Thanks,
> > -Vasia.
> >
> >
> > >
> > > I would really appreciate if someone could explain me know how to avoid
> > > this Exception. Otherwise, I could submit my code for testing.
> > >
> > > Best regards,
> > > Olga Golovneva
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message