flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Compilation error with Scala case class with private constructor
Date Thu, 03 Mar 2016 21:19:41 GMT
Yes Stephan, you’re spot on. We generate code for serialization/deserialization of the types
and in there it creates code like this:

new YourCaseClass(field1, field2, …)

to create the value that is given to the user function.

If the constructor is private compilation of that generated code will fail and we don’t
get the TypeInformation.

I’m afraid there is no workaround right now, except for not making the constructor private.
> On 03 Mar 2016, at 21:17, Andrew Whitaker <andrew.whitaker@braintreepayments.com>
wrote:
> 
> Thanks for the quick response, Stephen, that makes sense.
> 
> > Is it a problem for your use case to make the constructor public?
> 
> It's not a huge problem, I was mostly just curious as to what the problem was exactly.
> 
> On Thu, Mar 3, 2016 at 2:14 PM, Stephan Ewen <sewen@apache.org> wrote:
> Hi!
> 
> My guess is that this error is indirectly reporting that no TypeInformation could be
generated for the case class. That TypeInformation is generated using macros during program
compilation.
> 
> The generated TypeInformation will contain a partially code-generated serializer including
the code to instantiate the case class upon deserialization. That generated code at some point
calls the constructor and thus probably fails to compile when the constructor is private.
As a result, no TypeInformation will be created, and then Scala cannot provide one for the
implicit parameter.
> 
> I think Aljoscha can probably give a deeper insight into the TypeInformation generator
for Scala.
> 
> Is it a problem for your use case to make the constructor public?
> 
> Greetings,
> Stephan
> 
> 
> On Thu, Mar 3, 2016 at 9:06 PM, Andrew Whitaker <andrew.whitaker@braintreepayments.com>
wrote:
> Hi,
> 
> I've run up against a compilation error involving a case class with a private constructor:
> 
> [error] /Users/anwhitaker/code/flink-fold-issue/src/main/scala/TestApp.scala:18: could
not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[List[scala.util.Try[TestApp.Wrapper]]]
> [error]       .fold(List[Try[Wrapper]](), new FoldFunction[Tuple2[Int, Int], List[Try[Wrapper]]]
{
> [error]            ^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> 
> If I make the constructor public again, the error goes away. I've set up a simple example
that demonstrates the problem here: https://github.com/AndrewWhitaker/flink-case-class-private-ctor
> I've read this article on Flink's website: https://flink.apache.org/faq.html#why-am-i-getting-a-nonserializableexception-
but I think my issue is slightly different.
> 
> I'm just trying to understand this behavior and if there's a way I can work around it.
> 
> Thanks!
> 
> -- 
> Andrew Whitaker | andrew.whitaker@braintreepayments.com
> 
> 
> 
> 
> -- 
> Andrew Whitaker | andrew.whitaker@braintreepayments.com
> --
> Note: this information is confidential. It is prohibited to share, post online or otherwise
publicize without Braintree's prior written consent.


Mime
View raw message