hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaimin Shah <shahjaimin0...@gmail.com>
Subject Re: Possible ambiguity in HoodieKey
Date Tue, 11 Jun 2019 16:19:50 GMT
Sure I  will update once I make those changes thanks.

On Tuesday, 11 June 2019, Vinoth Chandar <vinoth@apache.org> wrote:

> Thanks for the link. I was grabbing in parallel as well :)
>
> So, the KeyGenerator class works off a GenericRecord and JSON->
> GenericRecord is already built in to the DeltaStreamer.
> I don't think this will add any particular performance overhead per se.
>
> It may be worth pulling this class (+ tweaked to suit your needs) out of
> the PR and merge it into master?
>
>
>
> On Tue, Jun 11, 2019 at 9:11 AM Jaimin Shah <shahjaimin0395@gmail.com>
> wrote:
>
> > Forgot to add class link
> >
> > https://github.com/apache/incubator-hudi/blob/
> e916b21cc5989ab00791467fcc11a02bb0de093a/hoodie-bench/src/
> main/java/com/uber/hoodie/integrationsuite/generator/
> ComplexKeyGenerator.java
> > This is the class I am referring to.
> >
> > On Tuesday, 11 June 2019, Jaimin Shah <shahjaimin0395@gmail.com> wrote:
> >
> > > Hi Vinoth,
> > >
> > >   Thanks for the prompt reply. This class was shared earlier on the
> > > mailing list by someone to handle complex key. I was thinking maybe we
> > can
> > > create a Jason object and then parse it as string to create key then it
> > > will be full proof because we don’t control the characters in the input
> > > data.
> > >
> > >   I am not sure about the performance implications of doing so maybe
> you
> > > can help there.
> > >
> > > Thanks,
> > > Jaimin
> > >
> > > On Tuesday, 11 June 2019, Vinoth Chandar <vinoth@apache.org> wrote:
> > >
> > >> Hi Jaimin,
> > >>
> > >> True. Is this a custom class you have? if we separate the
> concatenation
> > by
> > >> a standard special character, it should be fine?  for e.g  CA#US,
> C#AUS
> > ?
> > >>
> > >> Thanks
> > >> Vinoth
> > >>
> > >> On Mon, Jun 10, 2019 at 4:53 AM Jaimin Shah <shahjaimin0395@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi
> > >> >   I was going through the ComplexKeyGenerator class. I found that
> the
> > >> class
> > >> > generates key by concatenating the all keys to make compound key.
> But
> > I
> > >> am
> > >> > wondering that some cases can arise later which can create problems.
> > >> >
> > >> > For example our data has 2 attributes as key
> > >> > key1 key2  data
> > >> > CA   US       xyz
> > >> > C    AUS       abc
> > >> >
> > >> > In this case key for both rows will be same will it cause any
> problem?
> > >> > Instead of keeping keys as string keeping them as map will solve the
> > >> > problem?
> > >> >
> > >> > Thanks
> > >> >
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message