hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mithun Radhakrishnan (JIRA)" <>
Subject [jira] [Commented] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
Date Mon, 09 Feb 2015 21:03:36 GMT


Mithun Radhakrishnan commented on HIVE-9609:

Hey, Sush. I heartily agree with the intention. It would be good to remove cases where all
the partitions need to be loaded into memory at once. And for this, it makes sense to have
AddPartitionEvent initializable only with an Iterator<Partition>. 

I like the idea of the Iterable<Partition>. Perhaps the PartitionSpecProxy should have
implemented Iterable<Partition>. :/

I did run into a problem (that I can't completely recollect). You'll notice that {{PartitionSpecProxy.PartitionIterator}}
has functionality that ideally belongs in {{Partition}}, such as {{setCreateTime()}}, {{putToParameters()}},
etc. The reason is that subclasses (such as {{PartitionSpecWithSharedSDProxy}}) construct
Partition instances as needed, and won't be able to propagate setter-actions on Partitions
to the underlying implementation. (At least, not without subclassing {{Partition}} itself.)

I'm fine with changing the MessageFactory interface, and AddPartitionEvent initializers, to
eschew {{List<Partition>}} objects. We'll also need to change {{HiveMetaStore.fireMetaStoreAddPartitionEvent()}}
to conform to this.

> AddPartitionMessage.getPartitions() can return null
> ---------------------------------------------------
>                 Key: HIVE-9609
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.0
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-9609.patch
> DbNotificationListener and NotificationListener both depend on AddPartitionEvent.getPartitions()
to get their partitions to trigger a message, but this can be null if an AddPartitionEvent
was initialized on a PartitionSpec rather than a List<Partition>.
> Also, AddPartitionEvent seems to have a duality, where getPartitions() works only if
instantiated on a List<Partition>, and getPartitionIterator() works only if instantiated
on a PartitionSpec.

This message was sent by Atlassian JIRA

View raw message