tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjin Lee <dongjin.lee...@gmail.com>
Subject Re: Supporting tablespace feature in HiveStore
Date Thu, 15 Sep 2016 16:35:06 GMT
Please forget my first mail. After that, I inspected this issue more
precisely and concluded that we don't have to fill up HiveCatalogStore's
tablespace methods[^1] - it is better to remove them.
Here is why.

Those methods were added to CatalogStore interface and its implementation
classes, AbstractDBStore and HiveCatalogStore in commit 3ba2624[^2]
and commit 8982684[^3]. But actually, those methods in AbstractDBStore are
not related to tablespace feature. Its CatalogConstants#TB_SPACES table,
which is used in accompany with CatalogConstants#TB_DATABASES table, stores
only default tablespace name and warehouse directory. If you are in doubt,
please check where CatalogStore#createTablespace is invoked - it is only
invoked in TajoMaster#checkBaseTBSpaceAndDatabase via CatalogService RPC.
Because tablespaces are defined using json configuration file, those
methods seem useless.

In my opinion, it is better to remove them from CatalogStore and their
implementation classes. In that case, the fact they are not implemented in
HiveCatalogStore[^4] never be a problem.

I am looking forward to your answer. Thanks.

Regards,
Dongjin

p.s. I checked for HiveCatalogStore's statistics feature and concluded that
it should be not so hard to resolve TAJO-2173[^5], even it is not perfect.

[^1]: createTablespace, existTablespace, dropTablespace,
getAllTablespaceNames, getTablespaces, getTablespace, alterTablespace.
[^2]: TAJO-353: Add Database support.
[^3]: TAJO-755: ALTER TABLESPACE LOCATION support.
[^4]: https://issues.apache.org/jira/browse/TAJO-2172
[^5]: https://issues.apache.org/jira/browse/TAJO-2173

On Mon, Sep 12, 2016 at 6:15 PM, Hyunsik Choi <hyunsik@apache.org> wrote:

> Hi,
>
> I'm sorry for late. I'm figuring out this problem. I'll give some comments
> soon. Then, let's talk about it more.
>
> Warm regards,
> Hyunsik
>
> On Tue, Aug 16, 2016 at 8:41 AM Dongjin Lee <dongjin@apache.org> wrote:
>
> > Hello. Currently, I am working to add some missing features in
> > HiveCatalogStore.[^1] However, I found that this feature needs the
> > implementation of tablespace feature in HiveCatalogStore first so I
> decided
> > that it would be better to ask you some advice before I start writing
> code
> > - I have less experience on Tajo than almost all of you.
> >
> > As of current version(11.x), tablespace feature seems to be not supported
> > in HiveCatalogStore. It only supports default tablespace in
> > ${tajo.warehouse.directory}. Because there is no other storage that can
> > save the mapping from ${tablespace-name} to {base-uri}, I propose the
> > following scheme:
> >
> > 1. Basic Rule: Store tablespace under ${tajo.rootdir}/${tablespace-
> name}.
> > In other words, ${base-uri} for the tablespace is
> > ${tajo.rootdir}/${tablespace-name}.
> > 2. The base uri for the default tablespace is
> ${tajo.warehouse.directory}.
> > (no change)
> > 3. The tablespace name whose base uri conflicts the default tablespace's
> > base uri is disallowed.
> >
> > I think this strategy can be a roundabout way to the shortcomings of
> > HiveCatalogStore, but I cannot certain. Are there any conflicts to my
> > proposal with current implementation or our future plan? If you have any
> > opinions, don't hesitate to give me a message. Thanks for reading.
> >
> > Regards,
> > Dongjin
> >
> > [^1]: https://issues.apache.org/jira/browse/TAJO-2171
> >
> > --
> > *Dongjin Lee*
> >
> >
> > *Software developer in Line+.So interested in massive-scale machine
> > learning.facebook: www.facebook.com/dongjin.lee.kr
> > <http://www.facebook.com/dongjin.lee.kr>linkedin:
> > kr.linkedin.com/in/dongjinleekr
> > <http://kr.linkedin.com/in/dongjinleekr>github:
> > <http://goog_969573159/>github.com/dongjinleekr
> > <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
> > <http://www.twitter.com/dongjinleekr>*
> >
>



-- 
*Dongjin Lee*


*Software developer in Line+.So interested in massive-scale machine
learning.facebook: www.facebook.com/dongjin.lee.kr
<http://www.facebook.com/dongjin.lee.kr>linkedin:
kr.linkedin.com/in/dongjinleekr
<http://kr.linkedin.com/in/dongjinleekr>github:
<http://goog_969573159>github.com/dongjinleekr
<http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
<http://www.twitter.com/dongjinleekr>*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message