drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Padma Penumarthy <ppenumar...@mapr.com>
Subject Re: Parquet Metadata table on Rolling window
Date Tue, 17 Oct 2017 03:00:18 GMT
No, it does not. We rebuild metadata for all directories under mydata when you
query /mydata. 
If new files get added anywhere in the whole hierarchy, metadata gets regenerated
for all of them.

However, if you query only  /mydata/3 and nothing is changed under 3,
no metadata is generated.

Thanks
Padma

> On Oct 16, 2017, at 12:33 PM, François Méthot <fmethot78@gmail.com> wrote:
> 
> Thanks Padma,
> 
> Would we benefits at all from running metadata on directories that we know
> we will never modify?
> 
> We would end up with:
> /mydata/3/(Metadata generated...)
> /mydata/4/(Metadata generated...)
> /mydata/.../(Metadata generated...)
> /mydata/109/(Metadata generated...)
> /mydata/110/(Current partition : Metadata NOT generated yet...)
> 
> When users query /mydata, would drill take advantage of the metadata
> available in each subfolder?
> 
> Francois

Mime
View raw message