asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xikui Wang <xik...@uci.edu>
Subject Re: Do we have a method to append local files to existed dataset?
Date Fri, 04 Mar 2016 20:19:53 GMT
Hi Yingyi,

Thanks for your reply. I think the external dataset with scan query is a
good solution.
I will try that. Thank you.

Best,
Xikui

On Fri, Mar 4, 2016 at 11:53 AM, Yingyi Bu <buyingyi@gmail.com> wrote:

> Xikui,
>
> If the number of localfs files is too large,  a solution could be to put
> your files on HDFS and then load it.  Loading from HDFS always has a fixed
> degree of parallelism regardless of the number of files.
>
> >> I am wondering is there a way to append adm file to existed dataset?
> You can create an external dataset and then write an insert statement where
> the body is a scan query. AsterixDB doesn't load any data into its own
> storage for an external dataset but just keeps file paths.
> Here is a manual for external datasets:
> https://ci.apache.org/projects/asterixdb/aql/externaldata.html
>
> Best,
> Yingyi
>
>
> On Fri, Mar 4, 2016 at 11:47 AM, Xikui Wang <xikuiw@uci.edu> wrote:
>
> > Hi,
> >
> > I want to import data from multiple adm files into a same dataset.
> Merging
> > them together and then loading from localfs can be a viable solution, but
> > this may become a problem when the number become too large. I am
> wondering
> > is there a way to append adm file to existed dataset?
> >
> > Thank you.
> >
> > Best,
> > Xikui
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message