impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jin Chul Kim <jinc...@gmail.com>
Subject Re: Docker image for impala development
Date Thu, 16 Nov 2017 02:28:45 GMT
Hi Sandish,

The elapsed time of scratch build is depend on parallelism on your build
system and network speed due to downloading packages on the fly. You need
to check resources are fully utilized. As described in
bootstrap_development.sh
<https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=blob;f=bin/bootstrap_development.sh>,
the build takes about two hours.

Regarding disk space, my local environment(i.e. du -sh ${IMPALA_HOME}) uses
over 100G, which includes debug binary(non release) and loaded test data to
the cluster. Impala build relies on ccache
<https://cwiki.apache.org/confluence/display/IMPALA/Tips+for+Faster+Impala+Builds>
to accelerate fast building. The temporal objects are located at
${HOME}/.ccache. Please check the size. The wiki page guides 50G ccache
size, but you may need to decrease the size if you don't have enough disk
space.

It would be more helpful if you will share more detailed information such
as breakdown build time with your system environment. You may find a new
infra issue and this will help for us.

Best regards,
Jinchul



2017-11-16 4:26 GMT+09:00 Jim Apple <jbapple@cloudera.com>:

> I don't know of a up-to-date docker image or dockerfile.
>
> When you say "following the apache site docker image for impala dev
> steps", I assume you mean this page:
>
> https://cwiki.apache.org/confluence/display/IMPALA/
> Docker+for+Impala+Developers
>
> It does take a lot of time and a lot of disk space, which is
> unfortunate and unlikely to be easily fixable. You could run just the
> bootstrap_system.sh script, which will get a working system, but won't
> compile the code (which takes perhaps 40 minutes) or load the test
> data (which takes something like an hour).
>
> On Wed, Nov 15, 2017 at 10:43 AM, Sandish Kumar HN
> <sanysandish@gmail.com> wrote:
> > Hi,
> >
> > Does any have docker image for impala dev ? Atleast the docker file
> >  script?
> > I have been following the apache site docker image for impala dev steps,
> > which is taking lot of time to build and taking large amount of disk
> space,
> > which is an issue to upload to docker hub.
> >
> > Please suggest me some steps
> > --
> > Sent from Gmail Mobile
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message