hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Віталій Тимчишин <tiv...@gmail.com>
Subject Re: Hadoop Production Issue
Date Sun, 17 Jul 2011 16:54:00 GMT
2011/7/16 jagaran das <jagaran_das@yahoo.co.in>

> Hi,
>
> Due to requirements in our current production CDH3 cluster we need to copy
> around 11520 small size files (Total Size 12 GB) to the cluster for one
> application.
> Like this we have 20 applications that would run in parallel
>
> So one set would have 11520 files of total size 12 GB
> Like this we would have 15 sets in parallel,
>
> We have a total SLA for the pipeline from copy to pig aggregation to copy
> to local and sql load is 15 mins.
>
>
Have you tried to use HARs?
-- 
Best regards,
 Vitalii Tymchyshyn

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message