Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1DDC0708D for ; Thu, 11 Aug 2011 08:29:10 +0000 (UTC) Received: (qmail 45935 invoked by uid 500); 11 Aug 2011 08:29:08 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 45592 invoked by uid 500); 11 Aug 2011 08:28:59 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 45582 invoked by uid 99); 11 Aug 2011 08:28:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Aug 2011 08:28:55 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ophchu@gmail.com designates 209.85.214.41 as permitted sender) Received: from [209.85.214.41] (HELO mail-bw0-f41.google.com) (209.85.214.41) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Aug 2011 08:28:47 +0000 Received: by bkbzt4 with SMTP id zt4so1123576bkb.14 for ; Thu, 11 Aug 2011 01:28:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=sISaiO+z6gPtfICqVqmtaKh2kkqyzoG8Zcz4/R5rHXE=; b=eLA8huC4nzDysHw83He7cPYp7L+qEwKqSCKxm3qFYjNcrwdIvOtV4HjTyoSqXgNkHF DP0qERgD3SmPZBxAOwhxPXo8qgFswObyBaCI/OlwRKzCPvc+P5NtW00fZVjtLJLl/aJE 0B2dYs40JxEq3w1BfxKYpPVE1vC4GnqUU2qhg= Received: by 10.204.145.19 with SMTP id b19mr2953841bkv.192.1313051306138; Thu, 11 Aug 2011 01:28:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.205.82.76 with HTTP; Thu, 11 Aug 2011 01:28:06 -0700 (PDT) In-Reply-To: References: From: Ophir Cohen Date: Thu, 11 Aug 2011 11:28:06 +0300 Message-ID: Subject: Re: Bulk upload To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=0015173fe820ac9c6904aa3694ba --0015173fe820ac9c6904aa3694ba Content-Type: text/plain; charset=ISO-8859-1 I did some more tests and found the problem: on local run the distribtued cache does not work. On full cluster it works. Sorry for your time... Ophir PS Is there any way to use distributed cache localy as well (i.e. when I'm running MR from intellijIdea )? On Thu, Aug 11, 2011 at 11:20, Ophir Cohen wrote: > Now I see that it uses the distributed cache - but for some reason > the TotalOrderPartitioner does not grab it. > Ophir > > > On Thu, Aug 11, 2011 at 11:08, Ophir Cohen wrote: > >> Hi, >> I started to use bulk upload and encounter a strange problem. >> I'm using Cloudera cdh3-u1. >> >> I'm using HFileOutputFormat.configureIncrementalLoad() to configure my >> job. >> This method create partition file for the TotalOrderPartitioner and save >> it to HDFS. >> >> When the TotalOrderPartitioner initiated it tries to find the path for the >> file in the configuration: >> public static String getPartitionFile(Configuration conf) { >> return conf.get(PARTITIONER_PATH, DEFAULT_PATH); >> } >> >> The strange thing is that this parameter never assigned! >> It looks to me that it should have configured >> in HFileOutputFormat.configureIncrementalLoad() but it does not! >> >> Then it takes the default ("_part") or something similar and (of course) >> does not find it... >> >> BTW >> When I manually add this parameter it works great. >> >> Is that a bug or do I miss something? >> Thanks, >> Ophir >> >> > --0015173fe820ac9c6904aa3694ba--