incubator-accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse McConnell <jesse.mcconn...@gmail.com>
Subject Re: TableOperations import directory
Date Tue, 18 Oct 2011 20:30:17 GMT
I figured out the root of the problem..

If I pass in a qualified failure path to the dfs I was getting an
error about the wrong FS, that it needed to be file:///

I was able to resolve this by using the CachedConfiguration.set() to
the configuration I was using which allowed the BulkImportHelper to
get the correct cached configuration object so it could in turn create
the FAIL dir on the right file _and_ then call getGlobs on the [0-9]*
on the correct fs...

so it was able to copy the file in the importDirectory over just fine,
but when it went to make the failure directory that was being passed
in it would pull a bad configuration and then get itself hopelessly
lost.

I am seeing imported data now :)

thanks much guys,
jesse

--
jesse mcconnell
jesse.mcconnell@gmail.com



On Tue, Oct 18, 2011 at 15:14, Keith Turner <keith@deenlo.com> wrote:
>> Not seeing this either, again, while debugging here I don't ever see
>> any files being returned in the BulkImportHelper, it always returns an
>> empty list of mapFilesInfo which lead me down the rathole to getGlobs
>> pattern matching [0-9]* files.  Which is what lead us to start looking
>> at your example to see what was fundamentally different. :/
>>
>
> Seems like once we ran into an issue w/ bulk import where a source
> file was passed in instead of a source dir.  Seems like the hdfs
> listStatus call did something odd in this case, like returned an empty
> set.  So maybe nothing happened.  My memory is fuzzy on this though.
> Make sure you are passing the correct dir to the import call.
>

Mime
View raw message