incubator-accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse McConnell <jesse.mcconn...@gmail.com>
Subject Re: TableOperations import directory
Date Tue, 18 Oct 2011 20:00:01 GMT
>  1) Move files from src dir into an /accumulo/tables/<tid>/bulk_XXX
> dir.

> So after the bulk import files should be missing from source dir, is this the case?

Yep

> Do you see bulk dirs under the /accumulo/tables/<tid> dir, if so do they contain
anything?

Yep

> The move is done by a random tablet server.  In some tablet server debug log,
> should see messages like "Moved <src> to <dest>" coming from the class
> ClientServiceHandler.

I have seen this in the debugging as well

>  2) Inspect the files to determine first and last key.  I did not see
> any client side related debugging for this.

I see the first and last key using the rfile.PrintInfo and it looks
reasonable, though it does seem to mention only one of the column
families in play for the file.

>  3) Contact tablet servers to assign files to tablet.  There should
> be client and server side debugging related to this. If you set the
> log4j level to DEBUG on the client side for org.apache.accumulo.core,
> I am thinking you should see the following message.
>
>           log.debug("Assigning " + uniqMapFiles.size() + " map files to "
>                    + assignmentsPerTablet.size() + " tablets at " + location);

Not seeing this either, again, while debugging here I don't ever see
any files being returned in the BulkImportHelper, it always returns an
empty list of mapFilesInfo which lead me down the rathole to getGlobs
pattern matching [0-9]* files.  Which is what lead us to start looking
at your example to see what was fundamentally different. :/

> On the server side, you should see the following on a tablet server
>
>          log.log(TLevel.TABLET_HIST, extent+" import "+path+"
> "+paths.get(tpath));
>
> Could grep for TABLET_HIST and then grep for import to find this.

I don't see any 'import' text in the tserver_host.debug.log like this...

>  4) Anything that fails, is copied to the failure dir you supplied.
> Is the failure dir empty after the import?

Empty, but I should double check, I get errors if this failure
directory is on the same hdfs as the input to the importDirectory, it
errors and says its expecting file:///  But when I just pass in a new
Path("FAIL-" + uuid) relative path it runs.

thanks for the help

cheers,
jesse

Mime
View raw message