hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eitan Rosenfeld <eita...@gmail.com>
Subject The current/ directory
Date Wed, 24 Apr 2013 14:16:26 GMT
Hi all,

Two questions:

1. I'd like to store groups of blk_ and .meta files within different
partitions alongside the data/current/ folder. I'd like to store only the
blk and .meta files in these partitions, leaving out all other directories
and files.

This would result in the data directory having structure such as:

blocksBeingWritten/
current/    # /dev/sda1
current2/   # /dev/sda2
current3/   # /dev/sda3
current4/   # /dev/sda4
detach/
in_use.lock
storage
tmp/

I've concluded that to add partitions in such a manner, I can treat them as
volumes and ensure that they are added to the FSVolumeSet instantiated in
the FSDataset constructor.  (I would first have to mount the partitions
under the data directory).

Does my conclusion above demonstrate a correct understanding of how volumes
work?
Is modifying the FSDataset constructor a reasonable way of including
partitions?


2. My understanding is that blocks are first written to tmp/ or
blocksBeingWritten/, and moved to current/ only after they are written. Can
someone direct me to the class and method responsible for moving the files?
I wasn't able to locate it.

Thank you in advance!

Eitan Rosenfeld

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message