hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Li <car...@gmail.com>
Subject Re: Is there a way to get a Block through block id?
Date Wed, 22 Jan 2014 06:00:23 GMT
Hi Colin,

Thanks for the reply. I guess you're referring to the below methods? If so,
I'm afraid it can only get an empty block but not the real block I really
want to move.
=====================================
  public ExtendedBlock(final String poolId, final long blockId) {
    this(poolId, blockId, 0, 0);
  }
  public ExtendedBlock(final String poolId, final long blkid, final long
len,
      final long genstamp) {
    this.poolId = poolId;
    block = new Block(blkid, len, genstamp);
  }
=====================================

I'm now working around this by adding an option to directly move the block
during fsck (NmaenodeFsck). But still, if there's any method to directly
get the Block instance by blockid through any api, it would be great to
know. :-)

On 21 January 2014 03:46, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:

> In order to uniquely identify a block in hadoop 2.2, you are going to need
> both a block and a block pool ID.  You can construct a Block object with
> those two items.
>
> On Wed, Jan 15, 2014 at 8:46 AM, Yu Li <carp84@gmail.com> wrote:
>
> > Dear all,
> >
> > As titled, I actually have two questions here:
> >
> > 1. In current releases like hadoop-2.2.0, is block id unique and able to
> > locate a Block in HDFS? I'm asking because I could see HDFS-4645 is
> trying
> > to resolve the uniqueness issue. However, from the code comment it seems
> > block id is expected to be unique
> >
> >
> It's expected to be unique within a block pool.  You can get the block pool
> ID you are using from your FSNamesystem object in 2.2.
>
> best,
> Colin
>
> 2. It seems there's no method to get a Block object through a block id.
> > However, there's some scenarios need such method, like if I use the
> > FavouredNode feature to create logical datanode group and planned to put
> > some data within a group, then I might need to periodically check whether
> > there's block somehow placed outside the group, and move it back. In such
> > scenario, I would need to first get block ids, then move. But to move
> them,
> > it seems we need a Block instance to initiate the ExtendedBlock object.
> >
> > Any suggestion, or reference to existing JIRA would be highly
> appreciated,
> > and thanks in advance!
> >
> > --
> > Best Regards,
> > Li Yu
> >
>



-- 
Best Regards,
Li Yu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message