hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaobin She <xiaobin...@gmail.com>
Subject Re: Why other process can't see the change after calling hdfsHFlush unless hdfsCloseFile is called?
Date Thu, 19 Dec 2013 12:50:04 GMT
sorry to reply to my own thread.

Does anyone know the answer to this question?
If so, can you please tell me if my understanding is right or wrong?

thanks.



2013/12/17 Xiaobin She <xiaobinshe@gmail.com>

> hi,
>
> I'm using libhdfs to deal with hdfs in an c++ programme.
>
> And I have encountered an problem.
>
> here is the scenario :
> 1. first I call hdfsOpenFile with O_WRONLY flag to open an file
> 2. call hdfsWrite to write some data
> 3. call hdfsHFlush to flush the data,  according to the header hdfs.h,
> after this call returns, new readers shoule be able to see the data
> 4. I use an http get request to get the file list on that directionary
> through the webhdfs interface,
> here  I have to use the webhdfs interface because I need to deal with
> symlink file
> 5. from the json response which is returned by the webhdfs, I found that
> the lenght of the file is still 0.
>
> I have tried to replace hdfsHFlush with hdfsFlush or hdfsSync, or call
> these three together, but still doesn't work.
>
> Buf if I call hdfsCloseFile after I call the hdfsHFlush, then I can get
> the correct file lenght through the webhdfs interface.
>
>
> Is this right? I mean if you want the other process to see the change  of
> data, you need to call hdfsCloseFile?
>
> Or is there somethings I did wrong?
>
> thank you very much for your help.
>
>
>
>
>

Mime
View raw message