hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaobin She <xiaobin...@gmail.com>
Subject Why other process can't see the change after calling hdfsHFlush unless hdfsCloseFile is called?
Date Tue, 17 Dec 2013 11:35:47 GMT

I'm using libhdfs to deal with hdfs in an c++ programme.

And I have encountered an problem.

here is the scenario :
1. first I call hdfsOpenFile with O_WRONLY flag to open an file
2. call hdfsWrite to write some data
3. call hdfsHFlush to flush the data,  according to the header hdfs.h,
after this call returns, new readers shoule be able to see the data
4. I use an http get request to get the file list on that directionary
through the webhdfs interface,
here  I have to use the webhdfs interface because I need to deal with
symlink file
5. from the json response which is returned by the webhdfs, I found that
the lenght of the file is still 0.

I have tried to replace hdfsHFlush with hdfsFlush or hdfsSync, or call
these three together, but still doesn't work.

Buf if I call hdfsCloseFile after I call the hdfsHFlush, then I can get the
correct file lenght through the webhdfs interface.

Is this right? I mean if you want the other process to see the change  of
data, you need to call hdfsCloseFile?

Or is there somethings I did wrong?

thank you very much for your help.

View raw message