From hdfs-commits-return-565-apmail-hadoop-hdfs-commits-archive=hadoop.apache.org@hadoop.apache.org Wed Mar 10 23:26:06 2010 Return-Path: Delivered-To: apmail-hadoop-hdfs-commits-archive@minotaur.apache.org Received: (qmail 56608 invoked from network); 10 Mar 2010 23:26:06 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Mar 2010 23:26:06 -0000 Received: (qmail 84989 invoked by uid 500); 10 Mar 2010 23:25:34 -0000 Delivered-To: apmail-hadoop-hdfs-commits-archive@hadoop.apache.org Received: (qmail 84946 invoked by uid 500); 10 Mar 2010 23:25:34 -0000 Mailing-List: contact hdfs-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-commits@hadoop.apache.org Received: (qmail 84938 invoked by uid 99); 10 Mar 2010 23:25:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 23:25:34 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 23:25:33 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 04D26238897D; Wed, 10 Mar 2010 23:25:13 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r921622 - in /hadoop/hdfs/trunk: CHANGES.txt src/contrib/fuse-dfs/src/fuse_context_handle.h src/contrib/fuse-dfs/src/fuse_impls_readdir.c src/contrib/fuse-dfs/src/fuse_init.c Date: Wed, 10 Mar 2010 23:25:12 -0000 To: hdfs-commits@hadoop.apache.org From: tomwhite@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20100310232513.04D26238897D@eris.apache.org> Author: tomwhite Date: Wed Mar 10 23:25:12 2010 New Revision: 921622 URL: http://svn.apache.org/viewvc?rev=921622&view=rev Log: HDFS-961. dfs_readdir incorrectly parses paths. Contributed by Eli Collins. Modified: hadoop/hdfs/trunk/CHANGES.txt hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_context_handle.h hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_impls_readdir.c hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_init.c Modified: hadoop/hdfs/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/CHANGES.txt?rev=921622&r1=921621&r2=921622&view=diff ============================================================================== --- hadoop/hdfs/trunk/CHANGES.txt (original) +++ hadoop/hdfs/trunk/CHANGES.txt Wed Mar 10 23:25:12 2010 @@ -190,6 +190,8 @@ Trunk (unreleased changes) HDFS-861. fuse-dfs does not support O_RDWR. (Brian Bockelman via tomwhite) + HDFS-961. dfs_readdir incorrectly parses paths. (Eli Collins via tomwhite) + Release 0.21.0 - Unreleased INCOMPATIBLE CHANGES Modified: hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_context_handle.h URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_context_handle.h?rev=921622&r1=921621&r2=921622&view=diff ============================================================================== --- hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_context_handle.h (original) +++ hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_context_handle.h Wed Mar 10 23:25:12 2010 @@ -39,10 +39,6 @@ typedef struct dfs_context_struct { int direct_io; char **protectedpaths; size_t rdbuffer_size; - // todo: - // total hack city - use this to strip off the dfs url from the filenames. (in fuse_impls_readdir.c) - char dfs_uri[1024]; - int dfs_uri_len; } dfs_context; #endif Modified: hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_impls_readdir.c URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_impls_readdir.c?rev=921622&r1=921621&r2=921622&view=diff ============================================================================== --- hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_impls_readdir.c (original) +++ hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_impls_readdir.c Wed Mar 10 23:25:12 2010 @@ -46,9 +46,10 @@ int dfs_readdir(const char *path, void * return -EIO; } - // call dfs to read the dir + // Read dirents. Calling a variant that just returns the final path + // component (HDFS-975) would save us from parsing it out below. int numEntries = 0; - hdfsFileInfo *info = hdfsListDirectory(userFS,path,&numEntries); + hdfsFileInfo *info = hdfsListDirectory(userFS, path, &numEntries); userFS = NULL; // NULL means either the directory doesn't exist or maybe IO error. @@ -59,7 +60,6 @@ int dfs_readdir(const char *path, void * int i ; for (i = 0; i < numEntries; i++) { - // check the info[i] struct if (NULL == info[i].mName) { syslog(LOG_ERR,"ERROR: for <%s> info[%d].mName==NULL %s:%d", path, i, __FILE__,__LINE__); continue; @@ -68,11 +68,14 @@ int dfs_readdir(const char *path, void * struct stat st; fill_stat_structure(&info[i], &st); - // hack city: todo fix the below to something nicer and more maintainable but - // with good performance - // strip off the path but be careful if the path is solely '/' - // NOTE - this API started returning filenames as full dfs uris - const char *const str = info[i].mName + dfs->dfs_uri_len + path_len + ((path_len == 1 && *path == '/') ? 0 : 1); + // Find the final path component + const char *str = strrchr(info[i].mName, '/'); + if (NULL == str) { + syslog(LOG_ERR, "ERROR: invalid URI %s %s:%d", + info[i].mName, __FILE__,__LINE__); + continue; + } + str++; // pack this entry into the fuse buffer int res = 0; Modified: hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_init.c URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_init.c?rev=921622&r1=921621&r2=921622&view=diff ============================================================================== --- hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_init.c (original) +++ hadoop/hdfs/trunk/src/contrib/fuse-dfs/src/fuse_init.c Wed Mar 10 23:25:12 2010 @@ -112,12 +112,7 @@ void *dfs_init() dfs->rdbuffer_size = options.rdbuffer_size; dfs->direct_io = options.direct_io; - bzero(dfs->dfs_uri,0); - sprintf(dfs->dfs_uri,"dfs://%s:%d/",dfs->nn_hostname,dfs->nn_port); - dfs->dfs_uri_len = strlen(dfs->dfs_uri); - - // use ERR level to ensure it makes it into the log. - syslog(LOG_ERR, "mounting %s", dfs->dfs_uri); + syslog(LOG_INFO, "mounting %s:%d", dfs->nn_hostname, dfs->nn_port); init_protectedpaths(dfs); assert(dfs->protectedpaths != NULL);