Return-Path: X-Original-To: apmail-commons-issues-archive@minotaur.apache.org Delivered-To: apmail-commons-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5581C11099 for ; Wed, 18 Jun 2014 02:15:09 +0000 (UTC) Received: (qmail 77870 invoked by uid 500); 18 Jun 2014 02:15:09 -0000 Delivered-To: apmail-commons-issues-archive@commons.apache.org Received: (qmail 77784 invoked by uid 500); 18 Jun 2014 02:15:09 -0000 Mailing-List: contact issues-help@commons.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: issues@commons.apache.org Delivered-To: mailing list issues@commons.apache.org Received: (qmail 77773 invoked by uid 99); 18 Jun 2014 02:15:09 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Jun 2014 02:15:09 +0000 Date: Wed, 18 Jun 2014 02:15:09 +0000 (UTC) From: "Bernd Eckenfels (JIRA)" To: issues@commons.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (VFS-530) Update HDFS to latest release in prep for VFS 2.1 release MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/VFS-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034696#comment-14034696 ] Bernd Eckenfels edited comment on VFS-530 at 6/18/14 2:14 AM: -------------------------------------------------------------- If this is a unrelated/unneeded change I would do it in a seperate commit. In all cases the comment needs to be fixed (1M): {code}- conf.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 1024 * 100); // 100K blocksize + conf.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 1024 * 1024); // 100K blocksize{code} I tested it on windows, and it seems to have the problem to solve the cygwin dependency with a own binary and native code, but the POM changes alone do not work, it uses the wunutils.exe (bit recovers from it:) {code}Running org.apache.commons.vfs2.provider.hdfs.test.HdfsFileProviderTest 2014-06-18 03:13:44,939 [main] ERROR: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333) at org.apache.hadoop.util.Shell.(Shell.java:326) at org.apache.hadoop.util.StringUtils.(StringUtils.java:76) at org.apache.hadoop.conf.Configuration.getTrimmedStringCollection(Configuration.java:1707) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1238) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1219) at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:816) at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:702) at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:640) at org.apache.commons.vfs2.provider.hdfs.test.HdfsFileProviderTest.(HdfsFileProviderTest.java:95){code} and it also needs the hadoop.dll, especially the accesscheck method. Looking at the code this really can be stubbed out, but for now at least this one place is demanding in a way that the DLL would have to be installed to run the unittests. :-/ was (Author: b.eckenfels): If this is a unrelated/unneeded change I would do it in a seperate commit. In all cases the comment needs to be fixed (1M): {code}- conf.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 1024 * 100); // 100K blocksize + conf.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 1024 * 1024); // 100K blocksize{code} I tested it on windows, and it seems to have the problem to solve the cygwin dependency with a own binary and native code, but the POM changes alone do not work, it isses the wunutils.exe and a dependency. {code}Running org.apache.commons.vfs2.provider.hdfs.test.HdfsFileProviderTest 2014-06-18 03:13:44,939 [main] ERROR: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333) at org.apache.hadoop.util.Shell.(Shell.java:326) at org.apache.hadoop.util.StringUtils.(StringUtils.java:76) at org.apache.hadoop.conf.Configuration.getTrimmedStringCollection(Configuration.java:1707) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1238) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1219) at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:816) at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:702) at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:640) at org.apache.commons.vfs2.provider.hdfs.test.HdfsFileProviderTest.(HdfsFileProviderTest.java:95){code} > Update HDFS to latest release in prep for VFS 2.1 release > --------------------------------------------------------- > > Key: VFS-530 > URL: https://issues.apache.org/jira/browse/VFS-530 > Project: Commons VFS > Issue Type: Task > Affects Versions: 2.1 > Reporter: Dave Marion > Fix For: 2.1 > > Attachments: VFS-530-1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)