Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14728EA1C for ; Tue, 27 Nov 2012 04:13:56 +0000 (UTC) Received: (qmail 15405 invoked by uid 500); 27 Nov 2012 04:13:55 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 14866 invoked by uid 500); 27 Nov 2012 04:13:54 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 14797 invoked by uid 99); 27 Nov 2012 04:13:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 04:13:52 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 04:13:49 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 8DA1823888E7 for ; Tue, 27 Nov 2012 04:13:29 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1413992 [1/3] - in /hadoop/common/branches/branch-1-win: ./ bin/ conf/ src/packages/win/ src/packages/win/HadoopServiceHost/ src/packages/win/HadoopServiceHost/Properties/ src/packages/win/resources/ src/packages/win/scripts/ src/packages/... Date: Tue, 27 Nov 2012 04:13:26 -0000 To: common-commits@hadoop.apache.org From: suresh@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20121127041329.8DA1823888E7@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: suresh Date: Tue Nov 27 04:13:19 2012 New Revision: 1413992 URL: http://svn.apache.org/viewvc?rev=1413992&view=rev Log: HADOOP-9074. Hadoop install scripts for Windows. Contributed by Steve Maine, Ramya Nimmagadda, Chuan Liu, Brad Sarsfield and Ivan Mitic. Added: hadoop/common/branches/branch-1-win/bin/hadoop-daemon.cmd hadoop/common/branches/branch-1-win/src/packages/win/ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.csproj hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.sln hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Program.cs hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/AssemblyInfo.cs hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ServiceHost.cs hadoop/common/branches/branch-1-win/src/packages/win/resources/ hadoop/common/branches/branch-1-win/src/packages/win/resources/winpkg.utils.psm1 hadoop/common/branches/branch-1-win/src/packages/win/scripts/ hadoop/common/branches/branch-1-win/src/packages/win/scripts/InstallApi.psm1 hadoop/common/branches/branch-1-win/src/packages/win/scripts/TestInstallApi.ps1 hadoop/common/branches/branch-1-win/src/packages/win/scripts/install.cmd hadoop/common/branches/branch-1-win/src/packages/win/scripts/install.ps1 hadoop/common/branches/branch-1-win/src/packages/win/scripts/uninstall.cmd hadoop/common/branches/branch-1-win/src/packages/win/scripts/uninstall.ps1 hadoop/common/branches/branch-1-win/src/packages/win/template/ hadoop/common/branches/branch-1-win/src/packages/win/template/bin/ hadoop/common/branches/branch-1-win/src/packages/win/template/bin/start_daemons.cmd hadoop/common/branches/branch-1-win/src/packages/win/template/bin/stop_daemons.cmd hadoop/common/branches/branch-1-win/src/packages/win/template/conf/ hadoop/common/branches/branch-1-win/src/packages/win/template/conf/core-site.xml hadoop/common/branches/branch-1-win/src/packages/win/template/conf/hadoop-metrics2.properties hadoop/common/branches/branch-1-win/src/packages/win/template/conf/hdfs-site.xml hadoop/common/branches/branch-1-win/src/packages/win/template/conf/log4j.properties hadoop/common/branches/branch-1-win/src/packages/win/template/conf/mapred-site.xml Modified: hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt hadoop/common/branches/branch-1-win/bin/hadoop.cmd hadoop/common/branches/branch-1-win/bin/hdfs.cmd hadoop/common/branches/branch-1-win/bin/mapred.cmd hadoop/common/branches/branch-1-win/build.xml hadoop/common/branches/branch-1-win/conf/hadoop-env.cmd Modified: hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt (original) +++ hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt Tue Nov 27 04:13:19 2012 @@ -228,3 +228,6 @@ Branch-hadoop-1-win - unreleased HADOOP-9062. hadoop-env.cmd overwrites the value of *_OPTS set before install. (Ganeshan Iyer via suresh) + + HADOOP-9074. Hadoop install scripts for Windows. (Steve Maine, + Ramya Nimmagadda, Chuan Liu, Brad Sarsfield and Ivan Mitic via suresh) Added: hadoop/common/branches/branch-1-win/bin/hadoop-daemon.cmd URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/bin/hadoop-daemon.cmd?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/bin/hadoop-daemon.cmd (added) +++ hadoop/common/branches/branch-1-win/bin/hadoop-daemon.cmd Tue Nov 27 04:13:19 2012 @@ -0,0 +1,144 @@ +@echo off +@rem Licensed to the Apache Software Foundation (ASF) under one or more +@rem contributor license agreements. See the NOTICE file distributed with +@rem this work for additional information regarding copyright ownership. +@rem The ASF licenses this file to You under the Apache License, Version 2.0 +@rem (the "License"); you may not use this file except in compliance with +@rem the License. You may obtain a copy of the License at +@rem +@rem http://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. + + +@rem Runs a Hadoop command as a daemon. +@rem +@rem Environment Variables +@rem +@rem HADOOP_CONF_DIR Alternate conf dir. Default is %HADOOP_HOME%/conf. +@rem HADOOP_LOG_DIR Where log files are stored. PWD by default. +@rem HADOOP_MASTER host:path where hadoop code should be copied from +@rem HADOOP_IDENT_STRING A string representing this instance of hadoop. %USERNAME% by default +@rem + +setlocal +set usage= "Usage: Pass arguments in the given order : hadoop-daemon.cmd [--config ] [--hosts hostlistfile] (start|stop) " + +@rem if no args specified, show usage +if [%2] == [] ( + echo %usage% + goto :eof +) + +if not defined HADOOP_BIN_PATH ( + set HADOOP_BIN_PATH=%~dp0 +) + +if "%HADOOP_BIN_PATH:~-1%" == "\" ( + set HADOOP_BIN_PATH=%HADOOP_BIN_PATH:~0,-1% +) + +call :updatepath %HADOOP_BIN_PATH% +set hadoop-config-script=%HADOOP_BIN_PATH%\hadoop-config.cmd +call %hadoop-config-script% %* +if "%1" == "--config" ( + shift + shift +) + +if "%1" == "--hosts" ( + shift + shift +) + +@rem get remaining arguments + +set startStop=%1 +shift +set command=%1 +shift + +if exist %HADOOP_CONF_DIR%\hadoop-env.cmd ( + call %HADOOP_CONF_DIR%\hadoop-env.cmd +) + +set hdfscommands=namenode secondarynamenode datanode +for %%i in ( %hdfscommands% ) do ( + if %command% == %%i set hdfscommand=true + ) + +set mapredcommands=jobtracker tasktracker historyserver + for %%i in ( %mapredcommands% ) do ( + if %command% == %%i set mapredcommand=true + ) +@rem if an alternate config directory is specified generate new service xml +if defined %HADOOP_CONF_DIR% ( + move /Y %command%.xml %command%.backup.xml + if defined hdfscommand ( + %HADOOP_BIN_PATH%\hdfs.cmd --conf %HADOOP_CONF_DIR% makeServiceXml > %command%.xml + ) else ( + if defined mapredcommand ( + %HADOOP_BIN_PATH%\mapred.cmd --conf %HADOOP_CONF_DIR% makeServiceXml > %command%.xml + ) else ( + @echo %usage% + exit /b 1 + ) + ) + if not exist %command%.xml ( + @echo Generating new conf failed, copying back old conf + move /Y %command%.backup.xml %command%.xml + exit /b 2 + ) +) + +if %startStop% == start ( + @echo Stopping any existing services before starting + sc stop %command% + @echo starting %command% + set errorlevel= + sc start %command% + if %errorlevel% NEQ 0 ( + @echo Failed to start %command% service + exit /b 2 + ) + goto :end +) else ( + if %startStop% == stop ( + @echo stopping %command% + set errorlevel= + sc stop %command% + if %errorlevel% NEQ 0 ( + @echo Failed to stop %command% service + exit /b 2 + ) + goto :end + ) else ( + @echo %usage% + exit /b 1 + ) +) +@rem Adds the path sent as argument to path variable +:updatepath + set path_to_add=%* + set current_path_comparable=%path:(x86)=% + set current_path_comparable=%current_path_comparable: =_% + set path_to_add_comparable=%path_to_add:(x86)=% + set path_to_add_comparable=%path_to_add_comparable: =_% + for %%i in ( %current_path_comparable% ) do ( + if /i "%%i" == "%path_to_add_comparable%" ( + set path_to_add_exist=true + ) + ) + set system_path_comparable= + set path_to_add_comparable= + if not defined path_to_add_exist path=%path_to_add%;%path% + set path_to_add= + goto :eof + +:end + +endlocal \ No newline at end of file Modified: hadoop/common/branches/branch-1-win/bin/hadoop.cmd URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/bin/hadoop.cmd?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/bin/hadoop.cmd (original) +++ hadoop/common/branches/branch-1-win/bin/hadoop.cmd Tue Nov 27 04:13:19 2012 @@ -63,7 +63,11 @@ call :updatepath %HADOOP_BIN_PATH% set hadoop-hdfs-script=%HADOOP_BIN_PATH%\hdfs.cmd set hadoop-mapred-script=%HADOOP_BIN_PATH%\mapred.cmd - call %hadoop-config-script% + call %hadoop-config-script% %* + if "%1" == "--config" ( + shift + shift + ) set hadoop-command=%1 if not defined hadoop-command ( @@ -78,7 +82,7 @@ call :updatepath %HADOOP_BIN_PATH% ) if defined hdfscommand ( if exist %hadoop-hdfs-script% ( - call %hadoop-hdfs-script% %* + call %hadoop-hdfs-script% %hadoop-command% %hadoop-command-arguments% goto :eof ) ) @@ -89,7 +93,7 @@ call :updatepath %HADOOP_BIN_PATH% ) if defined mapredcommand ( if exist %hadoop-mapred-script% ( - call %hadoop-mapred-script% %* + call %hadoop-mapred-script% %hadoop-command% %hadoop-command-arguments% goto :eof ) ) @@ -109,8 +113,9 @@ call :updatepath %HADOOP_BIN_PATH% set CLASSPATH=%CLASSPATH%;%CD% set CLASS=%hadoop-command% ) - + set path=%PATH%;%HADOOP_BIN_PATH% + call %JAVA% %JAVA_HEAP_MAX% %HADOOP_OPTS% -classpath %CLASSPATH% %CLASS% %hadoop-command-arguments% goto :eof @@ -165,7 +170,11 @@ call :updatepath %HADOOP_BIN_PATH% @rem This changes %1, %2 etc. Hence those cannot be used after calling this. :make_command_arguments - if "%2" == "" goto :eof + if "%1" == "--config" ( + shift + shift + ) + if [%2] == [] goto :eof shift set _arguments= :MakeCmdArgsLoop @@ -184,7 +193,7 @@ call :updatepath %HADOOP_BIN_PATH% :print_usage - @echo Usage: hadoop COMMAND + @echo Usage: hadoop [--config confdir] COMMAND @echo where COMMAND is one of: @echo namenode -format format the DFS filesystem @echo secondarynamenode run the DFS secondary namenode Modified: hadoop/common/branches/branch-1-win/bin/hdfs.cmd URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/bin/hdfs.cmd?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/bin/hdfs.cmd (original) +++ hadoop/common/branches/branch-1-win/bin/hdfs.cmd Tue Nov 27 04:13:19 2012 @@ -24,7 +24,14 @@ if "%HADOOP_BIN_PATH:~-1%" == "\" ( set HADOOP_BIN_PATH=%HADOOP_BIN_PATH:~0,-1% ) -set HADOOP_LOGFILE=hadoop-%1-%computername%.log +@rem if we're being called by --service we need to use %2 otherwise use %1 +@rem for the command line so we log to the right file +if "%2" == "" ( + set HADOOP_LOGFILE=hadoop-%1-%computername%.log +) else ( + set HADOOP_LOGFILE=hadoop-%2-%computername%.log +) + set hdfs-config-script=%HADOOP_BIN_PATH%\hadoop-config.cmd call %hdfs-config-script% %* @@ -125,7 +132,7 @@ goto :eof @rem This changes %1, %2 etc. Hence those cannot be used after calling this. :make_command_arguments - if "%2" == "" goto :eof + if [%2] == [] goto :eof set _count=0 set _hdfsarguments= if defined service_entry (set _shift=2) else (set _shift=1) Modified: hadoop/common/branches/branch-1-win/bin/mapred.cmd URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/bin/mapred.cmd?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/bin/mapred.cmd (original) +++ hadoop/common/branches/branch-1-win/bin/mapred.cmd Tue Nov 27 04:13:19 2012 @@ -23,7 +23,14 @@ if "%HADOOP_BIN_PATH:~-1%" == "\" ( set HADOOP_BIN_PATH=%HADOOP_BIN_PATH:~0,-1% ) -set HADOOP_LOGFILE=hadoop-%1-%computername%.log +@rem if we're being called by --service we need to use %2 otherwise use %1 +@rem for the command line so we log to the right file +if "%2" == "" ( + set HADOOP_LOGFILE=hadoop-%1-%computername%.log +) else ( + set HADOOP_LOGFILE=hadoop-%2-%computername%.log +) + set mapred-config-script=%HADOOP_BIN_PATH%\hadoop-config.cmd call %mapred-config-script% %* @@ -114,7 +121,7 @@ goto :eof @rem This changes %1, %2 etc. Hence those cannot be used after calling this. :make_command_arguments - if "%2" == "" goto :eof + if [%2] == [] goto :eof set _count=0 set _mapredarguments= if defined service_entry (set _shift=2) else (set _shift=1) Modified: hadoop/common/branches/branch-1-win/build.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/build.xml?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/build.xml (original) +++ hadoop/common/branches/branch-1-win/build.xml Tue Nov 27 04:13:19 2012 @@ -66,6 +66,8 @@ + + @@ -94,6 +96,7 @@ + @@ -145,8 +148,7 @@ - - + @@ -200,6 +202,12 @@ + + + + + + @@ -396,6 +404,16 @@ + + + + + + + + + + @@ -620,23 +638,26 @@ /> - - - - - - - - + + + + + + + + + + + @@ -688,7 +709,7 @@ - + @@ -776,7 +797,7 @@ @@ -1295,6 +1316,44 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1698,119 +1757,6 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -1861,6 +1807,59 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2140,6 +2139,51 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2185,7 +2229,6 @@ - @@ -2971,7 +3014,7 @@ - + Modified: hadoop/common/branches/branch-1-win/conf/hadoop-env.cmd URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/conf/hadoop-env.cmd?rev=1413992&r1=1413991&r2=1413992&view=diff ============================================================================== --- hadoop/common/branches/branch-1-win/conf/hadoop-env.cmd (original) +++ hadoop/common/branches/branch-1-win/conf/hadoop-env.cmd Tue Nov 27 04:13:19 2012 @@ -7,9 +7,6 @@ @rem remote nodes. @rem The java implementation to use. Required. -if not defined JAVA_HOME ( - set JAVA_HOME=c:\apps\java\openjdk7 -) @rem Extra Java CLASSPATH elements. Optional. @rem set HADOOP_CLASSPATH= Added: hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.csproj URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.csproj?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.csproj (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.csproj Tue Nov 27 04:13:19 2012 @@ -0,0 +1,78 @@ + + + + Debug + x86 + 8.0.30703 + 2.0 + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05} + Exe + Properties + HadoopServiceHost + HadoopServiceHost + v4.0 + Client + 512 + + + x86 + true + full + false + bin\Debug\ + DEBUG;TRACE + prompt + 4 + + + x86 + pdbonly + true + bin\Release\ + TRACE + prompt + 4 + + + true + bin\Debug\ + DEBUG;TRACE + full + AnyCPU + prompt + false + + + bin\Release\ + TRACE + true + pdbonly + AnyCPU + prompt + + + + + + + + + + + + + + + + Component + + + + + \ No newline at end of file Added: hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.sln URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.sln?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.sln (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/HadoopServiceHost.sln Tue Nov 27 04:13:19 2012 @@ -0,0 +1,26 @@ + +Microsoft Visual Studio Solution File, Format Version 11.00 +# Visual Studio 2010 +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HadoopServiceHost", "HadoopServiceHost.csproj", "{DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Debug|x86 = Debug|x86 + Release|Any CPU = Release|Any CPU + Release|x86 = Release|x86 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Debug|Any CPU.Build.0 = Debug|Any CPU + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Debug|x86.ActiveCfg = Debug|x86 + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Debug|x86.Build.0 = Debug|x86 + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Release|Any CPU.ActiveCfg = Release|Any CPU + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Release|Any CPU.Build.0 = Release|Any CPU + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Release|x86.ActiveCfg = Release|x86 + {DE479F78-4AA1-4454-8DCE-0D2DAAC5BD05}.Release|x86.Build.0 = Release|x86 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal Added: hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Program.cs URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Program.cs?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Program.cs (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Program.cs Tue Nov 27 04:13:19 2012 @@ -0,0 +1,24 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.ServiceProcess; +using System.Text; + +namespace HadoopServiceHost +{ + static class Program + { + /// + /// The main entry point for the application. + /// + static void Main() + { + ServiceBase[] ServicesToRun; + ServicesToRun = new ServiceBase[] + { + new ServiceHost() + }; + ServiceBase.Run(ServicesToRun); + } + } +} Added: hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/AssemblyInfo.cs URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/AssemblyInfo.cs?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/AssemblyInfo.cs (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/Properties/AssemblyInfo.cs Tue Nov 27 04:13:19 2012 @@ -0,0 +1,36 @@ +using System.Reflection; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; + +// General Information about an assembly is controlled through the following +// set of attributes. Change these attribute values to modify the information +// associated with an assembly. +[assembly: AssemblyTitle("HadoopServiceHost")] +[assembly: AssemblyDescription("")] +[assembly: AssemblyConfiguration("")] +[assembly: AssemblyCompany("Microsoft")] +[assembly: AssemblyProduct("HadoopServiceHost")] +[assembly: AssemblyCopyright("Copyright -¬ Microsoft 2012")] +[assembly: AssemblyTrademark("")] +[assembly: AssemblyCulture("")] + +// Setting ComVisible to false makes the types in this assembly not visible +// to COM components. If you need to access a type in this assembly from +// COM, set the ComVisible attribute to true on that type. +[assembly: ComVisible(false)] + +// The following GUID is for the ID of the typelib if this project is exposed to COM +[assembly: Guid("fd4ab2db-e236-4b6b-8ee7-d07d9e52541c")] + +// Version information for an assembly consists of the following four values: +// +// Major Version +// Minor Version +// Build Number +// Revision +// +// You can specify all the values or you can default the Build and Revision Numbers +// by using the '*' as shown below: +// [assembly: AssemblyVersion("1.0.*")] +[assembly: AssemblyVersion("1.0.0.0")] +[assembly: AssemblyFileVersion("1.0.0.0")] Added: hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ServiceHost.cs URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ServiceHost.cs?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ServiceHost.cs (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/HadoopServiceHost/ServiceHost.cs Tue Nov 27 04:13:19 2012 @@ -0,0 +1,153 @@ +using System; +using System.Collections.Generic; +using System.ComponentModel; +using System.Data; +using System.Diagnostics; +using System.Linq; +using System.ServiceProcess; +using System.Text; +using System.Xml; +using System.Threading; + +namespace HadoopServiceHost +{ + // Simple service host that allows running of Hadoop services in the context + // of Windows services. The service host is configured through a simple + // xml file named after the exe. + // Example xml: + // + // datanode + // I:\git\tools\java\bin\java + // -server org.apache.hadoop.hdfs.server.datanode.DataNode + // + public partial class ServiceHost : ServiceBase + { + private String serviceExe; + private String serviceArgs; + private volatile bool inShutdown = false; + Process process; + TraceSource trace = new TraceSource("HadoopServiceTraceSource"); + + private void InitInternal(String fileName) + { + try + { + XmlDocument doc = new XmlDocument(); + trace.TraceEvent(TraceEventType.Information, 0, + "Loading service xml: " + fileName); + doc.Load(fileName); + + this.ServiceName = doc.SelectSingleNode("//service/name").InnerText; + this.CanShutdown = true; + this.CanStop = true; + this.CanPauseAndContinue = false; + + this.serviceExe = doc.SelectSingleNode("//service/executable").InnerText; + this.serviceArgs = doc.SelectSingleNode("//service/arguments").InnerText; + if (ServiceName == null || serviceExe == null || serviceArgs == null) + { + trace.TraceEvent(TraceEventType.Error, 0, + "Invalid service XML file format"); + throw new Exception("Invalid service XML file format"); + } + trace.TraceEvent(TraceEventType.Information, 0, + "Successfully parsed service xml for service " + ServiceName); + trace.TraceEvent(TraceEventType.Information, 0, "Command line: " + + serviceExe + " " + serviceArgs); + } + catch (Exception ex) + { + trace.TraceEvent(TraceEventType.Error, 0, + "Failed to parse the service xml with exception" + ex); + throw; + } + } + + private void TerminateOurselves(int exitCode) + { + trace.Flush(); + System.Environment.Exit(exitCode); + } + + private void InitTracing() + { + TextWriterTraceListener tr = new TextWriterTraceListener( + System.Reflection.Assembly.GetExecutingAssembly().Location.Replace(".exe", + ".trace.log")); + trace.Switch = new SourceSwitch("Switch", "switch"); + trace.Switch.Level = SourceLevels.Information; + tr.TraceOutputOptions = TraceOptions.DateTime | TraceOptions.Timestamp; + trace.Listeners.Clear(); + trace.Listeners.Add(tr); + Trace.AutoFlush = true; + + trace.TraceEvent(TraceEventType.Information, 0, + "Tracing successfully initialized"); + } + + public ServiceHost() + { + InitTracing(); + String xmlFileName = + System.Reflection.Assembly.GetExecutingAssembly().Location.Replace(".exe", + ".xml"); + InitInternal(xmlFileName); + process = new Process(); + } + + protected override void OnStart(string[] args) + { + trace.TraceEvent(TraceEventType.Information, 0, "ServiceHost#OnStart"); + + process.StartInfo.FileName = this.serviceExe; + process.StartInfo.Arguments = this.serviceArgs; + + if (!process.Start()) + { + trace.TraceEvent(TraceEventType.Error, 0, + "Process#Start failed, terminating service host"); + TerminateOurselves(1); + } + else + { + trace.TraceEvent(TraceEventType.Information, 0, + "Child process started, PID: " + process.Id); + + // Start a separate thread that will wait for child process to exit + new Thread(delegate() + { + process.WaitForExit(); + trace.TraceEvent(TraceEventType.Information, 0, + "Child process exited with exit code: " + process.ExitCode); + + if (!inShutdown) + { + // If not in shutdown state, child process terminated outside of + // our control, we should also stop ourselves + trace.TraceEvent(TraceEventType.Information, 0, + "Service host not in shutdown mode, terminating service host"); + TerminateOurselves(process.ExitCode); + } + // else, we should be stopped anyways + + }).Start(); + } + } + + protected override void OnStop() + { + trace.TraceEvent(TraceEventType.Information, 0, + "ServiceHost#OnStop, killing process " + process.Id); + inShutdown = true; + process.Kill(); + } + + protected override void OnShutdown() + { + trace.TraceEvent(TraceEventType.Information, 0, + "ServiceHost#OnShutdown, killing process " + process.Id); + inShutdown = true; + process.Kill(); + } + } +} Added: hadoop/common/branches/branch-1-win/src/packages/win/resources/winpkg.utils.psm1 URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/src/packages/win/resources/winpkg.utils.psm1?rev=1413992&view=auto ============================================================================== --- hadoop/common/branches/branch-1-win/src/packages/win/resources/winpkg.utils.psm1 (added) +++ hadoop/common/branches/branch-1-win/src/packages/win/resources/winpkg.utils.psm1 Tue Nov 27 04:13:19 2012 @@ -0,0 +1,203 @@ +### Licensed to the Apache Software Foundation (ASF) under one or more +### contributor license agreements. See the NOTICE file distributed with +### this work for additional information regarding copyright ownership. +### The ASF licenses this file to You under the Apache License, Version 2.0 +### (the "License"); you may not use this file except in compliance with +### the License. You may obtain a copy of the License at +### +### http://www.apache.org/licenses/LICENSE-2.0 +### +### Unless required by applicable law or agreed to in writing, software +### distributed under the License is distributed on an "AS IS" BASIS, +### WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +### See the License for the specific language governing permissions and +### limitations under the License. + + +### NOTE: This file is common across the Windows installer projects for Hadoop Core, Hive, and Pig. +### This dependency is currently managed by convention. +### If you find yourself needing to change something in this file, it's likely that you're +### either doing something that's more easily done outside this file or is a bigger change +### that likely has wider ramifications. Work intentionally. + + +param( [parameter( Position=0, Mandatory=$true)] + [String] $ComponentName ) + +function Write-Log ($message, $level, $pipelineObj ) +{ + switch($level) + { + "Failure" + { + $message = "$ComponentName FAILURE: $message" + Write-Error $message + break; + } + + "Info" + { + $message = "${ComponentName}: $message" + Write-Host $message + break; + } + + default + { + $message = "${ComponentName}: $message" + Write-Host "$message" + } + } + + + Out-File -FilePath $ENV:WINPKG_LOG -InputObject "$message" -Append -Encoding "UTF8" + + if( $pipelineObj -ne $null ) + { + Out-File -FilePath $ENV:WINPKG_LOG -InputObject $pipelineObj.InvocationInfo.PositionMessage -Append -Encoding "UTF8" + } +} + +function Write-LogRecord( $source, $record ) +{ + if( $record -is [Management.Automation.ErrorRecord]) + { + $message = "$ComponentName-$source FAILURE: " + $record.Exception.Message + + if( $message.EndsWith( [Environment]::NewLine )) + { + Write-Host $message -NoNewline + [IO.File]::AppendAllText( "$ENV:WINPKG_LOG", "$message", [Text.Encoding]::UTF8 ) + } + else + { + Write-Host $message + Out-File -FilePath $ENV:WINPKG_LOG -InputObject $message -Append -Encoding "UTF8" + } + } + else + { + $message = $record + Write-Host $message + Out-File -FilePath $ENV:WINPKG_LOG -InputObject "$message" -Append -Encoding "UTF8" + } +} + +function Invoke-Cmd ($command) +{ + Write-Log $command + $out = cmd.exe /C "$command" 2>&1 + $out | ForEach-Object { Write-LogRecord "CMD" $_ } + return $out +} + +function Invoke-CmdChk ($command) +{ + Write-Log $command + $out = cmd.exe /C "$command" 2>&1 + $out | ForEach-Object { Write-LogRecord "CMD" $_ } + if (-not ($LastExitCode -eq 0)) + { + throw "Command `"$out`" failed with exit code $LastExitCode " + } + return $out +} + +function Invoke-Ps ($command) +{ + Write-Log $command + $out = powershell.exe -InputFormat none -Command "$command" 2>&1 + #$out | ForEach-Object { Write-LogRecord "PS" $_ } + return $out +} + +function Invoke-PsChk ($command) +{ + Write-Log $command + $out = powershell.exe -InputFormat none -Command "$command" 2>&1 + #$out | ForEach-Object { Write-LogRecord "PS" $_ } + if (-not ($LastExitCode -eq 0)) + { + throw "Command `"$out`" failed with exit code $LastExitCode " + } + return $out +} + +### Sets HADOOP_NODE_INSTALL_ROOT if unset +### Initializes Winpkg Environment (ENV:WINPKG_LOG and ENV:WINPKG_BIN) +### Tests for Admin + +function Initialize-InstallationEnv( $scriptDir, $logFilename ) +{ + $HDP_INSTALL_PATH = $scriptDir + $HDP_RESOURCES_DIR = Resolve-Path "$HDP_INSTALL_PATH\..\resources" + + if( -not (Test-Path ENV:HADOOP_NODE_INSTALL_ROOT)) + { + $ENV:HADOOP_NODE_INSTALL_ROOT = "c:\hadoop" + } + + if( -not (Test-Path ENV:WINPKG_LOG )) + { + throw "ENV:WINPKG_LOG not set" + } + else + { + Write-Log "Logging to existing log $ENV:WINPKG_LOG" "Info" + } + + Write-Log "Logging to $ENV:WINPKG_LOG" "Info" + Write-Log "HDP_INSTALL_PATH: $HDP_INSTALL_PATH" + Write-Log "HDP_RESOURCES_DIR: $HDP_RESOURCES_DIR" + + $currentPrincipal = New-Object Security.Principal.WindowsPrincipal( [Security.Principal.WindowsIdentity]::GetCurrent( ) ) + if ( -not ($currentPrincipal.IsInRole( [Security.Principal.WindowsBuiltInRole]::Administrator ) ) ) + { + throw "install script must be run elevated" + } + + return $HDP_INSTALL_PATH, $HDP_RESOURCES_DIR +} + +function Test-JavaHome +{ + if( -not (Test-Path $ENV:JAVA_HOME\bin\java.exe)) + { + throw "JAVA_HOME not set properly; $ENV:JAVA_HOME\bin\java.exe does not exist" + } +} + +### Add service control permissions to authenticated users. +### Reference: +### http://stackoverflow.com/questions/4436558/start-stop-a-windows-service-from-a-non-administrator-user-account +### http://msmvps.com/blogs/erikr/archive/2007/09/26/set-permissions-on-a-specific-service-windows.aspx + +function Set-ServiceAcl ($service) +{ + $cmd = "sc sdshow $service" + $sd = Invoke-Cmd $cmd + + Write-Log "Current SD: $sd" + + ## A;; --- allow + ## RP ---- SERVICE_START + ## WP ---- SERVICE_STOP + ## CR ---- SERVICE_USER_DEFINED_CONTROL + ## ;;;AU - AUTHENTICATED_USERS + + $sd = [String]$sd + $sd = $sd.Replace( "S:(", "(A;;RPWPCR;;;AU)S:(" ) + Write-Log "Modifying SD to: $sd" + + $cmd = "sc sdset $service $sd" + Invoke-Cmd $cmd +} + +Export-ModuleMember -Function Initialize-InstallationEnv +Export-ModuleMember -Function Invoke-Cmd +Export-ModuleMember -Function Invoke-CmdChk +Export-ModuleMember -Function Invoke-Ps +Export-ModuleMember -Function Invoke-PsChk +Export-ModuleMember -Function Set-ServiceAcl +Export-ModuleMember -Function Test-JavaHome +Export-ModuleMember -Function Write-Log