hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 권병창 <magnu...@navercorp.com>
Subject RE: Installing just the HDFS client
Date Mon, 29 Aug 2016 07:18:35 GMT
Hi
 
refer to below pom.xml
you should modify hadoop version that you want.
 
build is simple:  mvn clean package 
It will make a jar  of 34mb size. 
 
usage is simple:
java -jar ${build_jar}.jar -mkdir /user/home
java -jar ${build_jar}.jar -ls /user/home
 
 
pom.xml
&lt;?xml version="1.0" encoding="UTF-8"?&gt; 
&lt;project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;

  &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;

  &lt;groupId&gt;com.naver.c3&lt;/groupId&gt;
  &lt;artifactId&gt;hdfs-connector&lt;/artifactId&gt;
  &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;

  &lt;dependencies&gt;
    &lt;dependency&gt;
      &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
      &lt;artifactId&gt;hadoop-common&lt;/artifactId&gt;
      &lt;version&gt;2.7.1&lt;/version&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;
      &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
      &lt;artifactId&gt;hadoop-hdfs&lt;/artifactId&gt;
      &lt;version&gt;2.7.1&lt;/version&gt;
    &lt;/dependency&gt;
  &lt;/dependencies&gt;

  &lt;build&gt;
    &lt;plugins&gt;
      &lt;plugin&gt;
        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
        &lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
        &lt;version&gt;2.3&lt;/version&gt;
        &lt;executions&gt;
          &lt;execution&gt;
            &lt;phase&gt;package&lt;/phase&gt;
            &lt;goals&gt;
              &lt;goal&gt;shade&lt;/goal&gt;
            &lt;/goals&gt;
            &lt;configuration&gt;
              &lt;minimizeJar&gt;false&lt;/minimizeJar&gt;
              &lt;transformers&gt;
                &lt;transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"&gt;
                  &lt;mainClass&gt;org.apache.hadoop.fs.FsShell&lt;/mainClass&gt;
                &lt;/transformer&gt;
                &lt;transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/&gt;
              &lt;/transformers&gt;
           &lt;/configuration&gt;
          &lt;/execution&gt;
        &lt;/executions&gt;
      &lt;/plugin&gt;
    &lt;/plugins&gt;
  &lt;/build&gt;

&lt;/project&gt; 
 
-----Original Message-----
From: "F21"&lt;f21.groups@gmail.com&gt; 
To: &lt;user@hadoop.apache.org&gt;; 
Cc: 
Sent: 2016-08-29 (월) 14:25:09
Subject: Installing just the HDFS client
 
Hi all,

I am currently building a HBase docker image. As part of the bootstrap 
process, I need to run some `hdfs dfs` commands to create directories on 
HDFS.

The whole hadoop distribution is pretty heavy contains things to run 
namenodes, etc. I just need a copy of the dfs client for my docker 
image. I have done some poking around and see that I need to include the 
files in bin/, libexec/, lib/ and share/hadoop/common and share/hadoop/hdfs.

However, including the above still takes up quite a bit of space. Is 
there a single JAR I can add to my image to perform operations against HDFS?


Cheers,

Francis


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org



Mime
View raw message