Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0FBFCC1B6 for ; Tue, 16 Jul 2013 13:52:23 +0000 (UTC) Received: (qmail 53122 invoked by uid 500); 16 Jul 2013 13:52:17 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 52810 invoked by uid 500); 16 Jul 2013 13:52:17 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 52776 invoked by uid 99); 16 Jul 2013 13:52:16 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jul 2013 13:52:16 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of shahab.yunus@gmail.com designates 209.85.214.52 as permitted sender) Received: from [209.85.214.52] (HELO mail-bk0-f52.google.com) (209.85.214.52) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jul 2013 13:52:10 +0000 Received: by mail-bk0-f52.google.com with SMTP id d7so274746bkh.25 for ; Tue, 16 Jul 2013 06:51:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=CUfKJiLTux5EkxwT7O3sDFWjyuR8jWFEgck4pNYns9A=; b=CZh1SOevcYILUjoddROIch44qz0se3WatxbFozCrhwB6l8VLKMdBNwNDR95WA6b5Nh EuukiWQySwZEFYsw+aJM6ryMMg6CXU36JNbr4sKTPUv74nixzygtFLDBmZUQZi2EpcA9 O2wwWLByBqNmMyNWMBSIYcsGN5usNLWjhlGqj+OgkNhM3SanL4gZjEczQJlejgC0Uncz T6eoXz5YkRI8aiDcUk1CF3bbwF69VhDjGEWWRvv86QTRPf6OtVzfHdyIx1zWsaCWB/cZ fpbS7yH6URMQspExpp/7wQlnQU7dWtrmuWtGeu8TEp13YbWJRyQSRDi2XY1/CYSHbGTR Jzow== MIME-Version: 1.0 X-Received: by 10.205.134.198 with SMTP id id6mr293828bkc.121.1373982710328; Tue, 16 Jul 2013 06:51:50 -0700 (PDT) Received: by 10.204.188.71 with HTTP; Tue, 16 Jul 2013 06:51:50 -0700 (PDT) In-Reply-To: References: Date: Tue, 16 Jul 2013 09:51:50 -0400 Message-ID: Subject: Re: java.io.IOException: error=2, No such file or directory From: Shahab Yunus To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=047d7b624d1860a1d604e1a14780 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b624d1860a1d604e1a14780 Content-Type: text/plain; charset=ISO-8859-1 The error is: *Please set $HBASE_HOME to the root of your HBase installation.* * * Have you checked whether it is set or not? Have you verified your HBase or Hadoop installation? Similarly, the following: *Cannot run program "psql": java.io.IOException: error=2, No such file or directory * Also seem to indicate that PostGres is not on the path. What and how are your environment variables set? Can you access/run this independently from this directory? Regards, Shahab On Tue, Jul 16, 2013 at 6:44 AM, Fatih Haltas wrote: > Hi everyone, > > I am trying to import data from postgresql to hdfs. But I am having some > problems, Here is the problem details: > > Sqoop Version: 1.4.3 > Hadoop Version:1.0.4 > > > *1) When I use this command:* > * > * > *./sqoop import-all-tables --connect jdbc:postgresql:// > 192.168.194.158:5432/IMS --username pgsql -P* > > > Here is the exact output > ====================================================== > Warning: /usr/lib/hbase does not exist! HBase imports will fail. > Please set $HBASE_HOME to the root of your HBase installation. > Warning: $HADOOP_HOME is deprecated. > > Enter password: > 13/07/16 13:52:28 INFO manager.SqlManager: Using default fetchSize of 1000 > 13/07/16 13:52:29 INFO tool.CodeGenTool: Beginning code generation > 13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SELECT > t.* FROM "publicinici" AS t LIMIT 1 > 13/07/16 13:52:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is > /home/hadoop/project/hadoop-1.0.4 > Note: > /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.java > uses or overrides a deprecated API. > Note: Recompile with -Xlint:deprecation for details. > 13/07/16 13:52:30 INFO orm.CompilationManager: Writing jar file: > /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publicinici.jar > 13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you are > importing from postgresql. > 13/07/16 13:52:30 WARN manager.PostgresqlManager: This transfer can be > faster! Use the --direct > 13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a > postgresql-specific fast path. > 13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of > publicinici > 13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: > SELECT MIN("budapablik"), MAX("budapablik") FROM "publicinici" > 13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146_0525 > 13/07/16 13:52:32 INFO mapred.JobClient: map 0% reduce 0% > ^X[hadoop@ADUAE042-LAP-V bin]./sqoop version > Warning: /usr/lib/hbase does not exist! HBase imports will fail. > Please set $HBASE_HOME to the root of your HBase installation. > Warning: $HADOOP_HOME is deprecated. > =================================================== > > *I have only one table which is "publicinici",that holds 2 entity in one > row* and it read but at mapreduce phase it is getting stucked. > > > *2) When I use this command in which --direct option is added, eveything > is going well, but getting one error, which is * > > ERROR tool.ImportAllTablesTool: Encountered IOException running import > job: java.io.IOException: Cannot run program "psql": java.io.IOException: > error=2, No such file or directory > > *Also, in the hdfs publicinici file has been created as empty. * > > Here is the exact output. > ======================== > 13/07/16 13:58:42 INFO manager.SqlManager: Using default fetchSize of 1000 > 13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation > 13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: SELECT > t.* FROM "publicinici" AS t LIMIT 1 > 13/07/16 13:58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is > /home/hadoop/project/hadoop-1.0.4 > Note: > /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.java > uses or overrides a deprecated API. > Note: Recompile with -Xlint:deprecation for details. > 13/07/16 13:58:44 INFO orm.CompilationManager: Writing jar file: > /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publicinici.jar > 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql > fast path import > 13/07/16 13:58:44 INFO manager.SqlManager: Executing SQL statement: SELECT > t.* FROM "publicinici" AS t LIMIT 1 > 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command is > COPY (SELECT "budapablik" FROM "publicinici" WHERE 1=1) TO STDOUT WITH > DELIMITER E'\54' CSV ; > 13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Performing import > of table publicinici from database IMS > 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop > complete. > 13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transferred 0 > bytes in 17,361,268.2044 seconds (0 bytes/sec) > 13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException > running import job: java.io.IOException: Cannot run program "psql": > java.io.IOException: error=2, No such file or directory > > > What should I do to be able to fix the problem. > Any help is gonna be appreciated. Thank you very much. > > > --047d7b624d1860a1d604e1a14780 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
The error is:

Please set $HBASE_HOME to the root of= your HBase installation.

Have you checked whether it is set or not? Have you verified your HB= ase or Hadoop installation?

Similarly, the following:

=A0Cannot run program "psql&q= uot;: java.io.IOException: error=3D2, No such file or directory=A0

Also seem to indicate that PostGres is not on the path. What and = how are your environment variables set? Can you access/run this independent= ly from this directory?

Reg= ards,
Shahab


On Tue,= Jul 16, 2013 at 6:44 AM, Fatih Haltas <fatih.haltas@nyu.edu> wrote:
Hi everyone,=A0

I am trying to import data from postgresql to hdfs. But I am having= some problems, Here is the problem details:

Sqoop Version: 1.4.3
Hadoop Version:1.0.4


1) When I use this command:
=

./sqoop import-all-tables --connect jdbc:post= gresql://192.= 168.194.158:5432/IMS --username pgsql -P


Here is the exact output
=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
Warning: /usr/lib/hbase does not exist! HBase imports w= ill fail.
Please set $HBASE_HOME to the root of your HBase instal= lation.
Warning: $HADOOP_HOME is deprecated.

Enter pa= ssword:=A0
13/07/16 13:52:28 INFO manager.SqlManager: Using defau= lt fetchSize of 1000
13/07/16 13:52:29 INFO tool.CodeGenTool: Beg= inning code generation
13/07/16 13:52:29 INFO manager.SqlManager: Executing SQL statement: SE= LECT t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:52= :29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/project= /hadoop-1.0.4
Note: /tmp/sqoop-hadoop/compile/0f484159ce27d8ed3c2d95ca13974f1a/publi= cinici.java uses or overrides a deprecated API.
Note: Recompile w= ith -Xlint:deprecation for details.
13/07/16 13:52:30 INFO orm.Co= mpilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/0f484159ce27d= 8ed3c2d95ca13974f1a/publicinici.jar
13/07/16 13:52:30 WARN manager.PostgresqlManager: It looks like you ar= e importing from postgresql.
13/07/16 13:52:30 WARN manager.Postg= resqlManager: This transfer can be faster! Use the --direct
13/07/16 13:52:30 WARN manager.PostgresqlManager: option to exercise a post= gresql-specific fast path.
13/07/16 13:52:30 INFO mapreduce.ImportJobBase: Beginning import of pu= blicinici
13/07/16 13:52:31 INFO db.DataDrivenDBInputFormat: Boun= dingValsQuery: SELECT MIN("budapablik"), MAX("budapablik&quo= t;) FROM "publicinici"
13/07/16 13:52:31 INFO mapred.JobClient: Running job: job_201302261146= _0525
13/07/16 13:52:32 INFO mapred.JobClient: =A0map 0% reduce 0= %
^X[hadoop@ADUAE042-LAP-V bin]./sqoop version
Warning:= /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

=
I have only one table which is "publicinici",that holds 2= entity in one row and =A0it read but at mapreduce phase it is getting = stucked.


2) When I use this command in which -= -direct option is added, eveything is going well, but getting one error, wh= ich is =A0

ERROR tool.ImportAllTablesTool: Enc= ountered IOException running import job: java.io.IOException: Cannot run pr= ogram "psql": java.io.IOException: error=3D2, No such file or dir= ectory=A0

Also, in the hdfs publicinici file has been created = as empty.=A0

Here is the exact output.=A0
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
13/07/16 13:58:42 INFO manager.SqlManager: Using default fetc= hSize of 1000
13/07/16 13:58:42 INFO tool.CodeGenTool: Beginning code generation
13/07/16 13:58:43 INFO manager.SqlManager: Executing SQL statement: = SELECT t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:= 58:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/proje= ct/hadoop-1.0.4
Note: /tmp/sqoop-hadoop/compile/56c2a5b04e83ace0112fe038eb4d1599/publi= cinici.java uses or overrides a deprecated API.
Note: Recompile w= ith -Xlint:deprecation for details.
13/07/16 13:58:44 INFO orm.Co= mpilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/56c2a5b04e83a= ce0112fe038eb4d1599/publicinici.jar
13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Beginning psql= fast path import
13/07/16 13:58:44 INFO manager.SqlManager: Exec= uting SQL statement: SELECT t.* FROM "publicinici" AS t LIMIT 1
13/07/16 13:58:44 INFO manager.DirectPostgresqlManager: Copy command i= s COPY (SELECT "budapablik" FROM "publicinici" WHERE 1= =3D1) TO STDOUT WITH DELIMITER E'\54' CSV ;
13/07/16 13:5= 8:44 INFO manager.DirectPostgresqlManager: Performing import of table publi= cinici from database IMS
13/07/16 13:58:45 INFO manager.DirectPostgresqlManager: Transfer loop = complete.
13/07/16 13:58:45 INFO manager.DirectPostgresqlManager:= Transferred 0 bytes in 17,361,268.2044 seconds (0 bytes/sec)
13/07/16 13:58:45 ERROR tool.ImportAllTablesTool: Encountered IOException r= unning import job: java.io.IOException: Cannot run program "psql"= : java.io.IOException: error=3D2, No such file or directory=A0

What should I do to be able to fix the problem.= =A0
Any help is gonna be appreciated. Thank you very much.
<= div>


--047d7b624d1860a1d604e1a14780--