hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: Unable to pick data from subdirectories into hive table in CDH 5.3.3
Date Thu, 19 May 2016 19:26:15 GMT
Hi,

I am not familiar with CDH, but in a default set -up, the hive directory is
under hdfs://<HOSENAME./PORT/hive/warehouse

hdfs dfs -ls hdfs://rhes564:9000/user/hive/warehouse with databases.
drwxr-xr-x   - hduser supergroup          0 2016-05-03 18:18
hdfs://rhes564:9000/user/hive/warehouse/accounts.db
drwxr-xr-x   - hduser supergroup          0 2016-05-17 07:03
hdfs://rhes564:9000/user/hive/warehouse/oraclehadoop.db
drwxr-xr-x   - hduser supergroup          0 2016-05-15 19:10
hdfs://rhes564:9000/user/hive/warehouse/test.db

HTH

Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 19 May 2016 at 10:50, Abhishek Dubey <Abhishek.Dubey@xoriant.com> wrote:

> Hi,
>
>
>
> In hdfs I have a directory structure like this.
>
> /user/hdfs/Data/Data1/File1
>
> /user/hdfs/Data/Data2/File2
>
>
>
> And I am creating an external table like:
>
> CREATE external TABLE db.tablename
>
> (
>
> amt1 STRING,
>
> amt2 STRING,
>
> amt3 STRING
>
> )
>
> row format delimited
>
> fields terminated by ','
>
> location '/user/hdfs/Data/';
>
>
>
> Also, I have set two properties:
>
> set mapred.input.dir.recursive=true;
>
> set hive.mapred.supports.subdirectories=true;
>
>
>
> This setup is working perfectly fine on my local single node vm, Having
> all vanilla apache installations and setup,
>
>
>
> But. on cloudera 5.3.3 cluster of 4 nodes, above mentioned properties for
> recursive lookup of sub directories for an external hive table is not
> working.
>
> In the Cloudera manager i have added the properties in Hive-Site.xml,
> deployed configuration and restarted Hive service but still not working.
>
> <property>
>
>   <name> mapred.input.dir.recursive</name>
>
>   <value>true</value>
>
> </property>
>
> <property>
>
>   <name>hive.mapred.supports.subdirectories</name>
>
>   <value>true</value>
>
> </property>
>
>
>
> When querying select *  on CDH What i get is this, Zero rows.
>
> hive> select * from tablename;
>
> OK
>
> Time taken: 0.322 seconds
>
> hive>
>
>
>
> Whereas on local vm it is giving desired output.
>
>
>
> Is there anything else on CDH that we need to take care to pick data from
> subdirectories into hive table?
>
>
>
> Thanks in advance.
> *Abhishek Dubey*
>
>
>
>
>

Mime
View raw message