impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Valencia Serrao" <vser...@us.ibm.com>
Subject Re: Fw: Issues with generating testdata for Impala
Date Sat, 30 Apr 2016 12:01:07 GMT

Hi Alex,

I was going more deeper through the logs. I have some findings and queries:

1. At the "Invalidating Metadata" step (as mentioned in below mail), i
noticed that, it is trying to use kerberos. Perhaps, this is preventing the
testdata generation from proceeding, as we are   not using Kerberos.
     I need to know how this can be done without involving Kerberos
support ?

2. I had executed the fe tests despite the incomplete testdata generation,
the tests started and surely have failed. Many of these (null pointer
exception in AuthorzationTests) have a common cause: "tpch database does
not exist."
     e.g. as shown
in  .Impala/cluster_logs/query_tests/test-run-workload.log.

     Does the "tpch" database gets created after the current blocker step
"Invalidating Metadata" ?

3. In the fe test console output log, another error shown:
    ============================= test session starts
==============================
	platform linux2 -- Python 2.7.5 -- py-1.4.30 -- pytest-2.7.2
	rootdir: /work/, inifile:
	plugins: random, xdist
	ERROR: file not found:/work/I
mpala/../Impala-auxiliary-tests/tests/aux_custom_cluster_tests/

   These are not present/created on my vm. May i know when these get
created ?

4. Could you also share the total number of  fe tests ?


Looking forward to your reply.

Regards,
Valencia




From:	Valencia Serrao/Austin/Contr/IBM
To:	dev@impala.incubator.apache.org, Alex Behm
            <alex.behm@cloudera.com>
Cc:	Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha
            Panpaliya/Austin/Contr/IBM@IBMUS, Valencia
            Serrao/Austin/Contr/IBM@IBMUS
Date:	04/30/2016 09:05 AM
Subject:	Fw: Issues with generating testdata for Impala



Hi Alex,

I've been able to make some progress on testdata generation, however, i
still face the following issues:

*******************************************************************************************************************************************************************
Invalidating Metadata
(load-functional-query-exhaustive-impala-load-generated-parquet-none-none.sql):
INSERT OVERWRITE TABLE functional_parquet.alltypes partition (year, month)
SELECT id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
float_col, double_col, date_string_col, string_col, timestamp_col, year,
month
FROM functional.alltypes

Data Loading from Impala failed with error: ImpalaBeeswaxException:
 INNER EXCEPTION: <class 'socket.error'>
 MESSAGE: [Errno 104] Connection reset by peer
Error in /root/nishidha/Impala/testdata/bin/create-load-data.sh at line 41:
while [ -n "$*" ]
Error in /root/nishidha/Impala/buildall.sh at line 368: $
{IMPALA_HOME}/testdata/bin/create-load-data.sh ${CREATE_LOAD_DATA_ARGS} <<<
Y
*************************************************************************************************************************************************************************

i continued with fe tests as is. Here is the complete output log.
[attachment "fe_test_output.zip" deleted by Valencia
Serrao/Austin/Contr/IBM]

Cluster logs: [attachment "cluster_logs.7z" deleted by Valencia
Serrao/Austin/Contr/IBM]

Kindly guide me on the same.

Regards,
Valencia
----- Forwarded by Valencia Serrao/Austin/Contr/IBM on 04/29/2016 10:57 AM
-----

From:	Sudarshan Jagadale/Austin/Contr/IBM
To:	Valencia Serrao/Austin/Contr/IBM@IBMUS
Date:	04/29/2016 10:49 AM
Subject:	Fw: Issues with generating testdata for Impala


FYI
Thanks and Regards
Sudarshan Jagadale
Power Open Source Solutions
----- Forwarded by Sudarshan Jagadale/Austin/Contr/IBM on 04/29/2016 10:48
AM -----

From:	Alex Behm <alex.behm@cloudera.com>
To:	dev@impala.incubator.apache.org
Cc:	Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha
            Panpaliya/Austin/Contr/IBM@IBMUS
Date:	04/28/2016 09:34 PM
Subject:	Re: Issues with generating testdata for Impala



Hi Valencia,

sorry I did not get the attachment. Would you be able to tar.gz and attach
the whole cluster_logs directory?

Alex

On Thu, Apr 28, 2016 at 6:23 AM, Valencia Serrao <vserrao@us.ibm.com>
wrote:
  Hi Alex,

  I tried building impala again with the following:
  HDFS CDH 5.7.0 (
  http://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_57.html#topic_3
  )
  HBASE CDH 5.7.0 SNAPSHOT (
  http://archive.cloudera.com/cdh5/cdh/5/hbase-1.2.0-cdh5.7.0.tar.gz )
  - this required to patch in a fix (
  https://issues.apache.org/jira/secure/attachment/12792536/HBASE-15322-branch-1.2.patch
  )
  HIVE CDH 5.8.0 SNAPSHOT

  With the above combination, i'm able to move past the exception and also
  have the RegionServer service up and running. However, it now gives error
  as below:

  ********************************************************************************************************************

  (load-functional-query-exhaustive-impala-generated-text-none-none.sql):
  CREATE EXTERNAL TABLE IF NOT EXISTS functional.decimal_tbl (
  d1 DECIMAL,
  d2 DECIMAL(10, 0),
  d3 DECIMAL(20, 10),
  d4 DECIMAL(38, 38),
  d5 DECIMAL(10, 5))
  PARTITIONED BY (d6 DECIMAL(9, 0))
  ROW FORMAT delimited fields terminated by ','
  STORED AS TEXTFILE
  LOCATION '/test-warehouse/decimal_tbl'

  (load-functional-query-exhaustive-impala-generated-text-none-none.sql):
  USE functional

  (load-functional-query-exhaustive-impala-generated-text-none-none.sql):
  ALTER TABLE decimal_tbl ADD IF NOT EXISTS PARTITION(d6=1)

  Data Loading from Impala failed with error: ImpalaBeeswaxException:
  INNER EXCEPTION: <class
  'impala._thrift_gen.beeswax.ttypes.BeeswaxException'>
  MESSAGE:
  Error: null
  ******************************************************************************************************************


  Here is the complete log for the same. (See attached file:
  data-load-functional-exhaustive.log)

  It would great if you could guide me on this issue, so i could proceed
  with the fe tests.

  Still awaiting link to the source code of HDFS CDH 5.8.0

  Regards,
  Valencia







Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message