hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abdullah Yousufi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14373) Add integration tests for hive on S3
Date Fri, 19 Aug 2016 20:12:21 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428752#comment-15428752
] 

Abdullah Yousufi commented on HIVE-14373:
-----------------------------------------

[~yalovyyi] So one issue right now I'm noticing due is that after the tests are finished,
the source and output directories still exist on the S3 bucket. Any thoughts on how to resolve
this? One option I'm considering is to create the test ables on the bucket's root directory,
so that there are no left over directories. In this case, the timestamp could be appended
to the beginning of the table name to prevent issues with parallel test execution.

> Add integration tests for hive on S3
> ------------------------------------
>
>                 Key: HIVE-14373
>                 URL: https://issues.apache.org/jira/browse/HIVE-14373
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergio Peña
>            Assignee: Abdullah Yousufi
>         Attachments: HIVE-14373.02.patch, HIVE-14373.03.patch, HIVE-14373.patch
>
>
> With Hive doing improvements to run on S3, it would be ideal to have better integration
testing on S3.
> These S3 tests won't be able to be executed by HiveQA because it will need Amazon credentials.
We need to write suite based on ideas from the Hadoop project where:
> - an xml file is provided with S3 credentials
> - a committer must run these tests manually to verify it works
> - the xml file should not be part of the commit, and hiveqa should not run these tests.
> https://wiki.apache.org/hadoop/HowToContribute#Submitting_patches_against_object_stores_such_as_Amazon_S3.2C_OpenStack_Swift_and_Microsoft_Azure



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message