Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4478B200BC2 for ; Thu, 17 Nov 2016 20:03:19 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 4330A160B0B; Thu, 17 Nov 2016 19:03:19 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8BBF7160AD8 for ; Thu, 17 Nov 2016 20:03:18 +0100 (CET) Received: (qmail 68117 invoked by uid 500); 17 Nov 2016 19:03:17 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 68108 invoked by uid 99); 17 Nov 2016 19:03:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Nov 2016 19:03:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 666D61A0040 for ; Thu, 17 Nov 2016 19:03:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -7.019 X-Spam-Level: X-Spam-Status: No, score=-7.019 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id L-aMMi1uegvv for ; Thu, 17 Nov 2016 19:03:16 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id 1C60A5FAE7 for ; Thu, 17 Nov 2016 19:03:16 +0000 (UTC) Received: (qmail 67475 invoked by uid 99); 17 Nov 2016 19:02:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Nov 2016 19:02:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 85CFC2C4C78 for ; Thu, 17 Nov 2016 19:02:58 +0000 (UTC) Date: Thu, 17 Nov 2016 19:02:58 +0000 (UTC) From: "Chris Riccomini (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Closed] (AIRFLOW-514) HiveCli hook should be able to load a pandas DataFrame MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 17 Nov 2016 19:03:19 -0000 [ https://issues.apache.org/jira/browse/AIRFLOW-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini closed AIRFLOW-514. ----------------------------------- Resolution: Done Fix Version/s: Airflow 1.8 > HiveCli hook should be able to load a pandas DataFrame > ------------------------------------------------------- > > Key: AIRFLOW-514 > URL: https://issues.apache.org/jira/browse/AIRFLOW-514 > Project: Apache Airflow > Issue Type: Improvement > Reporter: Daniel Frank > Assignee: Daniel Frank > Priority: Minor > Fix For: Airflow 1.8 > > > Currently the hive cli hook can load_df, which returns a pandas.DataFrame. Many of our workflows involve retrieving a hive table in pandas.DataFrame form, modifying it and saving it (perhaps elsewhere). In order to save the dataframe we have to manually translate the types, save to disk and run load_file() which is repetitive and tedious. This workflow could be easily automated with a load_df method for HiveCliHook -- This message was sent by Atlassian JIRA (v6.3.4#6332)