From user-return-473-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Sat May 30 15:07:46 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4356F180638 for ; Sat, 30 May 2020 17:07:46 +0200 (CEST) Received: (qmail 17638 invoked by uid 500); 30 May 2020 15:07:45 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 17628 invoked by uid 99); 30 May 2020 15:07:45 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 30 May 2020 15:07:45 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B822F1A412F for ; Sat, 30 May 2020 15:07:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.2 X-Spam-Level: X-Spam-Status: No, score=-0.2 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id dmbEbX7qK2Xx for ; Sat, 30 May 2020 15:07:43 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::d30; helo=mail-io1-xd30.google.com; envelope-from=wesmckinn@gmail.com; receiver= Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 048C27DC92 for ; Sat, 30 May 2020 15:07:42 +0000 (UTC) Received: by mail-io1-xd30.google.com with SMTP id k18so2515936ion.0 for ; Sat, 30 May 2020 08:07:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=jTkr5wK3Qu3Oog96qDh9cqT9K72pyxPAzDxxHkDN1zc=; b=t0EaAB09x3WYHI2ND0+XAq3Vgro8eHjsrdswtPaJPIqjDnpWRBfAuK6e5jDkspMwif l1FttbmUmQ3lLQIc/qwpazkW890nw0iulDGb73nKruY1nTsRNE5CEPB+n2Hk8X9jNwxv DiDD7Qqg+PAKBZDF0uHog7Tz94Ql7KFFcPu+xDhkkFJ+n2Y8dzsFJ+jlECNIXufQ08A7 4H915BO1yVle6EMycLGItmBi6zknizYyft1KG4Mfp5ssc7ENWzpFpqiUSM6zD0wej5j1 0GZuMuAyEiN43Wsf6rQfrzEIXYcOzXbeCz9tdojn81oeVVWbqvuGft33McUl45o9pKPW HSEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=jTkr5wK3Qu3Oog96qDh9cqT9K72pyxPAzDxxHkDN1zc=; b=aZV5iKQddpvY0NzMMiCt6bA2CXrYmzJ02c/+U9X6JzFH8OnyGrcq9vAha59cA4MEsN Qw8iTNroH7o/2ng5Xw8RJrHemSY5gDUlLcOIeQU2fU4AiBJpXULspM9V8BvgLJcgChCF kTzSOqR/dNbIp8HCTFIwSq1xAgxueo7TE+jScmCXSbnADg6l0bZV0eyLYbUrMvVfw9tT uc/y2i+GRhCr8IZCcMEr0zU+gr5OnMH8g6FNxtMPT61HOta1uSUuPuLO2YpDNmPt61Nv KYzuVa2+AQxmB21cB944pk5p1RlMWgC4+ii7ppeUUXTP49eQpMIf4FA1F7EWilqcEga+ r1TQ== X-Gm-Message-State: AOAM531eiSbz75ZkFlneMsCv1VDIjIqCuDdI+/w/KbdQaInqOhX2GvQ9 4zPfd99cAiKVWTWQYByta87SGSHPdW3WiAh/rwpH6GT2 X-Google-Smtp-Source: ABdhPJxCP3ePJnq++j7XIY23B7RJ6gIVePm21oZecSSFEI0nhBrfXvnbwH08WxVwPCW+f1kX3yGVCnMmdif8kQ08H1Y= X-Received: by 2002:a5d:8c95:: with SMTP id g21mr11311599ion.72.1590851261225; Sat, 30 May 2020 08:07:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Wes McKinney Date: Sat, 30 May 2020 10:07:05 -0500 Message-ID: Subject: Re: check whether pandas type is convertible to arrow type To: user@arrow.apache.org Content-Type: text/plain; charset="UTF-8" I don't think there is specifically (one could be added in theory). Is the goal to determine whether `pyarrow.array(pandas_object)` will succeed or not, or something else? Since a lot of pandas data is opaquely represented with object dtype it can be tricky unless you want to go to the expense of using `pandas.lib.infer_dtype` to determine the effective logical type of the values. On Fri, May 29, 2020 at 4:18 PM Sandy Ryza wrote: > > Hi all, > > If I have a pandas dtype and an arrow type, is there a pyarrow API that allows me to check whether the pandas dtype is convertible to the arrow type? > > It seems like "arrow_type.to_pandas_dtype() == pandas_dtype" would work in most cases, because pandas dtypes tend to be at least as wide as equivalent arrow types, but I'm wondering whether there's something more principled. > > Any help much appreciated, > Sandy >