From common-issues-return-155152-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue Jul 24 03:30:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 247CC180677 for ; Tue, 24 Jul 2018 03:30:04 +0200 (CEST) Received: (qmail 73552 invoked by uid 500); 24 Jul 2018 01:30:04 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 73511 invoked by uid 99); 24 Jul 2018 01:30:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jul 2018 01:30:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id B94F7C0E70 for ; Tue, 24 Jul 2018 01:30:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 79ByCbBpmrgC for ; Tue, 24 Jul 2018 01:30:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 432605F27E for ; Tue, 24 Jul 2018 01:30:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 8530DE0178 for ; Tue, 24 Jul 2018 01:30:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 44A2423F97 for ; Tue, 24 Jul 2018 01:30:00 +0000 (UTC) Date: Tue, 24 Jul 2018 01:30:00 +0000 (UTC) From: "Steve Loughran (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-15544) ABFS: validate packing, transient classpath, hadoop fs CLI MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553664#comment-16553664 ] Steve Loughran commented on HADOOP-15544: ----------------------------------------- Also worth exploring is my cloudstore/storediag module, which I keep to debug this stuff: [https://github.com/steveloughran/cloudstore/releases] {code} bin/hadoop -jar cloudstore-2.8.jar abfs://storeuri/path {code} That attempts to load the abfs, look for (and log) classes/dependencies it knows of, then does some basic IO. It's intended to be for remote debugging of support calls, so can be expanded to do stuff about hostname and proxy lookup, once the details are known > ABFS: validate packing, transient classpath, hadoop fs CLI > ---------------------------------------------------------- > > Key: HADOOP-15544 > URL: https://issues.apache.org/jira/browse/HADOOP-15544 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure > Affects Versions: 3.2 > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Major > Attachments: dependencies.txt > > > Validate the packaging and dependencies of ABFS > * hadoop-cloud-storage artifact to export everything needed > * {{hadoop fs -ls abfs://path}} to work in ASF distributions > * check transient CP (e.g spark) > Spark master;s hadoop-cloud module depends on hadoop-cloud-storage if you build with the hadoop-3.1 profile, so it should automatically get in there. Just need to check that it picks it up too -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org