From issues-return-184111-archive-asf-public=cust-asf.ponee.io@spark.apache.org Mon Feb 5 12:26:12 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 026DE180647 for ; Mon, 5 Feb 2018 12:26:12 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E6771160C4B; Mon, 5 Feb 2018 11:26:11 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E19C2160C46 for ; Mon, 5 Feb 2018 12:26:10 +0100 (CET) Received: (qmail 38384 invoked by uid 500); 5 Feb 2018 11:26:10 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 38375 invoked by uid 99); 5 Feb 2018 11:26:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Feb 2018 11:26:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id A493BC0118 for ; Mon, 5 Feb 2018 11:26:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id vkzFJoqsFzWa for ; Mon, 5 Feb 2018 11:26:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 43C3A5F475 for ; Mon, 5 Feb 2018 11:26:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 944FEE031C for ; Mon, 5 Feb 2018 11:26:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 8896721E87 for ; Mon, 5 Feb 2018 11:26:01 +0000 (UTC) Date: Mon, 5 Feb 2018 11:26:01 +0000 (UTC) From: "Takeshi Yamamuro (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SPARK-20425) Support an extended display mode to print a column data per line MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SPARK-20425?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D163= 52271#comment-16352271 ]=20 Takeshi Yamamuro commented on SPARK-20425: ------------------------------------------ yea, ok. If nobody takes on this, I'll make a pr. > Support an extended display mode to print a column data per line > ---------------------------------------------------------------- > > Key: SPARK-20425 > URL: https://issues.apache.org/jira/browse/SPARK-20425 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.1.0 > Reporter: Takeshi Yamamuro > Assignee: Takeshi Yamamuro > Priority: Minor > Fix For: 2.3.0 > > > In the master, when printing Dataset with many columns, the readability i= s too low like; > {code} > scala> val df =3D spark.range(100).selectExpr((0 until 100).map(i =3D> s"= rand() AS c$i"): _*) > scala> df.show(3, 0) > +------------------+------------------+------------------+---------------= ----+------------------+------------------+-------------------+------------= ------+------------------+------------------+------------------+-----------= --------+------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+-------------------+--= ----------------+-------------------+-------------------+------------------= -+--------------------+-------------------+------------------+-------------= ------+--------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+------------------+---= ----------------+-------------------+-------------------+------------------= +--------------------+--------------------+-------------------+------------= -------+-------------------+-------------------+-------------------+-------= ------------+--------------------+-------------------+-------------------+-= ------------------+-------------------+------------------+-----------------= -+-------------------+-------------------+------------------+--------------= -----+------------------+------------------+-----------------+-------------= ------+-------------------+------------------+-------------------+---------= ---------+-------------------+-------------------+-------------------+-----= -------------+-------------------+------------------+-------------------+--= -----------------+-------------------+-------------------+-----------------= --+-------------------+-------------------+-------------------+------------= ------+-------------------+-------------------+------------------+---------= ---------+------------------+-------------------+------------------+-------= ------------+------------------+-------------------+-------------------+---= ----------------+ > |c0 |c1 |c2 |c3 = |c4 |c5 |c6 |c7 = |c8 |c9 |c10 |c11 = |c12 |c13 |c14 |c15 = |c16 |c17 |c18 |c19 = |c20 |c21 |c22 |c2= 3 |c24 |c25 |c26 = |c27 |c28 |c29 |c30 = |c31 |c32 |c33 |c34 = |c35 |c36 |c37 |c38 = |c39 |c40 |c41 |c42= |c43 |c44 |c45 = |c46 |c47 |c48 |c49 = |c50 |c51 |c52 |c53 = |c54 |c55 |c56 |c= 57 |c58 |c59 |c60 = |c61 |c62 |c63 |c64 = |c65 |c66 |c67 |c68 = |c69 |c70 |c71 |c72 = |c73 |c74 |c75 |c76 = |c77 |c78 |c79 |c8= 0 |c81 |c82 |c83 = |c84 |c85 |c86 |c87 = |c88 |c89 |c90 |c91 = |c92 |c93 |c94 |c95 = |c96 |c97 |c98 |c99= | > +------------------+------------------+------------------+---------------= ----+------------------+------------------+-------------------+------------= ------+------------------+------------------+------------------+-----------= --------+------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+-------------------+--= ----------------+-------------------+-------------------+------------------= -+--------------------+-------------------+------------------+-------------= ------+--------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+------------------+---= ----------------+-------------------+-------------------+------------------= +--------------------+--------------------+-------------------+------------= -------+-------------------+-------------------+-------------------+-------= ------------+--------------------+-------------------+-------------------+-= ------------------+-------------------+------------------+-----------------= -+-------------------+-------------------+------------------+--------------= -----+------------------+------------------+-----------------+-------------= ------+-------------------+------------------+-------------------+---------= ---------+-------------------+-------------------+-------------------+-----= -------------+-------------------+------------------+-------------------+--= -----------------+-------------------+-------------------+-----------------= --+-------------------+-------------------+-------------------+------------= ------+-------------------+-------------------+------------------+---------= ---------+------------------+-------------------+------------------+-------= ------------+------------------+-------------------+-------------------+---= ----------------+ > |0.6306087152476858|0.9174349686288383|0.5511324165035159|0.3320844128641= 819 |0.7738486877101489|0.2154915886962553|0.4754997600674299 |0.9227806392= 80355 |0.7136894772661909|0.2277580838165979|0.5926874459847249|0.403114083= 92226633|0.467830264333843 |0.8330466896984213|0.1893258482389527|0.6320849= 515511165 |0.7530911056912044 |0.06700254871955424|0.370528597355559 |0.275= 5437445193154|0.23704391110980128|0.8067400174905822|0.13597793616251852|0.= 1708888820162453|0.01672725007605702|0.983118121881555 |0.2504019562862992= 4|0.060537253723083384|0.20000530582637488|0.3400572407133511|0.93756894333= 22597 |0.057039316954370256|0.8053269714347623|0.5247817572228813|0.2841930= 8820527944|0.9798908885194533 |0.31805988175678146|0.7034448027077574|0.540= 0575751346084|0.25336322371116216|0.9361634546853429|0.6118681368289798|0.6= 295081549153907 |0.13417468943957422|0.41617137072255794|0.7267230869252035= |0.023792726137561115|0.5776157058356362 |0.04884204913195467|0.2672871610= 3441275|0.646680370807925 |0.9782712690657244 |0.16434031314818154|0.20985= 522381321275|0.24739842475440077 |0.26335189682977334|0.19604841662422068|0= .10742950487300651|0.20283136488091502|0.3100312319723688|0.886959006630645= |0.25157102269776244|0.34428775168410786|0.3500506818575777|0.378114244191= 2052 |0.8560316444386715|0.4737104888956839|0.735903101602148|0.02236617130= 529006|0.8769074095835873 |0.2001426662503153|0.5534032319238532 |0.7289496= 620397098|0.41955191309992157|0.9337700133660436 |0.34059094378451005|0.641= 9144759403556|0.08167496930341167|0.9947099478497635|0.48010888605366586|0.= 22314796858167918|0.17786598882331306|0.7351521162297135 |0.542205717002009= 5 |0.9521927872726792 |0.7459825486368227 |0.40907708791990627|0.8903819313= 311575|0.7251413746923618 |0.2977174938745204 |0.9515209660203555|0.9375968= 604766713|0.5087851740042524|0.4255237544908751 |0.8023768698664653|0.48003= 189618006703|0.1775841829745185|0.09050775629268382|0.6743909291138167 |0.2= 498415755876865 | > |0.6866473844170801|0.4774360641212433|0.631696201340726 |0.3397911302146= 8343|0.5663049010847052|0.7280190472258865|0.41370958502324806|0.9977433873= 622218|0.7671957338989901|0.2788708556233931|0.3355106391656496|0.884789523= 19287 |0.0333974166999893|0.6061744715862606|0.9617779139652359|0.2248495= 4822341863|0.12770906021550898|0.5577789629508672 |0.2877649024640704|0.556= 6577406549361|0.9334933255278052 |0.9166720585157266|0.9689249324600591 |0.= 6367502457478598|0.7993572745928459 |0.23213222324218108|0.1192828405415413= 7|0.6173493362456599 |0.0505122058694798 |0.9050228629552983|0.17112767911= 121707|0.47395598348370005 |0.5820498657823081|0.6241124650645072|0.1858725= 8258036776|0.14987593554122225|0.3079446253653946 |0.9414228822867968|0.836= 2276265462365|0.9155655305576353 |0.5121559807153562|0.8963362656525707|0.2= 2765970274318037|0.8177039187132797 |0.8190326635933787 |0.5256005177032199= |0.8167598457269669 |0.030936807130934496|0.6733006585281015 |0.4208049626= 816347 |0.24603085738518538|0.22719198954208153|0.1622280557565281 |0.22217= 325159218038|0.014684419513742553|0.08987111517447499|0.2157764759142622 |0= .8223414104088321 |0.4868624404491777 |0.4016191733088167|0.616928190688926= 3|0.15603611040433385|0.18289285085714913|0.9538408988218972|0.150371548652= 95121|0.5364516961987454|0.8077254873163031|0.712600478545675|0.72774772410= 03857 |0.19822912960348305|0.8305051199208777|0.18631911396566114|0.8909532= 487898342|0.3470409226992506 |0.35306974180587636|0.9107058868891469 |0.332= 1327206004986|0.48952332459050607|0.3630403307479373|0.5400046826340376 |0.= 5387377194310529 |0.42860539421837585|0.23214101630985995|0.214389688397948= 47|0.15370603160082352|0.04355605642700022|0.6096006707067466 |0.6933354157= 094292|0.06302172470859002|0.03174631856164001|0.664243581650643 |0.7833239= 547446621|0.696884598352864 |0.34626385933237736|0.9263495598791336|0.40481= 8892816584 |0.2085585394755507|0.6150004897990109 |0.05391193524302473|0.2= 8188484028329097| > +------------------+------------------+------------------+---------------= ----+------------------+------------------+-------------------+------------= ------+------------------+------------------+------------------+-----------= --------+------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+-------------------+--= ----------------+-------------------+-------------------+------------------= -+--------------------+-------------------+------------------+-------------= ------+--------------------+------------------+------------------+---------= ----------+-------------------+-------------------+------------------+-----= -------------+-------------------+------------------+------------------+---= ----------------+-------------------+-------------------+------------------= +--------------------+--------------------+-------------------+------------= -------+-------------------+-------------------+-------------------+-------= ------------+--------------------+-------------------+-------------------+-= ------------------+-------------------+------------------+-----------------= -+-------------------+-------------------+------------------+--------------= -----+------------------+------------------+-----------------+-------------= ------+-------------------+------------------+-------------------+---------= ---------+-------------------+-------------------+-------------------+-----= -------------+-------------------+------------------+-------------------+--= -----------------+-------------------+-------------------+-----------------= --+-------------------+-------------------+-------------------+------------= ------+-------------------+-------------------+------------------+---------= ---------+------------------+-------------------+------------------+-------= ------------+------------------+-------------------+-------------------+---= ----------------+ > only showing top 2 rows > {code} > psql, CLI for PostgreSQL supports an extended display mode for this case = like: > http://stackoverflow.com/questions/9604723/alternate-output-format-for-ps= ql > {code} > -RECORD 0------------------- > c0 | 0.6306087152476858 > c1 | 0.9174349686288383 > c2 | 0.5511324165035159 > ... > c98 | 0.05391193524302473 =20 > c99 | 0.28188484028329097 =20 > -RECORD 1------------------- > c0 | 0.6866473844170801 > c1 | 0.4774360641212433 > c2 | 0.631696201340726 > ... > c98 | 0.05391193524302473 > c99 | 0.28188484028329097 > only showing top 2 rows > {code} > I feel it is useful to support this mode in spark-shell. > Thought? -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org