hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang" <xzh...@cloudera.com>
Subject Re: Review Request 23006: Escape control characters for explain result
Date Fri, 27 Jun 2014 13:56:34 GMT


> On June 26, 2014, 6:51 p.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java, line 195
> > <https://reviews.apache.org/r/23006/diff/1/?file=618089#file618089line195>
> >
> >     For my understanding, why cannot we just simply replace 0x00 with a different
character such as ' '? Why we are dealing with quotes and commas? Can you give an example
what's transformed to what?
> 
> Navis Ryu wrote:
>     With comments with spaces, just replacing 0x00 into a space would be confusing, IMHO.

>     
>     comment 10x00comment 20x00 will be printed like 'comment 1','comment 2',''. For 0x000x00,
null will be returned (nothing in comments).
> 
> Xuefu Zhang wrote:
>     I don't know how 0x00 came into the picutre at the first place. It seems reasonable
to me that comment should contain nothing but a string.
> 
> Navis Ryu wrote:
>     Like other schema information (column names and types), it's comments for multiple
columns, delimited by the special character, 0x00.
> 
> Xuefu Zhang wrote:
>     It seems anyway we cannot avoid separator conflicts completely if we replace 0x00
with any character. For instance, if a comment contain ' or , then your solution can also
lead to confusion. If that's the case, I'm wondering if replacing 0x00 with some uncommonly
used character such as | would make it simpler. That is:
>     
>     comment10x00comment20x00 => comment1|comment2|
>     0x000x00 => ||
>     
>     Of course, this is very subjective.
> 
> Navis Ryu wrote:
>     Agree. I thought quote+',' would be enough but it seemed better to use '|'. How about
null on no comment in any columns? Is '||' better option?

Using 0x00 as the separator doesn't solve the null problem either, so I think we don't have
to address that in this JIRA either. This is just for comments, so probably it doesn't really
matter much.


- Xuefu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23006/#review46773
-----------------------------------------------------------


On June 26, 2014, 9:05 a.m., Navis Ryu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23006/
> -----------------------------------------------------------
> 
> (Updated June 26, 2014, 9:05 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-7024
>     https://issues.apache.org/jira/browse/HIVE-7024
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Comments for columns are now delimited by 0x00, which is binary and make git refuse to
make proper diff file.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java 92545d8 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java 1149bda 
>   ql/src/test/results/clientpositive/alter_partition_coltype.q.out e86cc06 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out c7d58f6 
>   ql/src/test/results/clientpositive/annotate_stats_groupby.q.out 6f72964 
>   ql/src/test/results/clientpositive/annotate_stats_join.q.out cc816c8 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out a0b4602 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out 97e9473 
>   ql/src/test/results/clientpositive/annotate_stats_table.q.out bb2d18c 
>   ql/src/test/results/clientpositive/annotate_stats_union.q.out 6d179b6 
>   ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 3f4f902 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out 72640df 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out c660cd0 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 4abda32 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out 52a3194 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out d807791 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out 35e0a30 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out af3d9d6 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out 05ef5d8 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out e423d14 
>   ql/src/test/results/clientpositive/binary_output_format.q.out 294aabb 
>   ql/src/test/results/clientpositive/bucket1.q.out f3eb15c 
>   ql/src/test/results/clientpositive/bucket2.q.out 9a22160 
>   ql/src/test/results/clientpositive/bucket3.q.out 8fa9c7b 
>   ql/src/test/results/clientpositive/bucket4.q.out 032272b 
>   ql/src/test/results/clientpositive/bucket5.q.out d19fbe5 
>   ql/src/test/results/clientpositive/bucket_map_join_1.q.out 8674a6c 
>   ql/src/test/results/clientpositive/bucket_map_join_2.q.out 8a5984d 
>   ql/src/test/results/clientpositive/bucketcontext_1.q.out 1513515 
>   ql/src/test/results/clientpositive/bucketcontext_2.q.out d18a9be 
>   ql/src/test/results/clientpositive/bucketcontext_3.q.out e12c155 
>   ql/src/test/results/clientpositive/bucketcontext_4.q.out 77b4882 
>   ql/src/test/results/clientpositive/bucketcontext_5.q.out fa1cfc5 
>   ql/src/test/results/clientpositive/bucketcontext_6.q.out aac66f8 
>   ql/src/test/results/clientpositive/bucketcontext_7.q.out 78c4f94 
>   ql/src/test/results/clientpositive/bucketcontext_8.q.out ad7fec9 
>   ql/src/test/results/clientpositive/bucketmapjoin1.q.out 10f1af4 
>   ql/src/test/results/clientpositive/bucketmapjoin10.q.out 88ecf40 
>   ql/src/test/results/clientpositive/bucketmapjoin11.q.out 4ee1fa0 
>   ql/src/test/results/clientpositive/bucketmapjoin12.q.out 9253f4a 
>   ql/src/test/results/clientpositive/bucketmapjoin13.q.out b380fab 
>   ql/src/test/results/clientpositive/bucketmapjoin2.q.out 297412f 
>   ql/src/test/results/clientpositive/bucketmapjoin3.q.out 7f307a0 
>   ql/src/test/results/clientpositive/bucketmapjoin4.q.out f0f9aee 
>   ql/src/test/results/clientpositive/bucketmapjoin5.q.out 79e1c3d 
>   ql/src/test/results/clientpositive/bucketmapjoin7.q.out 76baf50 
>   ql/src/test/results/clientpositive/bucketmapjoin8.q.out 94fdbde 
>   ql/src/test/results/clientpositive/bucketmapjoin9.q.out c9f4c17 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 751e32f 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 3eb70d1 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out 34abe4f 
>   ql/src/test/results/clientpositive/columnstats_partlvl.q.out 6128770 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 35af846 
>   ql/src/test/results/clientpositive/ctas.q.out 0040f3c 
>   ql/src/test/results/clientpositive/disable_merge_for_bucketing.q.out 4f9bb94 
>   ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out 8b7afae 
>   ql/src/test/results/clientpositive/filter_join_breaktask.q.out b379f86 
>   ql/src/test/results/clientpositive/groupby_map_ppr.q.out c7ca521 
>   ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out 00e2b6d 
>   ql/src/test/results/clientpositive/groupby_ppr.q.out 57e886d 
>   ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out f8073ff 
>   ql/src/test/results/clientpositive/groupby_sort_1_23.q.out 38a0678 
>   ql/src/test/results/clientpositive/groupby_sort_6.q.out ca5ad8f 
>   ql/src/test/results/clientpositive/groupby_sort_skew_1_23.q.out ac54e7d 
>   ql/src/test/results/clientpositive/input23.q.out 1891aae 
>   ql/src/test/results/clientpositive/input42.q.out 8f52610 
>   ql/src/test/results/clientpositive/input_part1.q.out 22191f1 
>   ql/src/test/results/clientpositive/input_part2.q.out 9987769 
>   ql/src/test/results/clientpositive/input_part7.q.out 19985d6 
>   ql/src/test/results/clientpositive/input_part9.q.out 7ef66b0 
>   ql/src/test/results/clientpositive/join17.q.out 76741bb 
>   ql/src/test/results/clientpositive/join26.q.out bf8cf57 
>   ql/src/test/results/clientpositive/join32.q.out ff0d7cc 
>   ql/src/test/results/clientpositive/join32_lessSize.q.out 04ed6b2 
>   ql/src/test/results/clientpositive/join33.q.out ff0d7cc 
>   ql/src/test/results/clientpositive/join34.q.out b52777a 
>   ql/src/test/results/clientpositive/join35.q.out 3053af7 
>   ql/src/test/results/clientpositive/join9.q.out 16ce5a2 
>   ql/src/test/results/clientpositive/join_filters_overlap.q.out 94ca28e 
>   ql/src/test/results/clientpositive/join_map_ppr.q.out 51fb6c6 
>   ql/src/test/results/clientpositive/list_bucket_dml_1.q.out b48c36a 
>   ql/src/test/results/clientpositive/list_bucket_dml_10.q.out 5bfe938 
>   ql/src/test/results/clientpositive/list_bucket_dml_11.q.out acf80aa 
>   ql/src/test/results/clientpositive/list_bucket_dml_12.q.out fa5bc92 
>   ql/src/test/results/clientpositive/list_bucket_dml_13.q.out a5813a6 
>   ql/src/test/results/clientpositive/list_bucket_dml_14.q.out 730fa9e 
>   ql/src/test/results/clientpositive/list_bucket_dml_2.q.out 5462629 
>   ql/src/test/results/clientpositive/list_bucket_dml_3.q.out b786fc8 
>   ql/src/test/results/clientpositive/list_bucket_dml_4.q.out 083568a 
>   ql/src/test/results/clientpositive/list_bucket_dml_5.q.out 2db238e 
>   ql/src/test/results/clientpositive/list_bucket_dml_6.q.out c1ababc 
>   ql/src/test/results/clientpositive/list_bucket_dml_7.q.out 4c905e9 
>   ql/src/test/results/clientpositive/list_bucket_dml_8.q.out ff94687 
>   ql/src/test/results/clientpositive/list_bucket_dml_9.q.out 41872d7 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out 81c519e 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 9caaece 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out d4f833b 
>   ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 21fac22 
>   ql/src/test/results/clientpositive/list_bucket_query_oneskew_3.q.out 01a8081 
>   ql/src/test/results/clientpositive/load_dyn_part8.q.out 2299667 
>   ql/src/test/results/clientpositive/louter_join_ppr.q.out 3dec6a6 
>   ql/src/test/results/clientpositive/merge3.q.out 96f675d 
>   ql/src/test/results/clientpositive/metadataonly1.q.out f08ef9e 
>   ql/src/test/results/clientpositive/outer_join_ppr.q.out c9e3fbc 
>   ql/src/test/results/clientpositive/pcr.q.out b74dfbf 
>   ql/src/test/results/clientpositive/ppd_join_filter.q.out 95855c9 
>   ql/src/test/results/clientpositive/ppd_union_view.q.out 82406be 
>   ql/src/test/results/clientpositive/ppd_vc.q.out 928986f 
>   ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out 632fd4b 
>   ql/src/test/results/clientpositive/push_or.q.out 132a160 
>   ql/src/test/results/clientpositive/rand_partitionpruner1.q.out 1c49a25 
>   ql/src/test/results/clientpositive/rand_partitionpruner2.q.out 557dd9d 
>   ql/src/test/results/clientpositive/rand_partitionpruner3.q.out 93df215 
>   ql/src/test/results/clientpositive/reduce_deduplicate.q.out cab7215 
>   ql/src/test/results/clientpositive/regexp_extract.q.out 6218e77 
>   ql/src/test/results/clientpositive/router_join_ppr.q.out aa2789f 
>   ql/src/test/results/clientpositive/sample1.q.out e6adac9 
>   ql/src/test/results/clientpositive/sample10.q.out f72d35e 
>   ql/src/test/results/clientpositive/sample2.q.out 3f8a4cc 
>   ql/src/test/results/clientpositive/sample4.q.out e6ee033 
>   ql/src/test/results/clientpositive/sample5.q.out 57accb3 
>   ql/src/test/results/clientpositive/sample6.q.out 45d355f 
>   ql/src/test/results/clientpositive/sample7.q.out bc5a023 
>   ql/src/test/results/clientpositive/sample8.q.out 62c7d93 
>   ql/src/test/results/clientpositive/sample9.q.out 1dba5e1 
>   ql/src/test/results/clientpositive/serde_user_properties.q.out 8022177 
>   ql/src/test/results/clientpositive/smb_mapjoin_11.q.out d59b801 
>   ql/src/test/results/clientpositive/smb_mapjoin_12.q.out ab3f93f 
>   ql/src/test/results/clientpositive/smb_mapjoin_13.q.out e033da1 
>   ql/src/test/results/clientpositive/smb_mapjoin_15.q.out 2a4bc28 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out 5ff45d9 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out 6fdf738 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out 08b9d1e 
>   ql/src/test/results/clientpositive/stats0.q.out 712a474 
>   ql/src/test/results/clientpositive/stats11.q.out c5531c5 
>   ql/src/test/results/clientpositive/stats12.q.out 7afa17c 
>   ql/src/test/results/clientpositive/stats13.q.out 3034b15 
>   ql/src/test/results/clientpositive/tez/bucket2.q.out 3e7746f 
>   ql/src/test/results/clientpositive/tez/bucket3.q.out cabf30a 
>   ql/src/test/results/clientpositive/tez/bucket4.q.out 9d1e843 
>   ql/src/test/results/clientpositive/tez/ctas.q.out a58e166 
>   ql/src/test/results/clientpositive/tez/disable_merge_for_bucketing.q.out 0001443 
>   ql/src/test/results/clientpositive/tez/filter_join_breaktask.q.out 5e32403 
>   ql/src/test/results/clientpositive/tez/sample1.q.out 6f8dfe9 
>   ql/src/test/results/clientpositive/tez/transform_ppr1.q.out 719ae5b 
>   ql/src/test/results/clientpositive/tez/transform_ppr2.q.out c82137c 
>   ql/src/test/results/clientpositive/transform_ppr1.q.out 88baa2c 
>   ql/src/test/results/clientpositive/transform_ppr2.q.out ec52c91 
>   ql/src/test/results/clientpositive/truncate_column_list_bucket.q.out be8fafc 
>   ql/src/test/results/clientpositive/udf_explode.q.out 4eeedeb 
>   ql/src/test/results/clientpositive/udf_reflect2.q.out 6b19277 
>   ql/src/test/results/clientpositive/udtf_explode.q.out ae95907 
>   ql/src/test/results/clientpositive/union22.q.out 4422e71 
>   ql/src/test/results/clientpositive/union24.q.out dda55ca 
>   ql/src/test/results/clientpositive/union_ppr.q.out 3016b61 
> 
> Diff: https://reviews.apache.org/r/23006/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Navis Ryu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message