carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Shahid Khan (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (CARBONDATA-660) Bad Records Logs and Raw CSVs should get display under segment id instead of Tasks id
Date Thu, 19 Jan 2017 18:26:26 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mohammad Shahid Khan reassigned CARBONDATA-660:
-----------------------------------------------

    Assignee: Mohammad Shahid Khan

> Bad Records Logs and Raw CSVs should get display under segment id instead of Tasks id
> -------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-660
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-660
>             Project: CarbonData
>          Issue Type: Improvement
>          Components: data-load
>            Reporter: Priyal Sachdeva
>            Assignee: Mohammad Shahid Khan
>            Priority: Minor
>
> create table if not exists Badrecords_test (imei string,AMSize int) STORED BY 'org.apache.carbondata.format';
>  LOAD DATA INPATH 'hdfs://hacluster/CSVs/bad_records.csv' into table Badrecords_test
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='REDIRECT','FILEHEADER'='imei,AMSize');
> Bad Records Logs and raw csvs are getting display under Task ID
> linux-61:/srv/OSCON/BigData/HACluster/install/hadoop/datanode # 
> bin/hadoop fs -ls /tmp/carbon/default/badrecords_test
> drwxr-xr-x   - root users          0 2017-01-18 21:08 /tmp/carbon/default/badrecords_test/0--------------------------->Task
ID
> 0: jdbc:hive2://172.168.100.205:23040> show segments for table Badrecords_test;
> +--------------------+------------------+--------------------------+--------------------------+--+
> | SegmentSequenceId  |      Status      |     Load Start Time      |      Load End Time
      |
> +--------------------+------------------+--------------------------+--------------------------+--+
> | 8                  | Partial Success  | 2017-01-18 21:12:58.018  | 2017-01-18 21:12:59.652
 |
> | 7                  | Partial Success  | 2017-01-18 21:08:07.426  | 2017-01-18 21:08:11.791
 |
> | 6                  | Partial Success  | 2017-01-18 21:07:07.645  | 2017-01-18 21:07:08.747
 |
> | 5                  | Partial Success  | 2017-01-18 19:34:16.163  | 2017-01-18 19:34:18.163
 |
> | 4                  | Partial Success  | 2017-01-18 19:34:13.669  | 2017-01-18 19:34:15.811
 |
> | 3                  | Partial Success  | 2017-01-18 19:30:18.753  | 2017-01-18 19:30:19.644
 |
> | 2                  | Partial Success  | 2017-01-18 19:30:13.508  | 2017-01-18 19:30:15.578
 |
> | 1                  | Partial Success  | 2017-01-18 19:18:54.787  | 2017-01-18 19:18:54.94
  |
> | 0                  | Partial Success  | 2017-01-18 19:18:53.741  | 2017-01-18 19:18:54.614
 |
> +--------------------+------------------+--------------------------+--------------------------+--+
> Bad Records Logs and raw csvs are getting display under Task ID. It would be good to
have the information of bad records as per the load i.e under segment id..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message