commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tadhg Pearson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CSV-198) Cannot parse file by header with custom delimiter on Java 1.6
Date Fri, 02 Dec 2016 21:58:58 GMT

    [ https://issues.apache.org/jira/browse/CSV-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716644#comment-15716644
] 

Tadhg Pearson commented on CSV-198:
-----------------------------------

OK - I found why it was failing for me on Java 1.8. 
I needed to set these 2 properties in my pom.xml

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

>From looking at the Maven logs I can see that without this, Windows copies the file to
the target test folder using CP1252. My guess is that this then screws up the file read during
the test. Sorry for the false alarm, and I hope that helps someone!

> Cannot parse file by header with custom delimiter on Java 1.6
> -------------------------------------------------------------
>
>                 Key: CSV-198
>                 URL: https://issues.apache.org/jira/browse/CSV-198
>             Project: Commons CSV
>          Issue Type: Bug
>          Components: Parser
>    Affects Versions: 1.4
>         Environment: Java 1.6, Windows
>            Reporter: Tadhg Pearson
>
> Reading a CSV file from the parser using the header line results in "IllegalArgumentException:
Mappng for <column> not found" - even when the column exists in the file. In this case,
we are using Java 1.6 and the file uses the ^ symbol to delimit columns. This works correctly
in Java 7 & Java 8.
> The code required to reproduce the issue is below. You can find the optd_por_public.csv
file referenced at https://raw.githubusercontent.com/opentraveldata/opentraveldata/master/opentraveldata/optd_por_public.csv
> It will need to be on the classpath to run the unit test.  Hope that helps, Tadhg
> You should get the following output
> --------------------------------------
> {code}
> java.lang.IllegalArgumentException: Mapping for location_type not found, expected one
of [iata_code,icao_code,faa_code,is_geonames,geoname_id,envelope_id,name,asciiname,latitude,longitude,fclass,fcode,page_rank,date_from,date_until,comment,country_code,cc2,country_name,continent_name,adm1_code,adm1_name_utf,adm1_name_ascii,adm2_code,adm2_name_utf,adm2_name_ascii,adm3_code,adm4_code,population,elevation,gtopo30,timezone,gmt_offset,dst_offset,raw_offset,moddate,city_code_list,city_name_list,city_detail_list,tvl_por_list,state_code,location_type,wiki_link,alt_name_section,wac,wac_name]
> 	at org.apache.commons.csv.CSVRecord.get(CSVRecord.java:104)
> 	at com.amadeus.ui.CSVRecordTest.test(CSVRecordTest.java:31)
> {code}
> ------------
> {code:java}
> import static org.junit.Assert.assertNotNull;
> import java.io.IOException;
> import java.io.InputStream;
> import java.io.InputStreamReader;
> import java.io.UnsupportedEncodingException;
> import org.apache.commons.csv.CSVFormat;
> import org.apache.commons.csv.CSVParser;
> import org.apache.commons.csv.CSVRecord;
> import org.junit.Test;
> public class CSVRecordTest {
> 	private static final CSVFormat CSV_FORMAT = CSVFormat.EXCEL.withDelimiter('^').withFirstRecordAsHeader();
> 	
> 	@Test
> 	public void test() throws UnsupportedEncodingException, IOException {
> 		InputStream pointsOfReference = getClass().getResourceAsStream("/optd_por_public.csv");
> 		CSVParser parser = CSV_FORMAT.parse(new InputStreamReader(pointsOfReference, "UTF-8"));
> 		for (CSVRecord record : parser) {
> 			String locationType = record.get("location_type");
> 			assertNotNull(locationType);
> 		}
> 	}
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message