Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
Received-SPF: pass (athena.apache.org: domain of garydgregory@gmail.com
 designates 209.85.214.48 as permitted sender)
MIME-Version: 1.0
Date: Wed, 31 Jul 2013 19:38:04 -0400
Message-ID: 
 <CACZkXPzGd+u15i0LF3scH+3Q1OtUrPJira5_uwPM6a78v2M-Tw@mail.gmail.com>
Subject: [csv] the plot thinkens: multi-record headers
From: Gary Gregory <garydgregory@gmail.com>
To: Commons Developers List <dev@commons.apache.org>
Content-Type: multipart/alternative; boundary=20cf301ee46f8e366b04e2d73782

--20cf301ee46f8e366b04e2d73782
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi All,

I found an interesting set of sample CSV files [1] on the IBM Tivoli web
site. Each file seems to have a, for lack of a better term, "pre-header".
For example, the simplest file [2] looks like this (3 lines):

SRM_SaaS_ES,MXASSETInterface,AddChange,EN
ASSETNUM,AS_SITEID
mesa01,SDASITE

A more complex one is [3] (4 lines):

SRM_SaaS_ES,MXASSETInterface,AddChange,EN
ASSETNUM,ASSETTAG,AUTOWOGEN,BUDGETCOST,CALNUM,CHANGEBY,CHANGEDATE,CHANGEPMS=
TATUS,CHILDREN,AS_DESCRIPTION,DISABLED,FAILURECODE,HIERARCHYPATH,INVCOST,IS=
LINEAR,ISRUNNING,AS_ITEMNUM,AS_ITEMSETID,MAINTHIERCHY,AS_MANUFACTURER,MOVED=
,AS_ORGID,PRIORITY,PURCHASEPRICE,REMOVEFROMACTIVEROUTES,REMOVEFROMACTIVESP,=
REPLACECOST,ROLLTOALLCHILDREN,ROTSUSPACCT,AS_SENDERSYSID,AS_SITEID,AS_STATU=
S,AS_STATUSDATE,TOTALCOST,TOTDOWNTIME,TOTUNCHARGEDCOST,UNCHARGEDCOST,USAGE,=
VENDOR,YTDCOST,ACTIVE,ASSETMETERID,AVGCALCMETHOD,AM_CHANGEBY,AM_CHANGEDATE,=
LIFETODATE,LINEARASSETMETERID,MEASUREUNITID,METERNAME,AM_ORGID,READINGTYPE,=
ROLLDOWNSOURCE,SEQUENCE,SINCEINSTALL,SINCELASTINSPEC,SINCELASTOVERHAUL,SINC=
ELASTREPAIR
cent41,6491,0,0,COMPANY1,MAXADMIN,2010-04-15T17:18:18-07:00,0,0,Centrifugal
Pump 100GPM/60FT HD,0,PUMPS,PUMP \
CNTRFGL,0,0,1,PUMP100,SET1,0,IR,0,SDAORG,2,0,0,0,0,0,6600-869-800,MX,SDASIT=
E,OPERATING,2010-04-15T17:18:18-07:00,0,0,0,0,,IR,0,1,29,ALL,MAXADMIN,2010-=
04-15T17:19:11-07:00,0,0,HOURS,FLTHRS,SDAORG,DELTA,ASSET,1,0,0,0,0
cent41,6491,0,0,COMPANY1,MAXADMIN,2010-04-15T17:18:18-07:00,0,0,Centrifugal
Pump 100GPM/60FT HD,0,PUMPS,PUMP \
CNTRFGL,0,0,1,PUMP100,SET1,0,IR,0,SDAORG,2,0,0,0,0,0,6600-869-800,MX,SDASIT=
E,OPERATING,2010-04-15T17:18:18-07:00,0,0,0,0,,IR,0,1,30,,MAXADMIN,2010-04-=
15T17:19:38-07:00,0,0,DEG
F,TEMP-F,SDAORG,,,2,0,0,0,0

The first line of both and the other files I checked include:

SRM_SaaS_ES,MXASSETInterface,AddChange,EN

which is NOT the column names for the data, as far as I can tell.

To properly process these files, it looks like we need to either:

(1) expand the concept of a header, to include multiple records, specifying
which one is the header record for column names, or,
(2) add a skipFirstRecords settings.

Thoughts?

Gary
[1]
http://pic.dhe.ibm.com/infocenter/tivihelp/v41r1/index.jsp?topic=3D%2Fcom.i=
bm.ismsaas.doc%2Fimport%2Fr_sample_csv_files.html
[2]
http://pic.dhe.ibm.com/infocenter/tivihelp/v41r1/topic/com.ibm.ismsaas.doc/=
reference/AssetsImportMinimumSample.csv
)
[3]
http://pic.dhe.ibm.com/infocenter/tivihelp/v41r1/topic/com.ibm.ismsaas.doc/=
reference/AssetsImportExtendedSample.csv

--=20
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition<http://www.manning.com/baue=
r3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

--20cf301ee46f8e366b04e2d73782--