metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Júnior <fjunio...@gmail.com>
Subject Re: Create a CSV in a empty file
Date Thu, 17 Jul 2014 13:22:03 GMT
It looks nice to me :)

When do you think this could be available on maven central reppository?

Thanks,
Junior


2014-07-17 10:16 GMT-03:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com>:

> Hi Junior,
>
> I guess the test is now breaking because there are two columns, but you
> assert that there are none. I think actually your fix worked, but the
> unittest is invalid.
>
> I rewrote the test to do these assertions, which work and to me looks
> correct:
>
>     public void testEmptyFileNoColumnHeaderLine() throws Exception {
>         final File file = new
> File("target/testEmptyFileNoColumnHeaderLine.csv");
>         FileHelper.copy(new File("src/test/resources/empty_file.csv"),
> file);
>
>         CsvConfiguration csvConfiguration = new
> CsvConfiguration(CsvConfiguration.NO_COLUMN_NAME_LINE,
>                 FileHelper.DEFAULT_ENCODING,
> CsvConfiguration.DEFAULT_SEPARATOR_CHAR, CsvConfiguration.NOT_A_CHAR,
>                 CsvConfiguration.DEFAULT_ESCAPE_CHAR);
>         final CsvDataContext dc = new CsvDataContext(file,
> csvConfiguration);
>         assertEquals(1, dc.getDefaultSchema().getTableCount());
>
>         dc.executeUpdate(new UpdateScript() {
>
>             @Override
>             public void run(UpdateCallback callback) {
>                 callback.createTable(dc.getDefaultSchema(),
> "new_table").withColumn("COL_1").withColumn("COL_2")
>                         .execute();
>                 callback.insertInto("new_table").value(0, "1").value(1,
> 2).execute();
>             }
>         });
>
>         CsvDataContext dc1 = new CsvDataContext(file, csvConfiguration);
>
>         Table[] tables = dc1.getDefaultSchema().getTables();
>         assertEquals(1, tables.length);
>
>         Table table = tables[0];
>         assertEquals("testEmptyFileNoColumnHeaderLine.csv",
> table.getName());
>         assertEquals(2, table.getColumnCount());
>
>         DataSet ds = dc1.query().from(table).selectAll().execute();
>         assertTrue(ds.next());
>         assertEquals("Row[values=[1, 2]]", ds.getRow().toString());
>         assertFalse(ds.next());
>         ds.close();
>     }
>
> What do you think?
>
>
> 2014-07-17 14:48 GMT+02:00 Júnior <fjunior87@gmail.com>:
>
> > Hi Kasper,
> >
> > After some fight, I was able to write on an empty CSV, I needed to change
> > the :
> > CsvCreateTableBuilder.java
> >
> > public Table execute() {
> >         CsvUpdateCallback csvUpdateCallback = getUpdateCallback();
> >
> >         MutableTable table = getTable();
> >         String[] columnNames = table.getColumnNames();
> >
> >         CsvDataContext csvDataContext =
> > (CsvDataContext)csvUpdateCallback.getDataContext();
> >
> >         if(csvDataContext.getConfiguration().getColumnNameLineNumber() !=
> >                         CsvConfiguration.NO_COLUMN_NAME_LINE) {
> >                 csvUpdateCallback.writeRow(columnNames, false);
> >         }
> >
> >         CsvSchema schema = (CsvSchema) table.getSchema();
> >         CsvTable csvTable = new CsvTable(schema, table.getName(),
> > table.getColumnNames());
> >         schema.setTable(csvTable);
> >         return csvTable;
> >     }
> >
> >
> > Then I've changed the
> >
> > CsvDataContextTest.testEmptyFile
> >
> > public void testEmptyFile() throws Exception {
> >         CsvConfiguration csvConfiguration =
> >         new CsvConfiguration(CsvConfiguration.NO_COLUMN_NAME_LINE,
> > FileHelper.DEFAULT_ENCODING, CsvConfiguration.DEFAULT_SEPARATOR_CHAR,
> > CsvConfiguration.NOT_A_CHAR,
> >                         CsvConfiguration.DEFAULT_ESCAPE_CHAR);
> >         final CsvDataContext dc = new CsvDataContext(new
> > File("src/test/resources/empty_file.csv"), csvConfiguration);
> >         assertEquals(1, dc.getDefaultSchema().getTableCount());
> >
> >         dc.executeUpdate(new UpdateScript() {
> >
> >                         @Override
> >                         public void run(UpdateCallback callback) {
> >
> callback.createTable(dc.getDefaultSchema(),
> > "new_table").withColumn("COL_1").withColumn("COL_2").execute();
> >                                 callback.insertInto("new_table").value(0,
> > "1").value(1, 2).execute();
> >                         }
> >                 });
> >         CsvDataContext dc1 = new CsvDataContext(new
> > File("src/test/resources/empty_file.csv"), csvConfiguration);
> >
> >
> System.out.println(dc1.query().from("empty_file.csv").selectAll().execute().toRows().size());
> >
> >         Table table = dc1.getDefaultSchema().getTables()[0];
> >         assertEquals("empty_file.csv", table.getName());
> >         assertEquals(0, table.getColumnCount());
> >     }
> >
> >
> > The test is breaking, but it wrote the lines in the empty csv.
> >
> > One thing I notice is that when we call the dropTable, if the
> > DataContext was created pointing to a file, it simply delete the file,
> > and that's the why I'm not calling it.
> >
> > Do you think it is working as expected after this change?
> >
> > Thanks,
> >
> > Junior
> >
> >
> >
> > 2014-07-16 18:29 GMT-03:00 Kasper Sørensen <
> i.am.kasper.sorensen@gmail.com
> > >:
> >
> > > I think you're right about that. Would you be able to maybe give it a
> > shot?
> > > We can add a unittest for this particular case to show if it works.
> > >
> > >
> > > 2014-07-16 23:26 GMT+02:00 Júnior <fjunior87@gmail.com>:
> > >
> > > > As I could see it is not possible to create without header because of
> > > this
> > > > line on CsvCreateTableBuilder:
> > > >
> > > >  csvUpdateCallback.writeRow(columnNames, false);
> > > >
> > > > on execute method, it is always creating the header column, maybe
> just
> > a
> > > if
> > > > before that would fix that
> > > >
> > > >
> > > >
> > > > 2014-07-16 18:22 GMT-03:00 Kasper Sørensen <
> > > i.am.kasper.sorensen@gmail.com
> > > > >:
> > > >
> > > > > I'm just now simultaniously looking into the MM csv code ... I get
> > the
> > > > > feeling you're maybe a bit in untested territory. At least it seems
> > > that
> > > > > CsvDataContextTest does not exhibit a lot of test cases around
> having
> > > no
> > > > > column headers ... Might be we need your feedback here if there are
> > > > issues.
> > > > >
> > > > >
> > > > > 2014-07-16 23:19 GMT+02:00 Kasper Sørensen <
> > > > i.am.kasper.sorensen@gmail.com
> > > > > >:
> > > > >
> > > > > > Ah, you want a header-less CSV file? You then need to instantiate
> > > your
> > > > > > CsvDataContext with a CsvConfiguration that
> > > > > > specifies CsvConfiguration.NO_COLUMN_NAME_LINE as the
> > > > > > 'columnNameLineNumber' property. If you otherwise want defaults,
> > that
> > > > > would
> > > > > > be like this:
> > > > > >
> > > > > > new CsvConfiguration(CsvConfiguration.NO_COLUMN_NAME_LINE);
> > > > > >
> > > > > >
> > > > > > 2014-07-16 23:14 GMT+02:00 Júnior <fjunior87@gmail.com>:
> > > > > >
> > > > > > Right, what I'd like to know is to create the table without
the
> > > > header, I
> > > > > >> think the only way would be changing the CsvCreateTableBuilder.
> > > > > >> Thanks,
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> 2014-07-16 18:12 GMT-03:00 Kasper Sørensen <
> > > > > >> i.am.kasper.sorensen@gmail.com>:
> > > > > >>
> > > > > >> > Hmm actually I just now realized that you can completely
omit
> > the
> > > > Drop
> > > > > >> > Table part ... if you use create table on an empty
file, it
> will
> > > > > simply
> > > > > >> > create the header.
> > > > > >> >
> > > > > >> >
> > > > > >> > 2014-07-16 23:10 GMT+02:00 Kasper Sørensen <
> > > > > >> i.am.kasper.sorensen@gmail.com
> > > > > >> > >:
> > > > > >> >
> > > > > >> > > Yes, you do that while creating the table. Like
this:
> > > > > >> > >
> > > > > >> > >         CsvDataContext dc = ...;
> > > > > >> > >         dc.executeUpdate(new UpdateScript() {
> > > > > >> > >             @Override
> > > > > >> > >             public void run(UpdateCallback callback)
{
> > > > > >> > >
> > > > > callback.dropTable(dc.getDefaultSchema().getTable(0));
> > > > > >> > >
> > > > > >> > >                 callback.createTable(dc.getDefaultSchema(),
> > > > > >> > > "newtable").withColumn("foo").withColumn("bar").execute();
> > > > > >> > >             }
> > > > > >> > >         });
> > > > > >> > >
> > > > > >> > > (replace with your own column names of course)
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > 2014-07-16 23:04 GMT+02:00 Júnior <fjunior87@gmail.com>:
> > > > > >> > >
> > > > > >> > > Hi Kasper,
> > > > > >> > >>
> > > > > >> > >> thanks for your reply, but what about the
columns?
> > > > > >> > >>
> > > > > >> > >> Do I need to add columns to this newly created
table?
> > > > > >> > >>
> > > > > >> > >> Att.
> > > > > >> > >>
> > > > > >> > >>
> > > > > >> > >> 2014-07-16 18:00 GMT-03:00 Kasper Sørensen
<
> > > > > >> > >> i.am.kasper.sorensen@gmail.com>:
> > > > > >> > >>
> > > > > >> > >> > Hi there,
> > > > > >> > >> >
> > > > > >> > >> > There is a little known trick to get
around that issue,
> so
> > I
> > > > > guess
> > > > > >> > it's
> > > > > >> > >> a
> > > > > >> > >> > very fair question.
> > > > > >> > >> >
> > > > > >> > >> > The trick is to first do a drop table,
and then a new
> > create
> > > > > table.
> > > > > >> > >> >
> > > > > >> > >> > When MM is initialized with an existing
empty file, it
> will
> > > > > >> determine
> > > > > >> > >> that
> > > > > >> > >> > there IS a table, since the file is there,
but that table
> > has
> > > > no
> > > > > >> > >> columns,
> > > > > >> > >> > because the file does not define any
columns. Maybe it's
> a
> > > bit
> > > > > >> silly
> > > > > >> > - I
> > > > > >> > >> > actually think that's worth discussing
... But the reason
> > it
> > > is
> > > > > >> there
> > > > > >> > >> is I
> > > > > >> > >> > guess to indicate that at least the file
is there, it's
> > not a
> > > > > >> > >> non-existing
> > > > > >> > >> > file (in which case there would not be
any table).
> > > > > >> > >> >
> > > > > >> > >> > Best regards,
> > > > > >> > >> > Kasper
> > > > > >> > >> >
> > > > > >> > >> >
> > > > > >> > >> > 2014-07-16 21:37 GMT+02:00 Júnior <fjunior87@gmail.com>:
> > > > > >> > >> >
> > > > > >> > >> > > Hi,
> > > > > >> > >> > >
> > > > > >> > >> > > I'm trying to use the metamodel
to write on an empty
> csv,
> > > > > >> > >> > >
> > > > > >> > >> > > But I'm getting errors saying that
there is no column.
> > > > > >> > >> > >
> > > > > >> > >> > > I was able to do that calling create
table with a
> > different
> > > > > name
> > > > > >> > from
> > > > > >> > >> the
> > > > > >> > >> > > file.
> > > > > >> > >> > > Then added the columns.
> > > > > >> > >> > >
> > > > > >> > >> > > I worked, but it created the header
line with the
> column
> > > > names,
> > > > > >> > would
> > > > > >> > >> it
> > > > > >> > >> > be
> > > > > >> > >> > > possible to do that without adding
the column names in
> > the
> > > > csv
> > > > > >> file?
> > > > > >> > >> > >
> > > > > >> > >> > > Att.
> > > > > >> > >> > > --
> > > > > >> > >> > > Francisco Ribeiro
> > > > > >> > >> > > *SCEA|SCJP|SCWCD|IBM Certified SOA
Associate*
> > > > > >> > >> > >
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> > >>
> > > > > >> > >>
> > > > > >> > >> --
> > > > > >> > >> Francisco Ribeiro
> > > > > >> > >> *SCEA|SCJP|SCWCD|IBM Certified SOA Associate*
> > > > > >> > >>
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Francisco Ribeiro
> > > > > >> *SCEA|SCJP|SCWCD|IBM Certified SOA Associate*
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Francisco Ribeiro
> > > > *SCEA|SCJP|SCWCD|IBM Certified SOA Associate*
> > > >
> > >
> >
> >
> >
> > --
> > Francisco Ribeiro
> > *SCEA|SCJP|SCWCD|IBM Certified SOA Associate*
> >
>



-- 
Francisco Ribeiro
*SCEA|SCJP|SCWCD|IBM Certified SOA Associate*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message