quickstep-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harshad Deshmukh <hars...@cs.wisc.edu>
Subject Re: Bulk Insert in Quickstep creates a new block for every row?
Date Sun, 10 Dec 2017 04:25:21 GMT
Hi Lokananda,


There is a support for bulk insert. If you have a csv file in which the columns are delimited
by the symbol '|' (without the single quotes), the bulk insert syntax is :



COPY table_name FROM '/full/path/to/csvfile' WITH (DELIMITER '|');


An example of the COPY command can be found in the README file on Github (https://github.com/apache/incubator-quickstep)


Thanks,

Harshad

________________________________
From: LOKANANDA DHAGE MUNISAMAPPA <dhagemunisam@wisc.edu>
Sent: Saturday, December 9, 2017 9:45:16 PM
To: dev@quickstep.incubator.apache.org
Cc: Om Jadhav
Subject: Bulk Insert in Quickstep creates a new block for every row?

Hi Quickstep Devs,

I am trying to bulk insert tuples into a table in Quickstep created with BlockProperties value
of BLOCKSIZEMB to 2MB as below.

Create table Query
CREATE TABLE lineorder (
  lo_orderkey      INT NOT NULL,
  lo_linenumber    INT NOT NULL,
  lo_custkey       INT NOT NULL,
  lo_orderpriority CHAR(1024) NOT NULL
) WITH BLOCKPROPERTIES (
TYPE split_rowstore,
BLOCKSIZEMB 2);

Insert table Query

INSERT INTO lineorder VALUES (22, 6, 0, 'high'),(20, 6, 1, 'high'),(20, 6, 2, 'high'),(20,
6, 3, 'high'),(20, 6, 4, 'high'),(20, 6, 5, 'high'),(20, 6, 6, 'high'),(20, 6, 7, 'high'),(20,
6, 8, 'high')….;

As a result of this, I am observing that, it is creating a separate block of 2MB for every
tuple, thereby creating a number of half-filled blocks of size 2MB. Is this the expected behavior
for bulk-insert in quickstep?

Is there a way to bulk-insert without creating a new block for every row in Quickstep? This
would be helpful for creating the workload for our project, as insertion using multiple separate
queries is taking a lot of time, although in this case, it is not really creating a separate
block for every tuple and seems to work as expected.

Best,
Lokananda

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message