lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-66) bulk data loader
Date Mon, 06 Nov 2006 16:08:37 GMT
    [ http://issues.apache.org/jira/browse/SOLR-66?page=comments#action_12447460 ] 
            
Yonik Seeley commented on SOLR-66:
----------------------------------

To most efficiently load a large export file from some other datasource, we should allow an
upload of a local file and avoid going through the network/servlet container for each individual
record.

Way to slurp a local file: a post to /solr/upload/localfile?file=full_path_to_file
Parameter Ideas:
  file   #the full path name of the local file
  separator=,   #the field separator
  charset=utf8
  commit=false  #do a commit after finished
  fieldnames=foo,bar,baz   #define field names, if not taken from the header 
  header=true  #read field names from the header
  skip=baz        #fields to skip
  skiplines=1   #number of lines at the start of the file to skip
  skipempty=true  #don't index zero length values 
  trim=true  #trim whitespace from field values

  map=Yes:true  #map a field value

  #per-field params
  f.myfield.map=No:false

  escape=true  #backslash escaping
  quotedfields=true  #optionally quoted fields (like CVS)

  csv=true #sets options correctly for a CSV file

  #support comments in the file?

> bulk data loader
> ----------------
>
>                 Key: SOLR-66
>                 URL: http://issues.apache.org/jira/browse/SOLR-66
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>         Assigned To: Yonik Seeley
>
> A way to efficiently load simple formatted text files, including CSV files.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message