asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenhai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1478) using localfs goes wrong
Date Fri, 10 Jun 2016 17:20:21 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wenhai updated ASTERIXDB-1478:
------------------------------
    Description: 
I load data file using localfs but AsterixDB keeps running for a long long time.(ps:CPU utilization
~= 100%) I have been waiting over 8 hours yesterday.Then i found that the storage folder in
workingdir didn't add any files.
Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.

About data files:one file is big object insertion(each record has over 30,000 characters);
records' size of another file has 100~200 characters.
{nofromat}
local.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cluster xmlns="cluster">
    <java_home>/usr/local/jdk1.8.0_91/jre</java_home>
    <log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
    <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
    <store>storage</store>
    <working_dir>
        <dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
        <NFS>true</NFS>
    </working_dir>
  <iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
    <master_node>
        <id>master</id>
        <client_ip>127.0.0.1</client_ip>
        <cluster_ip>127.0.0.1</cluster_ip>
        <client_port>1098</client_port>
        <cluster_port>1099</cluster_port>
        <http_port>8888</http_port>
        <debug_port>8900</debug_port>
    </master_node>
    <node>
        <id>nc1</id>
        <cluster_ip>127.0.0.1</cluster_ip>
        <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
     
        <debug_port>8901</debug_port>
    </node>
    <node>
        <id>nc2</id>
        <cluster_ip>127.0.0.1</cluster_ip>
        <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
      
        <debug_port>8902</debug_port>
    </node>
    <heartbeat_period>10000</heartbeat_period>
    <max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
    <profile_dump_period>0</profile_dump_period>
    <default_max_job_attempts>5</default_max_job_attempts>
    <job_history_size>10</job_history_size>
    <result_time_to_live>86400000</result_time_to_live>
    <result_sweep_threshold>60000</result_sweep_threshold>
</cluster>
{nofromat}

  was:
I load data file using localfs but AsterixDB keeps running for a long long time.(ps:CPU utilization
~= 100%) I have been waiting over 8 hours yesterday.Then i found that the storage folder in
workingdir didn't add any files.
Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.

About data files:one file is big object insertion(each record has over 30,000 characters);
records' size of another file has 100~200 characters.

local.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cluster xmlns="cluster">
    <java_home>/usr/local/jdk1.8.0_91/jre</java_home>
    <log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
    <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
    <store>storage</store>
    <working_dir>
        <dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
        <NFS>true</NFS>
    </working_dir>
  <iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
    <master_node>
        <id>master</id>
        <client_ip>127.0.0.1</client_ip>
        <cluster_ip>127.0.0.1</cluster_ip>
        <client_port>1098</client_port>
        <cluster_port>1099</cluster_port>
        <http_port>8888</http_port>
        <debug_port>8900</debug_port>
    </master_node>
    <node>
        <id>nc1</id>
        <cluster_ip>127.0.0.1</cluster_ip>
        <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
     
        <debug_port>8901</debug_port>
    </node>
    <node>
        <id>nc2</id>
        <cluster_ip>127.0.0.1</cluster_ip>
        <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
      
        <debug_port>8902</debug_port>
    </node>
    <heartbeat_period>10000</heartbeat_period>
    <max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
    <profile_dump_period>0</profile_dump_period>
    <default_max_job_attempts>5</default_max_job_attempts>
    <job_history_size>10</job_history_size>
    <result_time_to_live>86400000</result_time_to_live>
    <result_sweep_threshold>60000</result_sweep_threshold>
</cluster>


> using localfs goes wrong
> ------------------------
>
>                 Key: ASTERIXDB-1478
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1478
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: Error Reporting
>         Environment: AsterixDB 0.8.9(pulled from github today)
> Ubuntu 14.0
> Java -version 1.8.0_79
> Computer has 1 ssd and 1 hhd
>            Reporter: yipzifai
>              Labels: features
>
> I load data file using localfs but AsterixDB keeps running for a long long time.(ps:CPU
utilization ~= 100%) I have been waiting over 8 hours yesterday.Then i found that the storage
folder in workingdir didn't add any files.
> Exectly, I try to use feeding or AQL "Insert", but it still goes wrong.
> About data files:one file is big object insertion(each record has over 30,000 characters);
records' size of another file has 100~200 characters.
> {nofromat}
> local.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <cluster xmlns="cluster">
>     <java_home>/usr/local/jdk1.8.0_91/jre</java_home>
>     <log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/logs</log_dir>
>     <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/txnLogs</txn_log_dir>
>     <store>storage</store>
>     <working_dir>
>         <dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir</dir>
>         <NFS>true</NFS>
>     </working_dir>
>   <iodevices>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p1,/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc/p2</iodevices>
>     <master_node>
>         <id>master</id>
>         <client_ip>127.0.0.1</client_ip>
>         <cluster_ip>127.0.0.1</cluster_ip>
>         <client_port>1098</client_port>
>         <cluster_port>1099</cluster_port>
>         <http_port>8888</http_port>
>         <debug_port>8900</debug_port>
>     </master_node>
>     <node>
>         <id>nc1</id>
>         <cluster_ip>127.0.0.1</cluster_ip>
>         <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc1/txnLogs</txn_log_dir>
>      
>         <debug_port>8901</debug_port>
>     </node>
>     <node>
>         <id>nc2</id>
>         <cluster_ip>127.0.0.1</cluster_ip>
>         <txn_log_dir>/home/to/adbSrc/asterix-mgmt/clusters/local/working_dir/nc2/txnLogs</txn_log_dir>
>       
>         <debug_port>8902</debug_port>
>     </node>
>     <heartbeat_period>10000</heartbeat_period>
>     <max_heartbeat_lapse_periods>5</max_heartbeat_lapse_periods>
>     <profile_dump_period>0</profile_dump_period>
>     <default_max_job_attempts>5</default_max_job_attempts>
>     <job_history_size>10</job_history_size>
>     <result_time_to_live>86400000</result_time_to_live>
>     <result_sweep_threshold>60000</result_sweep_threshold>
> </cluster>
> {nofromat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message