phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas D'Silva (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-2690) Exceeding OS limit for native threads with large number of guide posts
Date Thu, 18 Feb 2016 21:11:18 GMT


Thomas D'Silva commented on PHOENIX-2690:

[~mujtabachohan] When I run the query on a table with a large number of guideposts (>4000)
it crashes my local HBase but I wasn't able to repro the OutOfMemoryError. 

After I restart my local HBase any query on the table fail with the following exception

Error: Task org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask@df76c5e rejected from
org.apache.phoenix.job.JobManager$1@51f722f1[Running, pool size = 128, active threads = 128,
queued tasks = 5000, completed tasks = 5168] (state=08000,code=101)

 BaseQueryServicesImpl seems to be setting up the executor correctly. 
this.executor =  JobManager.createThreadPoolExec(

Can you run ulimit -a on one of the datanodes to see what the process limit is set to? Was
anything else running while you ran your query?

> Exceeding OS limit for native threads with large number of guide posts
> ----------------------------------------------------------------------
>                 Key: PHOENIX-2690
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.7.0
>            Reporter: Mujtaba Chohan
>            Assignee: Thomas D'Silva
>            Priority: Minor
> With >32K guidepost count for a table (which equates to a 320 GB table size with 10MB
guidepost width), following OOM occurs on 64bit Linux client
> {code}
> select * from table_with_32k_guideposts limit 1000;
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(
> 	at java.util.concurrent.ThreadPoolExecutor.addWorker( at
java.util.concurrent.ThreadPoolExecutor.execute( at org.apache.phoenix.job.JobManager$InstrumentedThreadPoolExecutor.execute(
at java.util.concurrent.AbstractExecutorService.submit( at
org.apache.phoenix.iterate.ParallelIterators.submitWork( at org.apache.phoenix.iterate.BaseResultIterators.getIterators(
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(
at org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(
> {code}

This message was sent by Atlassian JIRA

View raw message