flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hilmi Yildirim <hilmi.yildi...@neofonie.de>
Subject Re: Read from HBase flink-0.9.0-milestone-1-bin-hadoop2
Date Fri, 29 May 2015 10:21:23 GMT
I solved the problem. The problem was that I created an HBase table only 
on 1 machine. 13 machines requested data from this sigle machine which 
lead to a bottleneck. Now, I have split the table on 13 machines and it 
is working now. It continues to throw UnknownScannerExceptions but it is 
working. Maybe it is beneficially to create a task to catch this 
exception or to handle it correctly.

Best Regards,
Hilmi


Am 28.05.2015 um 11:34 schrieb Flavio Pompermaier:
>
>    The unknown scanner exception is just a warning due to the fact 
> that the table scan is not eager but lazy..so between two consecutive 
> getNextRecord() call the Habase server timeouts the scanner 
> resource's..In the house extension I just recreate the scanner from 
> the last read key. This is a workaround to avoid too much tuning with 
> caching and timeouts hbase params...
>
> On 28 May 2015 10:24, "Hilmi Yildirim" <hilmi.yildirim@neofonie.de 
> <mailto:hilmi.yildirim@neofonie.de>> wrote:
>
>     Hi,
>     I also tried to read from HBase using the
>     flink-0.9.0-milestone-1-bin-hadoop2 version. It works great and it
>     uses the config declared in the hbase-site.xml.
>
>     I configured a flink cluster with 2 machines. For that I deployed
>     the flink folder with the corresponding configurations on each
>     machine. To start the cluster I use the start script
>     ./start-cluster.sh. Then, I run the job and it works. Now, I
>     configured a flink cluster with 13 machines. All machines read
>     from hbase correctly besides 1 or 2 machines. These 1 or 2
>     machines get following exception ||
>     "org.apache.hadoop.hbase.UnknownScannerException:
>     org.apache.hadoop.hbase.UnknownScannerException: Name 2423,
>     already closed?"
>
>     When I restart the job then the exception occurs for other 1 or 2
>     machines.
>
>     Does anyone know why this exception occurs?
>
>     Best Regards,
>
>     -- 
>     --
>     Hilmi Yildirim
>     Software Developer R&D
>
>     T:+49 30 24627-281  <tel:%2B49%2030%2024627-281>
>     Hilmi.Yildirim@neofonie.de  <mailto:Hilmi.Yildirim@neofonie.de>
>
>     http://www.neofonie.de
>
>     Besuchen Sie den Neo Tech Blog für Anwender:
>     http://blog.neofonie.de/
>
>     Folgen Sie uns:
>     https://plus.google.com/+neofonie
>     http://www.linkedin.com/company/neofonie-gmbh
>     https://www.xing.com/companies/neofoniegmbh
>
>     Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>     Handelsregister Berlin-Charlottenburg: HRB 67460
>     Geschäftsführung: Thomas Kitlitschko
>

-- 
--
Hilmi Yildirim
Software Developer R&D

T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de

http://www.neofonie.de

Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/

Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh

Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko


Mime
View raw message