flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oytun Tez <oy...@motaword.com>
Subject Re: Is Queryable State not working in standalone cluster with 1 task manager?
Date Tue, 30 Jul 2019 14:20:33 GMT
Wow, I think QS state and proxy servers are actually running, but the logs
are not sent to container logs.

I was able to see the activation in /opt/flink/log/taskexecutor-*.log file:
[image: image.png]

These lines however are not visible in the typical docker container log.

Thank you for the support, Fabian. I am now structuring our QS
interactions, let's see what comes.

---
Oytun Tez

*M O T A W O R D*
The World's Fastest Human Translation Platform.
oytun@motaword.com — www.motaword.com


On Tue, Jul 30, 2019 at 10:15 AM Fabian Hueske <fhueske@gmail.com> wrote:

> JM, TM and QS client are communicating via the network.
> I'm not super familiar with Docker setups, but you might need to open some
> ports to make it work.
> I'd check the default port settings for the QS configuration and open them.
>
> Am Di., 30. Juli 2019 um 16:11 Uhr schrieb Oytun Tez <oytun@motaword.com>:
>
>> Both task manager and job manager are running in the same container,
>> could that be the reason?
>>
>> The container is using StandaloneJobClusterEntryPoint. Verified
>> flink-conf and ./lib/runtime jar, both present, I can see the config is
>> loaded from the logs, and nothing else about queryable state in the logs.
>>
>> ---
>> Oytun Tez
>>
>> *M O T A W O R D*
>> The World's Fastest Human Translation Platform.
>> oytun@motaword.com — www.motaword.com
>>
>>
>> On Tue, Jul 30, 2019 at 9:05 AM Oytun Tez <oytun@motaword.com> wrote:
>>
>>> [image: image.png]
>>>
>>> ---
>>> Oytun Tez
>>>
>>> *M O T A W O R D*
>>> The World's Fastest Human Translation Platform.
>>> oytun@motaword.com — www.motaword.com
>>>
>>>
>>> On Tue, Jul 30, 2019 at 9:05 AM Oytun Tez <oytun@motaword.com> wrote:
>>>
>>>> [image: image.png]
>>>>
>>>> ---
>>>> Oytun Tez
>>>>
>>>> *M O T A W O R D*
>>>> The World's Fastest Human Translation Platform.
>>>> oytun@motaword.com — www.motaword.com
>>>>
>>>>
>>>> On Tue, Jul 30, 2019 at 9:04 AM Oytun Tez <oytun@motaword.com> wrote:
>>>>
>>>>> We extend the official container and the jar is available under ./lib
>>>>> – so it wasn't enabled by default, we moved the jar under ./lib and
then
>>>>> build the container+application.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---
>>>>> Oytun Tez
>>>>>
>>>>> *M O T A W O R D*
>>>>> The World's Fastest Human Translation Platform.
>>>>> oytun@motaword.com — www.motaword.com
>>>>>
>>>>>
>>>>> On Tue, Jul 30, 2019 at 3:42 AM Fabian Hueske <fhueske@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Oytun,
>>>>>>
>>>>>> Is QS enabled in your Docker image or did you enable QS by
>>>>>> copying/moving flink-queryable-state-runtime_2.11-1.8.0.jar from
./opt to
>>>>>> ./lib [1]?
>>>>>>
>>>>>> Best, Fabian
>>>>>>
>>>>>> [1]
>>>>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#activating-queryable-state
>>>>>>
>>>>>> Am Mo., 29. Juli 2019 um 22:46 Uhr schrieb Oytun Tez <
>>>>>> oytun@motaword.com>:
>>>>>>
>>>>>>> And, Flink version: 1.8, incl. Docker container (official flink:1.8
>>>>>>> tag)
>>>>>>>
>>>>>>> ---
>>>>>>> Oytun Tez
>>>>>>>
>>>>>>> *M O T A W O R D*
>>>>>>> The World's Fastest Human Translation Platform.
>>>>>>> oytun@motaword.com — www.motaword.com
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 29, 2019 at 4:32 PM Oytun Tez <oytun@motaword.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi there,
>>>>>>>>
>>>>>>>> We have a job that is run inside Docker via `standalone-job.sh
>>>>>>>> start-foreground --job-classname`, with 1 task manager. I've
been
>>>>>>>> trying to make QueryableState available in this setup for
2 days now and I
>>>>>>>> can't seem to enable it.
>>>>>>>>
>>>>>>>> If I create a LocalEnvironment within the code itself and
provide queryable-state.enable:
>>>>>>>> true config directly via Configuration, then I can at least
see
>>>>>>>> that the queryable state server is spinning up.
>>>>>>>>
>>>>>>>> Any pointers? With standalone-job.sh, it seems that it doesn't
>>>>>>>> care about queryable-state.enable config inside flink-conf.yaml.
>>>>>>>>
>>>>>>>> ---
>>>>>>>> Oytun Tez
>>>>>>>>
>>>>>>>> *M O T A W O R D*
>>>>>>>> The World's Fastest Human Translation Platform.
>>>>>>>> oytun@motaword.com — www.motaword.com
>>>>>>>>
>>>>>>>

Mime
View raw message