The solution I can think of is to have the Python process serialize the DataFrame and save it to redis, and have the Java process parse the data.
The solution I find serializes the DataFrame to 'pybytes':
import pandas as pd
import pyarrow as pa
r = redis.Redis(host='localhost', port=6379, db=0)
context = pa.default_serialization_context()
I wonder if this serialized 'pybytes' can be parsed at the Java end? If not, how can I achieve this properly?