arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Badger, Bruce" <>
Subject [Python] testing custom pyarrow.fs filesystems
Date Fri, 18 Jun 2021 12:08:39 GMT
Dear pyarrow.fs team,

We are implementing a custom pyarrow.fs filesystem to map the contents of internal file stores
as filesystems for use in Arrow.

We have a suite of unit tests which exercise the internal parts of our implementation, and
we can run pyarrow.tests.test_fs to ensure that the supplied pyarrow.fs filesystems work as
we have them installed.

I would like to include our custom filesystem as a sibling of the included pyarrow.fs filesystems
in the pyarrow.tests.test_fs tests in order to ensure that our filesystem conforms to the
expectations of the pyarrow.fs implementation, and continues to conform as pyarrow.fs and
our internal systems evolve.

Are the pyarrow.tests.test_fs tests extensible to allow the testing of custom filesystems
in addition to, and as peer of, the supplied filesystems?  If so, how is this intended to
work?  ... and if not, may I suggest that this be added as a feature for the (hopefully not
too distant) future?

Many thanks,

This message, and any attachments, is for the intended recipient(s) only, may contain information
that is privileged, confidential and/or proprietary and subject to important terms and conditions
available at .  If you are not the intended recipient,
please delete this message. Please note you may be contacted by a different BofA entity acting
for and on behalf of your service provider where permitted by applicable law. This does not
change your service provider.

Where applicable please note:
Merrill Lynch International is Registered in England (No.2312079). Registered Office: 2 King
Edward Street, London EC1A, 1HQ. VAT No. GB 245 1224 93. Authorised by the Prudential Regulation
Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority.
Member of the London Stock Exchange.

View raw message