Parquet S3 FDW 1.0.0 released

Posted on 2023-01-20 by Toshiba
Related Open Source

We have just released version 1.0.0 of the Foreign Data Wrapper for Parquet file on Amazon S3.

This release can work with PostgreSQL 13, 14 and 15.

This release improves following item (from 0.3.0):

  • Support PosgreSQL 15.0
  • Support modify feature
  • Support aws region and endpoint options
  • Fix duplicated data issue when file name contains space
  • Allow insert without key column

The FDW supports following features :

  • Support SELECT of parquet file on local file system or Amazon S3
  • Support INSERT, DELETE, UPDATE (Foreign modification).
  • Support MinIO access instead of Amazon S3.
  • Allow control over whether foreign servers keep connections open after transaction completion. This is controlled by keep_connections and defaults to on.
  • Support parquet_s3_fdw function parquet_s3_fdw_get_connections() to report open foreign server connections.
  • Support to create a foreign table for multiple files in a directory by specifying a directory path.
  • Support schemaless mode that allows access to elements in the data file without changing the table definition by using jsonb typed column.

This is developed by Toshiba Software Engineering & Technology Center.
Please see the repository for details.
Source repository : https://github.com/pgspider/parquet_s3_fdw