Skip to content

Changelog

Source: NEWS.md

s3pe 0.3.0

  • Multipart upload: write_s3() and upload_s3() send bodies larger than multipart_threshold (64 MiB by default; s3_config(), S3PE_MULTIPART_THRESHOLD) in parts of part_size (16 MiB), one by one, aborted on any failure, with every part’s ETag and the final ETag checked.
  • write_s3() and upload_s3() take metadata (user metadata, returned by head_s3() in its new metadata column) and headers (any extra header: Cache-Control, Content-Disposition, x-amz-storage-class, server-side encryption, …).
  • copy_s3() copies an object on the server.
  • delete_s3() accepts several keys and deletes them in batches of 1000; it now returns the number of keys deleted.
  • list_s3() gains delimiter: delimiter = "/" lists one level, “folders” included.
  • presign_s3() returns a pre-signed URL (GET or PUT) valid for up to seven days.
  • sync_up_s3() and sync_down_s3() mirror a directory and a prefix by size and ETag, optionally deleting extras and uploading a manifest last.
  • Not implemented, by design: ACLs (manage access with keys and policies) and versioning (no versionId on read, head or delete).

s3pe 0.2.0

  • The endpoint has no default any more: set S3PE_ENDPOINT, options(s3pe.endpoint) or call use_endpoint(). A missing endpoint is an s3pe_error_config at request time, like missing credentials.
  • use_bucket() and use_endpoint() set the bucket and the endpoint for the session.
  • Documentation and examples are generic: the package serves any S3-compatible store.

s3pe 0.1.0

  • Initial release.
  • read_s3() and write_s3() move R objects to and from the store, with the format chosen by the key’s extension (CSV, TSV, JSON, GeoJSON, Parquet, RDS, XLSX, text, raw).
  • download_s3() and upload_s3() move files, streaming.
  • list_s3(), head_s3(), exists_s3(), delete_s3() and buckets_s3() for the rest of the bucket.
  • s3_config() resolves endpoint, region, credentials and bucket from arguments, options and the environment, in that order.
  • Requests signed with AWS Signature Version 4 (header style, payload hashed); transient failures retried with exponential backoff; ETag and content length checked after every transfer.