MinIO

What is it?

MinIO is an open-source object storage system, compatible with the Amazon S3 API. It keeps files (objects) inside buckets, just like S3, but running on your own server or machine. It is the physical layer where a data lake's raw data lives — beneath the formats and the engines.

It's important not to confuse it with a database: MinIO does not understand SQL or tables. It only stores and serves files at scale — durably and elastically. The structure (Parquet, Iceberg) and the queries (Trino, Spark) are other layers that sit on top of it.

What is it for?

• Running your own data lake. Store the files (usually Parquet) of your lake in MinIO, on-premise or locally, without depending on a cloud.
• Storage backend for a lakehouse. Iceberg or Delta tables store their data as objects in MinIO.
• S3 compatibility without the cloud. Since it speaks the S3 API, tools like Spark, Trino, or DuckDB read and write to MinIO without changing a line — ideal for local development or closed environments.

When to use it / when not

Use it when you need to store files at scale (a lake, backups, artifacts) and want S3 compatibility without locking into a cloud provider — or to have S3 on your laptop while developing.

Think twice for:

• Structured data you query with SQL: that's a database's job (PostgreSQL) or an engine's (Trino) — MinIO only stores the files, it doesn't query them.
• You need the format, not the store: if what you want is how to store data efficiently, that's Parquet (format), not MinIO (where it's stored).
• You're already in the cloud with managed S3/GCS/Azure Blob: there MinIO adds operations you may not need.

Get started in 1 minute

Spin up MinIO locally with Docker (web console + S3 API on port 9000):

docker run -d -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=admin \
  -e MINIO_ROOT_PASSWORD=admin12345 \
  minio/minio server /data --console-address ":9001"

Open the console at http://localhost:9001 (user admin, password admin12345), create a bucket and upload a file. From Python you can use the same S3 client:

pip install boto3

import boto3
s3 = boto3.client("s3", endpoint_url="http://localhost:9000",
                  aws_access_key_id="admin", aws_secret_access_key="admin12345")
s3.create_bucket(Bucket="lake")
s3.upload_file("sales.parquet", "lake", "sales.parquet")