AWS S3 vs RDS — when to use which?
When should I use S3 versus RDS in an AWS data pipeline?
S3 for files (raw data, processed parquet, backups). RDS for transactional or operational data your app reads and writes. Pipelines typically extract from RDS, transform in compute (Glue/Lambda/EMR), and land results in S3 or a warehouse.
S3 is object storage — you put files there (CSVs, JSONs, Parquet, images, anything) and retrieve them by key. It is cheap, virtually infinite, and the foundation of every AWS data lake. You never query S3 like a database — you list and read files.
RDS is managed relational databases (Postgres, MySQL, etc). You query it with SQL, it enforces schemas, supports transactions. RDS is for application data and structured queries. As a Data Engineer you use both: S3 holds raw and processed data, RDS often holds operational data your pipelines extract from.