| Feature | Kinesis Data Streams | Kinesis Firehose |
|---|---|---|
| Latency | ~70ms (real-time) | 60s–900s (near real-time) |
| Retention | 24h–365 days | No retention |
| Consumers | Custom (KCL, Lambda, SDK) | Only S3, Redshift, OpenSearch, Splunk |
| Scaling | Shard-based (manual/auto) | Fully managed, auto-scales |
| Data Transform | Custom via KCL/Lambda | Built-in Lambda transform |
| Use Case | Real-time analytics, custom apps | Simple delivery pipeline |
| Component | Purpose | Key Details |
|---|---|---|
| Crawlers | Discover and catalog data | Auto-detects schema, writes to Data Catalog |
| Data Catalog | Central metadata store | Compatible with Hive metastore, used by Athena/EMR/Redshift |
| ETL Jobs | Transform data | Spark-based, PySpark, Scala, or Ray |
| Triggers | Schedule or event-based | Schedule, job event, on-demand |
| Workflows | Orchestrate ETL | Visual editor for multi-step pipelines |
| Glue Studio | Visual ETL design | Drag-and-drop job creation |
Read aws-data-engineer-associate notes without distractions
Open Foci to run a focused study block while you review domains, tables, and exam tips.