Kafka

Transfer Protocols

Kafka uses protocol buffers over TCP
Does sequential disk IO wherever possible

Kafka vs. AWS SQS

SQS uses HTTPS

Brokers

Cluster - multiple brokers
Broker - individual servers
Partitions - a sequence of messages that's appended to. These are split up by key
- Each partition has a leader to handle writes
Messages are written to a WAL on disk
Data is replicated across brokers