Communication & Data Transfer/Deep Cuts (Optional)

Resumable & Multipart Protocols

tus protocol details, S3 MPU edge cases, checksum assembly, and client SDK design for unreliable networks.

3/5Overview: 25m

tus protocol details

Headers: Upload-Length, Upload-Offset, Tus-Resumable: 1.0.0

  • POST201 + Location upload URL
  • HEAD → current offset (resume after crash)
  • PATCH → append chunk, response new offset
  • Checksum extension optional

Termination: DELETE to abort.

S3 MPU edge cases

  • Incomplete MPU — parts stored, object invisible; lifecycle abort after N days avoids storage leak
  • Part size — last part can be smaller; min 5 MB except last (except last part rule)
  • CopyObject for server-side assemble when client uploaded parts out of order (must fix order at complete)
  • Checksum — new data integrity features; verify on complete

Client SDK design

  • Parallel part uploads (4–8 workers)
  • Exponential backoff on 5xx
  • Persist uploadId + completed part ETags locally (IndexedDB)
  • Progress callbacks for UI
  • Cancel → AbortMultipartUpload

Download resume

Range: bytes=N- on GET; store partial file + metadata. HTTP 416 when range invalid.

Comparison

tusS3 MPU
AbstractionProtocolCloud API
PortabilityAny tus serverS3-compatible
ResumeCore specListParts

Many products: tus in front → translates to S3 MPU server-side.

Further Reading

Hands-On Tasks (Optional)

API design drills and whiteboard exercises — protocol selection, contract design, and bulk-transfer architecture. Assumes Networking and sibling tracks on the hub page (Distributed Systems, Databases, Concurrency, LLD).

  • Design MPU lifecycle policy

    Orphaned multipart uploads after client crash. S3 lifecycle abort rules, server-side tracking, and billing impact of incomplete uploads.

    15m