Security for Engineers/Data Privacy & Compliance

GDPR & Retention (Engineer View)

Right to erasure, data minimization, retention TTLs, cross-border transfer awareness, and pipeline implications for DE.

3/5Overview: 25m

Engineer-relevant GDPR rights (not legal advice)

RightEngineering action
AccessExport user's data API
ErasureDelete/anonymize across systems
PortabilityMachine-readable export
RectificationUpdate flows with audit

Legal/compliance teams define policy; engineers implement data map and deletion pipelines.

Right to erasure complexity

User in: Postgres, Redis cache, Elasticsearch, S3 backups, Kafka compacted topic, warehouse partition.

Tombstone event + async purge vs synchronous delete all stores.

Document residual copies (backups expire in 30 days).

Retention TTLs

S3 lifecycle, partition drops, DELETE WHERE created_at < ....

Automate — manual retention fails audits.

Cross-border transfer (awareness)

EU data in EU region; SCCs/BCRs for transfers — architecture decision (region pinning).

Microservices Deep Cuts — multi-region; legal nuance out of scope.

Data engineer interview

"PII in lakehouse?" — bronze restricted ACL, tokenize in silver, aggregate-only gold, column masks in BI tool, lineage for audit.

Anonymization vs pseudonymization

  • Pseudonymous — reversible with key (still regulated)
  • Anonymous — cannot re-identify (statistical thresholds)

K-anonymity mention enough; don't dive into differential privacy unless AI/ML role.

Cross-reference: Data Engineering → Quality & Governance for contracts and lineage implementation.

Further Reading

Hands-On Tasks (Optional)

Security design drills — threat modeling, auth flows, and incident playbooks. Assumes Networking (TLS) fundamentals.

  • Design user deletion across microservices

    GDPR delete request: orders, analytics events, S3 exports, search index. Sync vs async purge; tombstone vs hard delete; audit trail.

    20m