Files
HAFHOUF_5997200_KHALIL_67382000.pdf
Open access - Adobe PDF
- 12.55 MB
Details
- Supervisors
- Faculty
- Degree label
- Abstract
- Privacy protection in the context of real-time applications that continuously process user data, such as IoT sensor networks, healthcare monitoring, or financial fraud detection, is extremely valuable to protect the user. In recent years, Differential privacy (DP) has emerged as the gold standard for formal privacy guarantees, yet its practical deployment in distributed stream processing engines remains largely unexplored. Recent theoretical work aims to close the gap by providing a state-of-the-art DP mechanism for infinite streams. This thesis investigates the integration of the state-of-the-art user-level Differential Privacy algorithm into Apache Flink, an open-source distributed stream processing engine. We deliver a fault-tolerant implementation of the mechanism, coupled with Apache Kafka, and evaluate it on eighteen real-world datasets spanning multiple domains. Through this evaluation, we identify the failure point of the algorithm, caused by skewed user participation in the stream. This leads to stalling in the algorithm’s internal mechanics, which in turn can yield extremely high error and very low utility. We conduct a com- prehensive evaluation covering both utility and performance. On the performance side, our system achieves a sustained throughput of approximately 595, 000k events/s, incurs a CPU overhead of only 30%, and adds fewer than 5μs of operator latency per event, demonstrating that strong user-level privacy guarantees are compatible with high-speed production environments. While the mechanism yields good utility on datasets with large and balanced user populations, it breaks down entirely on datasets where a small number of users dominate the stream. In those cases, the relative error can approach 100%, rendering the released counts practically useless. This reveals a fundamental gap between theory and practice: the formal utility guarantees of the mechanism hold under data conditions that some real-world streams do not satisfy.