The ΔQ oscilloscope: Real-time observation of large erlang applications using ΔQSD

Details

Supervisors

Van Roy, Peter

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] en sciences informatiques, à finalité spécialisée

Abstract

It is difficult to study the detailed behaviour of large distributed systems while they are running. Many important questions are hard to answer. What happens when there is an overload? How can we feel something is wrong with the system early enough? The purpose of the thesis is to create the ∆Q oscilloscope, a real time graphical dashboard that can be used to study the behaviour of running Erlang systems and explore tradeoffs in system design. It is based on the principles of ∆QSD. Furthermore, we have developed an Erlang interface, the ∆Q adapter (dqsd_otel). It allows sending real time insights about the running system to the oscilloscope. The adapter works on top of the OpenTelemetry framework macros. This allows for users who have already instrumented their applications with OpenTelemetry to easily integrate the interface. The oscilloscope performs statistical computations on the time series data it receives from the adapter and displays the results in real time, thanks to the ∆QSD paradigm. We provide a set of triggers to capture rare events, like an oscilloscope would, and give a snapshot of the system under observation, as if it was frozen in time. An implementation of a textual syntax allows the creation of outcome diagrams which give an “observational view” of the system. Additionally, the implementation of efficient algorithms for complex operations, such as convolution, allows for the computations to be done rapidly on precise representations of components. We introduce the work by giving a summary of ∆QSD concepts. We also provide a summary of the observability tools available for Erlang, namely, OpenTelemetry. We then present the overall design of the project, describing how to build a bridge from OpenTelemetry to the oscilloscope. Subsequently, we explain the user level concepts which are essential to understand how the oscilloscope works and understand what is displayed on the screen, delving later on into the mathematical foundations of the concepts. Lastly, we provide synthetic applications which prove the soundness of ∆QSD and show how the oscilloscope is able to detect problems in a running system, diagnose it and explore design tradeoffs.