At the highest layer, the Distributed Data Analytics Platform (DDAP) deals with the management, storage, and retrieval of data at the time it is needed and at the location where it is needed. It also offers the necessary Application Programming Interfaces (APIs) to the programmer, enabling the development of extreme data analytics workflows able to leverage the distributed nature of the ELASTIC architecture.
The figure below shows an overview of the DDAP deployed for the ELASTIC smart mobility use cases.
The two key software components of DDAP are:
- dataClay, employed for data-in-motion generated at the edge and consumed by real-time data analytics methods, and
- Druid aggregating all data for offline data-at-rest analysis taking place in the cloud.
Other software components, such as Kafka, were employed as an intermediate step for data transition from the edge to the cloud.
Two data analytics platforms, composed of microservices for data extraction, data aggregation, data cleaning, data filtering, and geo triangulation, were also developed as part of the DDAP:
- The ICE Knowledge Discovery tool is employed for the offline analysis of datasets coming from the connected trams and the city infrastructure as part of the public-private transport interaction use case.
- The Predator tool implements learning algorithms for the detection and potential prediction of defective track segments as part of the predictive maintenance use case.
Check all the details in the ELASTIC GitLab sub-group repository: https://gitlab.bsc.es/elastic-h2020/elastic-sa/ddap