Written by Álvaro González Vila (IKERLAN Technology Research Centre, Basque Research and Technology Alliance - BRTA)

Nowadays, we live in an increasingly connected world, where more and more people have access to the internet through the use of computers, smartphones or tablets, among others. Additionally, devices of different nature are connected to the internet by themselves, without requiring human-to-human or human-to-computer interaction, shaping what we currently know as Internet of Things, or IoT [1]. The use of these devices to carry out some autonomous computation has been possible thanks to the progressive increase in computing capacity, leading to powerful mobile or Edge Computing. Consequently, Fog Computing architectures have emerged during the recent years, distributing the computational load in both the Edge devices and the devices in between the Edge and the Cloud, such as switches, routers, workstations.

Fog Computing brings the benefits of Edge and Cloud computing together [2], with the Edge devices continuously growing in terms of computing capability and the Cloud infrastructures having matured substantially both in availability and scalability.  While Fog Computing has been in the spotlight by both the research and industry communities, just its foundations have been named, being the holistic implementation of the concept still missing.

There are many scenarios where Edge Computing is crucial [3], such as assisted-driving systems, medical monitoring, smart structures, etc. Fog Computing reduces the amount of data traditionally interchanged between these devices and the Cloud or datacenter, being able to work with unreliable network connections and reduce the information latency to fulfill the final user requirements, i.e. for real-time applications. As shown in the Figure [4], the Fog Computing architecture could be schematically located in an intermediate layer between the Edge devices and the Cloud infrastructure, consequently bringing the computing capabilities closer to the devices themselves. The aim of ELASTIC is therefore obtaining the best from this kind of architecture and demonstrating its application on a Smart Mobility use-case scenario.

Overview of Fog Computing architecture. Obtained from [4].

In a traditional IoT architecture, data is moved from the devices to the Cloud and, when required, decisions are sent back from the Cloud to the devices, which is generally known as Data Movement. Fog Computing extends this perspective with the concept of Computation Movement, being some processes executed directly in the devices while others are sent to intermediate computing resources to finally leave the most demanding ones to be computed in the Cloud. Dynamic data and processing management enables a distributed schema at different levels of the Fog architecture, allocating resources according to different criteria. This is complemented with storage capability provided in all computing resources, with the possibility of using distributed storage solutions as well [5].

Among its benefits, Fog Computing makes explicit use of various computing layers, each of them with different capabilities. Layers close to the devices exhibit faster response times, providing the closest to real-time computation. In a complementary manner, the Cloud offers higher computing resources, which can be used for heavier tasks. In the meanwhile, computing nodes from the intermediate layers can carry out a variety of tasks. The elasticity concept reduces network usage, improves the response times, and allows the fulfilment of non-functional requirements, providing an added value for the full architecture.

Some of the requirements that will be satisfied in the Fog Computing architecture proposed in the ELASTIC Software Architecture and for the specific scenarios of the Smart Mobility use-cases in the city of Florence will be the following:

- The Fog Computing architecture will have at least three layers: the sensor layer, the station layer, and the Cloud. Information will flow between the layers for distributed storage, processing and decision making.
- The Edge nodes will be able to temporarily store and associate different kinds of data gathered locally by physically connected devices. By doing so, a data buffer will be available in case an issue arises, either at the node or the network side.
- The system will provide long term storage for big data in the cloud, in order to perform demanding data analytics and preserve historical data.
- Part of the obtained data will be transferred to the system to be stored and associated with global data, such as videos or maps, so data consistency will be achieved.
- The system will synchronize data between architectural levels without harming performance, by transferring only the necessary data and only when required.

References:

[1] P. Asghari, A.M.Rahmani, H.H.S. Javadi, “Internet of Things applications: A systematic review”, Computer Networks 148, 241-261 (2019). DOI: 10.1016/j.comnet.2018.12.008
[2] C. Puliafito, E. Mingozzi, F. Longo, A. Puliafito, O. Rana, “Fog computing for the internet of things: A survey”, ACM Transactions on Internet Technology (TOIT) 19(2), 1-41 (2019). DOI: 10.1145/3301443
[3] W.Z. Khan, E. Ahmed, S. Hakak, I. Yaqoob, A. Ahmed, “Edge computing: A survey”, Future Generation Computer Systems 97, 219-235 (2019). DOI: 10.1016/j.future.2019.02.050
[4] Cisco Systems, Inc., USA, “Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are”, White Paper, April 2015. URL: https://www.cisco.com/c/dam/en_us/solutions/trends/iot/docs/computing-overview.pdf
[5] J. Martí, A. Queralt, D. Gasull, A. Barceló, J.J. Costa, T. Cortes, “Dataclay: A distributed data store for effective inter-player data sharing”, Journal of Systems and Software 131, 129-145 (2017). DOI: 10.1016/j.jss.2017.05.080

As with many other meetings and events during the past months, our second project meeting had to be done remotely due to the Covid-19 constraints. The first day was packed with presentations and video demos from the technical work packages (WP) on the smart mobility use case and data analytics platform. The second day was dedicated to more technical discussions on the software and computing architecture, as well as dissemination, exploitation, and project management tasks.

Although not the same as meeting physically, we managed to make the most out of it and also learnt a few interesting lessons.

•    Structure and flexibility: The meeting was structured as a physical meeting with consequent presentations by each WP. However, the agenda had to be maneuvered around depending on the needs for more depth in technical discussions while everyone was able to ask for clarifications at any time. Frequent breaks helped to maintain concentration.
•    Demonstrations: We found the video demos presented by the technical WPs particularly helpful. They provided a visual way to demonstrate the progress in the ELASTIC use case and software platforms.

Snapshots from the THALIT demos

•    Slides: The same principle applies to presentation slides that are ought to be interactive with rich infographics that are easier to engage with. If possible, it would help to send around the slides in advance so participants can have a look and come prepared.
•    Break out rooms: We did not use break out rooms for smaller group discussions this time. However, it is a useful feature in order to mix representatives from different partners for more in-depth queries. Specific technical meetings will follow to address these needs.
•    Finally, don’t forget the group photo/screenshot!

All in all, the meeting proved very fruitful considering the circumstances. The digital format worked out well and the consortium felt the usefulness of the discussions in order to proceed with the project developments.

We may have missed our physical meeting in Porto, Portugal, that would have been hosted by our partner ISEP this time, but we will make sure to keep working productively from home for as long as it is needed and to reschedule our date. Until next time!

Flavia Gaudio is an engineer at Gestione Ed Eserizio del Sistema Tranviario SPA (GEST). Her research and professional interests revolve around Rolling stock maintenance and railway safety systems. She is a member of the ELASTIC project, working on the predictive maintenance use case and sensors installation on trams. In this interview, Flavia talks about her experience as a woman in STEM and as a member of the ELASTIC team.

• How did you become interested in engineering? What influenced your decision in taking this career path?

I’ve been always fascinated by science and technology since I was a little girl. My dream at first was to be an astronaut, as I love the idea of learning and discovering new things. I’ve always believed that technology can improve quality of life and can help humans in challenging situations. Therefore, I decided to choose electronic engineering as my study path, believing it was the best way to build my future in this field.

• How has your experience as a woman studying and working in STEM been? Have you faced any challenges?

I think I struggled a bit at first to gain the respect of my male schoolmates and of my male colleagues afterwards, but then I built with them a very good environment and cooperation that has always helped me to grow as a professional and as a person.

It might be a matter of “first sight”, when some people see a woman in an environment traditionally managed by males. You need to be yourself, work hard and be simple, and then you see things change for the better day by day.

• What are you working on in the ELASTIC project and how has this experience been so far?

It is the first time I took part in a European Project and I found it challenging and exciting at the same time.  We need to invest a lot of energy in it, but we are also supported by experienced colleagues from all over Europe, and this makes the project a precious space to learn and grow.

I work on the predictive maintenance use case - in particular, I take care of sensors installation on trams and maintenance vehicles in order to collect data to be used by the other project partners.

• What message would you give to young girls and women who are interested in pursuing a career in STEM?

Women shouldn’t be afraid to work in STEM. There is still a lot of work to be done on gender stereotypes in this field and we can all be a part of it. I encourage them to be confident about their capabilities and motivation. I can’t wait to see more girls joining the field.

Barcelona, 21 May 2020 - The innovative software architecture developed by the EU-funded project ELASTIC is being applied to a real smart city use case on the public tram vehicles in the city of Florence, Italy. The ELASTIC framework is fostering next-generation urban mobility applications for tramway solutions through the support of big data and extreme-scale analytics.

Current trends in the field of big data analytics in the context of smart cities show a need for original software ecosystems upon which advanced mobility functionalities can be developed. These new architectures need to be able to collect and process a vast amount of data, coming from geographically dispersed data sources, and convert it to valuable knowledge for the public sector, private enterprises and citizens.

The ELASTIC technology is facing this challenge by developing a software framework capable of exploiting the distributed computing capabilities of the compute continuum of the smart city, while guaranteeing additional properties, known as non-functional requirements of the system: the real-time, energy, communication quality and security.

“We are excited to apply a smart mobility use case that will help for better and safer public transportation”, states Eduardo Quiñones, senior researcher at the Barcelona Supercomputing Center (BSC) and coordinator of the ELASTIC project. “By creating a novel software architecture for extreme-scale analytics, ELASTIC will form the technological basis for advanced mobility systems and autonomous transport networks.”

The following smart mobility applications are being elaborated on to show the great potential of the ELASTIC technology for advanced tramway solutions:

• Obstacle detection: By combining data from different sensors, it helps to detect obstacles and avoid collisions in real time using Next Generation Autonomous Positioning (NGAP) and Advanced Driving Assistant System (ADAS).
• Predictive maintenance: It detects maintenance needs at an early stage in order to minimise operational costs and increase the reliability of the service.
• Public/private transport interaction: It provides support to assess the overall traffic conditions and enables enforcement of control strategies for a more efficient interaction between public and private transportation.

Watch the ELASTIC video to learn about the project’s technology and smart city use case here.

About ELASTIC

ELASTIC (A Software Architecture for Extreme-ScaLe Big-Data AnalyticS in Fog CompuTIng ECosystems) is a European-funded project with a budget of €5.9 million, which started on 1 December 2018 and lasts for three years. Coordinated by the BSC, the project brings together a multidisciplinary consortium of stakeholders from smart mobility and research domain sectors: Barcelona Supercomputing Center (BSC, Spain), Ikerlan (Spain), Instituto Superior da Engenharia do Porto (ISEP, Portugal), Information Catalyst (ICE, UK), SixSq (Switzerland), Thales TRT (France), Thales Italia (Italy), Gestione ed Esercizio del Sistema Tranviario (GEST, Italy) and Città Metropolitana di Firenze (Italy).

The ELASTIC project has received funding from the European Union’s Horizon 2020 research and innovation programme under the grant agreement Nº 825473.

Download the Press Release in PDF.

Written by Luis Miguel Pinho (School of Engineering, Polytechnic Institute of Porto).

One of the main challenges to be tackled by ELASTIC is the necessity to fulfill the non-functional properties inherited from smart systems, such as real-time, energy efficiency, communication quality or security. In these systems, large volumes of data are collected from distributed sensors, transformed, processed and analysed, through a range of hardware and software stages from the physical world sensors (commonly referred to as edge computing), to the analytics back-bone in the data-centres (commonly referred to as cloud computing).

This complex and heterogeneous layout presents several challenges, of which an important one refers to non-functional properties inherited from the application domain including real-time, energy-efficiency, quality of communications and security:

• Real-time data analytics is becoming a main pillar in industrial and societal ecosystems. The combination of different data sources and prediction models within real-time control loops, will have an unprecedented impact in domains such as smart cities. Unfortunately, the use of remote cloud technologies makes challenging to provide strict real-time guarantees due to the large and unpredictable communication costs on cloud environments.
• Mobility shows even increased trade-offs and technological difficulties. Mobile devices are largely constrained by the access to energy, as well as suffering from unstable communication, which may increase random communication delays, unstable data throughput, loss of data and temporal unavailability.
• Security is a continuously growing priority for organizations of any size, as it affects data integrity, confidentiality and potentially impacting safety. However, strict security policy management may hinder the communication among services and applications, shrinking overall performance and real-time guarantees.

Overall, while processing time and energy cost of computation is reduced as data analytics is moved to the cloud, the end-to-end communication delay and the performance of the system (in terms of latency) increases and becomes unpredictable, making not possible to derive real-time guarantees. Moreover, as computation is moved to the cloud, the required level of security increases to minimise potential attacks, which may end up affecting the safety assurance levels, hindering the execution and data exchange among edge and cloud resources.

It is thus necessary that the ELASTIC architecture includes mechanisms which allow the specification of the required level of non-functional properties, the offline analysis of these parameters to determine an appropriate system configuration which enables their fulfillment,  and an online monitoring and analysis capability which is able to trigger configuration changes upon detection of level violations. This will be provided via the Non-Functional Requirements (NFR) tool of the ELASTIC Software Architecture.

Non-Functional Requirements (NFR) tool

Contemporary cloud computing solutions, both research projects and commercial products, have mainly focused on providing functionalities at levels close to the infrastructure. Furthermore, they tend to focus on functional aspects only. In order to provide an improved ecosystem, which considers the full compute continuum, there is a great need for analysis and monitoring tools that support higher-level concerns and non-functional aspects in a comprehensive manner, from the edge to the cloud. Therefore, the NFR tool of the ELASTIC architecture will operate both at the analysis phase, and during execution.

The goal of the analysis phase is to guarantee the fulfilment of the system non-functional properties, considering the potential trade-offs between performance, predictability, energy-efficiency, communication quality and security. The result of this analysis is a set of possible initial deployment configurations. This phase carefully identifies how satisfying and fulfilling one requirement can impair the satisfaction of other requirements in the system. Establishing and maintaining such interdependencies during the development process and the lifecycle of the system is also an important point, taking into account the evolution of the software architecture and the introduction of new requirements or the modification of existing ones.

Deployment decisions should be made in light of a target system, aiming for high quality of the system deployed under given constraints. However, in order to support deployment decisions, it is essential to identify concrete measures as a basis for decision making and evaluation of the proposed solutions. Such measures need to be dynamic and distributed along the compute continuum, in the case of systems that evolve continuously as the workloads, allocated resources and requirements of these systems change over time. Therefore, runtime monitoring of requirements is used to guide this evolution towards configurations that are guaranteed to satisfy the system’s overall requirements. Monitoring identifies the scenario the system operates in, and selects a model whose quantitative verification enables the detection or, sometimes, prediction of violations. The subsequent execution of a correct reconfiguration plan helps the system to re-instate or maintain compliance with the expected level of service. The monitoring phase makes use of the ELASTIC software architecture ability to provide information on the resource usage and application execution in the nodes, and dynamically re-map and schedule components considering the execution profile identified by the monitor.

Elli Kartsakli is a researcher at the Computer Sciences Department at the Barcelona Supercomputing Center (BSC). Her research interests revolve around wireless telecommunications and the 5G evolution. She is part of the ELASTIC project, working on how to model the impact of wireless transfer times to the execution of distributed tasks at the network edge. In this interview, Elli talks about her experience as a woman in STEM and as a member of the ELASTIC team.

• How did you become interested in computer science? What influenced your decision in taking this career path?

Maths and physics have always been my favorite subjects at school, and when the time came to select a career path, I didn’t hesitate to choose electrical engineering and computer sciences. My dream was to be involved in robotics, but then I became fascinated by wireless telecommunications and networking.  I also love problem solving and critical thinking, so this career direction matched my personality and skills.

• How has your experience as a woman studying and working in STEM been? Have you faced any challenges?

During my university studies both in Greece and in Spain, from bachelor to PhD, I was one of very few women in my year. This felt strange at the beginning, but I was very lucky and never faced any problem with my professors or fellow students. I think that the academia is a relatively protected environment for women, even though we still have a long way to go to achieve true gender equality. The most challenging experience so far for me has been to balance motherhood with career advancement, but luckily, working as a researcher gave me some degree of flexibility in terms of time management. 

• What are you working on in the ELASTIC project and how has this experience been so far?

ELASTIC leverages distributed computing across the computing continuum, from edge to cloud, for extreme scale analytics. Towards the network edge, most devices are connected through wireless technologies such as WiFi or cellular communications. However, wireless network environments are highly volatile and the link quality varies with time. This uncertainty brings new challenges in the scheduling of computing tasks, since the properties of the communication channel must be taken in to account. My role in ELASTIC is to help incorporate the communication delays in the scheduling of the distributed tasks, in order to provide specific time guarantees

• What message would you give to young girls and women who are interested in pursuing a career in STEM?

I would encourage all girls and young women to pursue whichever career path they like without feeling restrained or intimidated by gender stereotypes. Working in STEM is both challenging and rewarding and anyone with an interest in technology and sciences should definitely give it a try.

We are all experiencing a challenging time during the Covid-19 pandemic and trying to find our ways of understanding and adapting to the situation. The ELASTIC team, as many others lately, is working from home. However, this has not kept us from carrying on with dissemination tasks and activities! In an attempt to continue with the project’s communication efforts and to offer different material for our audience to read during the lockdown, we have gathered all the latest ELASTIC resources. Happy reading!

• Our new Software Infrastructure page offers information on the ELASTIC novel software architecture and a description of the different layers integrated into it.
• Our new Use case page provides details on the project’s real smart city use case implemented in the public tram network of the city of Florence. It also includes details on the three applications identified to assess the benefits of the ELASTIC technology for newly conceived tramway solutions.
• Our ELASTIC entry in the BDVA Marketplace, addressed particularly to industries and corporate audiences. It explains the project’s technological novelty, how it fits in certain market areas, and what benefits there are for potential customers.
• An article written by the EC Cordis journalist team that describes how ELASTIC is laying the groundwork for autonomous transport networks by developing a novel software architecture for big data analytics to be applied in urban mobility systems and smart cities.
• A project video is also on its way. It will demonstrate ELASTIC’s technology, a short interview with the project’s coordinator Eduardo Quiñones, and scenes from the tram vehicles in Florence describing the smart city use case. Stay tuned!

We hope these resources prove useful during this “Staying at home” challenge and beyond. Don’t forget to keep up to date with our latest news and developments by checking our project website, Twitter and LinkedIn accounts.

Written by Maria A. Serrano (Barcelona Supercomputing Center).

Current trends towards the use of big data analytics in the context of smart cities suggest the need of novel software development ecosystems upon which advanced mobility functionalities can be developed. These new ecosystems must have the capability of collecting and processing vast amount of geographically-distributed data, and transform it into valuable knowledge for public sector, private companies and citizens. ELASTIC is facing this need by developing a software architecture framework capable of efficiently exploiting the computing capabilities of the compute continuum, while guaranteeing the real-time, energy, communication quality and security non-function properties of the system.

COMPSs [1] is at the heart of the software architecture: it is the component responsible for efficiently distributing, across the compute continuum, the different data analytics methods, each described as a COMPSs workflow. Moreover, COMPSs includes the deployment capabilities to interact with hybrid resources in a transparent way for the programmer. A novel elasticity concept is considered to dynamically adapt the workflow distribution whilst fulfilling non-functional properties. Next, the programming model and the deployment capabilities of COMPSs are described.

COMPSs programming framework

COMPSs provides a simple, yet powerful, tasking programming model, in which the programmer identifies the data analytics functions, named COMPSs tasks, on top of general purpose programming languages, such as Python, Java or C/C++. The COMPSs runtime is then in charge of selecting the most suitable computing resources in which tasks can execute, while maintaining the hybrid fog computing platform transparent to the programmer. A COMPSs data analytics workflow can be represented as a Direct Acyclic Graph (DAG) to express its parallelism. Each node corresponds to a COMPSs task and edges represent data dependencies between them. As an example, Figure 1 represents the DAG of a COMPSs data analytics workflow composed of five tasks (1,… 5) and four data dependencies between them (A, B, C, D). For instance, due to data dependency B, Task 4 must execute after Task 2 finishes its execution.

Figure 1. DAG representation of a COMPSs data analytics workflow.

COMPSs deployment capabilities

One of the main features of COMPSs is that the model abstracts the application from the underlying distributed infrastructure, hence COMPSs programs do not include any detail that could tie them to a particular platform boosting portability among diverse infrastructures and enabling execution in fog  environments. It is the COMPSs runtime the one that features the capabilities to setup the execution environment.

The COMPSs runtime is organised as a master-worker structure. The Master executes in the resource where the application is launched, and it is responsible for steering the distribution of the application, as well as for implementing most of the features for initialising the execution environment, processing tasks, or data management. The Worker(s) are in charge of responding to task execution requests coming from the Master. There are three different scenarios currently supported for the deployment of COMPSs workers:

•    Native Linux: data analytics tasks running directly on top of the OS.
•    Docker containerized: data analytics tasks are encapsulated in Docker containers, running in a Docker platform [2].
•    Cloud: data analytics tasks are encapsulated in Docker containers, running in a Docker Swarm [2] or Kubernetes [3] cloud platform. In the ELASTIC project this deployment is done through Nuvla [4].

As an example, Figure 2 shows the master-worker execution environment for the workflow in Figure 1. It is composed a COMPSs master, running in Node 1, where the workflow starts, and four extra COMPSs workers running in:

•    Node 2, one COMPSs worker deployed as a Docker container,
•    Node 3, one COMPSs worker deployed as a native Linux execution, and
•    Cloud, two COMPSs workers deployed through the Nuvla API.

Figure 2. COMPSs master-worker execution environment.

In this example, tasks 1 and 4 are executed in a container in the Cloud; task 2 is executed in a different container, also in the Cloud; and tasks 3 and 5 are executed in Nodes 2, and 3, respectively.

This kind of execution scenarios, managed by COMPSs, ensure the coordination and interoperability of different edge/cloud resources, promoting a novel elascity concept among, not only cloud computing resources, but also edge devices.

References

[1] COMP Superscalar (COMPSs) http://compss.bsc.es/
[2] Docker and Docker Swarm https://www.docker.com/
[3] Kubernetes (K8s) https://kubernetes.io/
[4] Nuvla https://nuvla.io/