ITC global ecosystem is in transition to a new generation of applications that require data acquisition, processing and intensive storage systems from all kind of sensors that generate immense amounts of data. To provide the need of high-performance and very low processing latency, such complex applications, in many cases are structured as workflows that process data through a combination of data streams distributed in Edge Computing environments with Cloud processing. However, existing frameworks are very complex, require many resources, which are not usually available in Edge Computing systems, and do not adapt well to the execution of complex applications that require high performance and low latency due to lack of global coordination. For this type of applications, the great disparity between computing power and data management remains a problem, as well as the lack of adequate programming mechanisms and models to efficiently exploit the performance offered by hybrid architectures jointly formed by High-End (HEC) and Edge Computing platforms. There are other pending challenges such as the possibility of being able to deploy high-performance applications on the nodes of the system edge (Edge) that can filter/add data with low latency, have the ability to move applications and data dynamically through different layers according to the needs of the system, as well as the use of collaborative mechanisms to ensure and trace compliance with data requirements and execution conditions.
The main objective of this proposal is to design new techniques that combine the facilities of High-End Computing and Edge Computing for the realization of an efficient decentralized and hybrid environment for the execution of complex data intensive applications using the resources of the chain Optimally and safely. The specific objectives of the project are:
- To design an environment for the execution of data intensive applications in HEC and Edge Computing hybrid environments;
- To design and develop programming models at node level and distributed for applications to run in hybrid environments;
- To design and develop data management methods that integrate heterogeneous resources from HEC and Edge Computing;
- To design communication mechanisms coordination and cooperation between different levels of framework; evaluate the feasibility of the proposed solutions through use cases.