A Taxonomy for Management and Optimization of Multiple Resources in Edge Computing

Edge computing is promoted to meet increasing performance needs of data-driven services using computational and storage resources close to the end devices, at the edge of the current network. To achieve higher performance in this new paradigm one has to consider how to combine the efficiency of resource usage at all three layers of architecture: end devices, edge devices, and the cloud. While cloud capacity is elastically extendable, end devices and edge devices are to various degrees resource-constrained. Hence, an efficient resource management is essential to make edge computing a reality. In this work, we first present terminology and architectures used within the field of edge computing. Then, we review recent literature, consisting of a wide range of papers and categorise them using 4 perspectives, namely resource type, resource management objective, resource location, and resource use. This taxonomy and the ensuing analysis is used to identify some gaps in the research currently conducted. While the combination of dynamic load changes and mobility would seem to require novel edge-based resource estimation and discovery methods, we found these areas less studied. As for resource types, we observe that energy and data as a resource are not as well-studied as computation and communication resources. Finally, we find that few works are dedicated to the study of the footprint of resource management techniques.


INTRODUCTION
R ECENTLY, the edge computing paradigm, which consists in having network nodes with computational and storage resources close to the devices (mobile phones, sensors), at the edge of the current network, has attracted interest from both industry and researchers, carrying the promise of a new communication era in which industry can meet the rising performance needs of future applications.
Indeed, with a forecast of 9 billion mobile subscriptions in the world by 2022, of which 90% will include mobile broadband, coupled to an eightfold increase in mobile traffic and 17.6 billion of Internet of Things (IoT) devices also sending data [1], there will be a considerable strain put on the network. The current network technologies need to undergo a paradigm shift in order to handle this situation [2]. Therefore, the aim is to avoid overwhelming the network up to the cloud, and, when possible, move some computing and data analysis closer to the users, to enable better scalability [3]. Thus, the main idea of edge (or fog) computing is to have intermediate computing facilities between the end devices and the current cloud. As suggested by Amardeep et al. [4], this would also enable the current telecom network operators to reduce their operational costs.
In addition to this, moving computing and storage to the edge of the network has other benefits [3] such as reducing the latency and jitter [5], which is especially important for real-time applications such as self-driving cars. Moreover, it enables more privacy for the users by making it possible to keep private data at the edge and enforce privacy policies • K. Toczé  for the data sent to the cloud (such as blurring sensitive info on a video [2]). Finally, edge networking makes the applications more resilient by being able to process requests at the edge even if the central cloud is down. In order to achieve this and to make edge computing a reality and a success, there is a need for an efficient resource management at the edge. Indeed, mobile devices or IoT devices are resource-constrained devices, whereas the cloud has almost unlimited but far away resources. Providing and/or managing the resources at the edge will enable the end device to spare resources (e.g. stored energy in batteries), speed up computation, and allows using resources it does not possess. Moreover, keeping data close to where it was generated enables better control, especially for privacy related issues. Finally, being located close to the user, edge computing makes it possible to increase the quality of provided services through the use of profiling within a local context, without compromising the privacy or having to handle a large number of users. This is know as context adaptation.
Even though this is still an emerging research area, there is a lot of work ongoing under different denominations: mobile cloud computing [6], fog computing [7], edge computing [3], mobile edge computing [8], mobile edge cloud [9], mobile edge network [4], infinite cloud [10], follow-me cloud [11], multi-tier cloud federations [12] and distributed clouds [13]. A common concept here is an intermediate level between the device and the traditional cloud. It is possible to find in the literature numerous surveys about those paradigms in general [6], [9], [14], [15], [16], [17], [18] or specific aspects of them such as security [8], [19]. However, those typically do not consider the resource aspect. The existing surveys about resources either consider it at a high level [20], consider only resource/service provisioning met- rics [21], or focus on offloading [22], [23], [24], i.e. executing a task in another device than the requesting device, usually one having more powerful computational capacities. However, there are more areas in resource management than just offloading, which this survey aims to complement, by providing an overview of work where the resources are at the edge or where the resource management is performed at the edge. Therefore, work considering direct interactions from a device to a cloud [25], from the cloud to the edge [26] or with interactions towards edge/cloud but where the resources are in the cloud [27] are not considered. This paper is a substantial extension of our previous much shorter review [28], and will provide a current exposure to edge computing with resource management as a focus.
In the remaining parts of this paper, we will explain the terminology used in Section 2, and then propose and describe a taxonomy of resource management at the edge in Section 3. The taxonomy is exemplified with an extensive review of papers, which are then categorised using the (sub)categories. Section 4 will summarise our findings and the identified gaps.

TERMINOLOGY
Edge computing is an innovative area bringing together previously separated parties such as telecommunication actors, vehicle vendors, cloud providers, and emerging application or device providers e.g.for augmented reality. Therefore, the terminology used in research works is diverse and still evolving. In this section, we present the relevant concepts associated with edge computing that will be used in the rest of the paper.
Following the development of the IoT, it is nowadays not only computers or smartphones which can be connected to the network, but a large variety of things such as cars, sensors, drones, robots, or home appliances. In this survey, all those objects located at the user end of the network, which are producing data or need cloud/edge resources will be called end devices. Devices installed at the edge specifically for edge computing purposes are called edge devices. We also include under this term the devices that are already now connecting the end devices to the rest of the network, for example home routers, gateways, access points, or base stations which are becoming increasingly powerful [29]. Finally, physical components of the cloud are referred to by the term cloud devices. Moreover, we consider those three network areas as different levels in the network: the device level, the edge level and the cloud level. Resources which are managed are used to perform tasks, which can be composed to provide a service to the user.
There is currently no standard architecture for edge computing, although industry and research initiatives exist such as the Open Edge Computing 1 community, the Open Fog Consortium 2 , and a European Telecommunications Standards Institute (ETSI) standardization group working on Multi-access Edge Computing 3 . Therefore, current research on edge computing is using several different architectures and there is ongoing work for defining edge computing architectures [9]. In this survey, we classify the different architectures into three main categories inspired by the work of Mtibaa et al. [30] and presented in Figure 1.
The first category, named Edge server and depicted in Figure 1a, is a generic architecture where devices are connected to an edge server, which itself is connected to the rest of the network, including the cloud. In this type of architecture, the edge server is at a fixed physical location and has relatively high computational power, though it remains less powerful than a conventional datacenter used in the cloud computing paradigm. Moreover, there is a clear separation between the device level and the edge level. In the literature, such edge servers are named for example cloudlets [31], [32], micro datacenters [33], nano datacenters [34] or local cloud [35]. They can be located for example in shops, enterprises, or colocated with the base stations of the telecom access network. Indeed, in the ongoing work on what the fifth generation (5G) of telecommunication networks will look like, a cloud radio access network (C-RAN) is envisaged [36], [37], with possible interwork with other edge computing areas such as mobile cloud computing [38].
The second category, named Coordinator device and depicted in Figure 1b, is an architecture where one end device acts as a coordinator between the other end devices. It also acts as a proxy towards an edge device and/or the cloud if such connectivity is needed. The difference between a coordinator device and an edge server is that the coordinator device can be mobile and has less computational power and bandwidth than an edge server. In this architecture category, the border between the device level and the edge level is not a sharp one, as the coordinator level providing edge functionality is actually an end device. Solutions using this category of architecture are named for example fog colonies with a control node [39], vehicular clouds with a cluster head [40] and local clouds with a local resource coordinator [41]. It is interesting to note here that the term local cloud, which was already used for describing a part of the edge server architecture category described in the previous paragraph, is used to describe various architectural solutions, illustrating well the fact that the terminology used in edge computing is not yet set.
The last category, named Device cloud and depicted in Figure 1c, is an architecture where the end devices communicate with each other to find needed resources and deliver the wanted services. The devices might communicate with an edge device connected with the cloud if needed but this is not necessary. In this architecture category, the device level and the edge level are thus merged. Research work considering this category of architecture call it opportunistic computing [42], cooperation-based mobile cloud computing [43], [44], or transient clouds [45].
While all these architectures need to be populated with dedicated resource management elements there is no general agreement about where to place the needed policies. A recent proposal for a generic software architecture that encompasses the edge server version in figure 1a is an enabler for evaluation of multiple resource management policies within common test beds [46].

TAXONOMY OF EDGE RESOURCE MANAGEMENT
In this section, we present a taxonomy of resource management at the edge with the aim to get an overview of stateof-the-art research in this area. This taxonomy, illustrated in Figure 2, presents four main aspects: resource types, objective of resource management, resource location, and resource use.
In the coming subsections, we will describe the different parts of the taxonomy, and how they have been addressed in existing work.

Resource type
The first step in evaluating the benefit of an edge solution is to decide what are the resource types that can be managed in a better way compared to a centralised system.

Physical vs. virtual
An obvious justification for using edge architectures is reducing the response time, which can be done if computation and communication resources are provided and used adequately. Storage as a resource is also a concern since local storage may benefit security or timeliness due to customised fetching and secure storing mechanisms. A less obvious type of resource is having access to a special type of data (e.g. availability of sensors) that provides local benefits in an application. Examples are use of cameras or location sensors. The amount and type of data captured in turn affects computation and communication resources (how often to shuffle data, how much to process or filter before shuffling), and implicitly the choice of where and how much of other resources to deploy. The fifth category we consider is energy as a resource, which is clearly influenced by amount of computation, communication, storage and data capturing that goes on.
Some resources are dealt with mainly as physical elements whereas others naturally lend themselves to be defined in abstract ways. For example, sensors are present in the end devices, which can produce useful data needed for the completion of the task (as in [39], [41], [45], [60]), whereas bandwidth (throughput) is a natural abstraction for distinguishing between different radio interfaces, difference physical environments (abstracting the impacts of reduced signal strength, interference, etc).
Some works consider a minimum resource unit that corresponds to a device. For example, Fricker et al. [58] use servers as an abstraction for computing resources (one request occupying one server), and Meng et al. [35] consider one vehicle as the minimal computing resource unit and aggregate them in a resource pool (together with resource units from the cloud and the edge).
Sometimes the VMs are used as a means to ensure that a task can run given that there are enough underlying resources in the device hosting the VM. Some examples are Gu et al. [55] who consider the available computation and storage resources and Tärneberg et al. [56] who consider computation and bandwidth resources.
Liu et al. [47] consider mostly wireless bandwidth and computing resource when deciding whether to handle a request in a cloudlet or in the cloud. Bianzino et al. [65] study the trade-off between bandwidth and energy consumption when an end device has access to multiple networking in-terfaces and can switch between them. Bittencourt et al. [61] consider bandwidth between the cloud and cloudlet, and cloudlet processing capabilities when evaluating different scheduling strategies.
For handling communication, some works consider the communication power needed, either with the aim of minimizing it [52] or as a part of the cost considered when sharing resources [53]. Other works characterize the communication by a delay term impacting the task completion time [39], [42], [62].
Confais et al. [48] present how a storage service can be provided for fog/edge infrastructure, based on the In-terPlanetary file system, and scale-out network-attached systems. Their aim is to propose a service similar to the Amazon Simple Storage Service solution 4 for the edge.
Data as a resource, in addition to sensor data mentioned earlier can also be seen as content. Gomes et al. [11] propose an algorithm for content migration at the edge, together with mobility prediction as an enabler within their new Mobile Follow-me Cloud architecture.
Athwani et al. [64] aim at making resource discovery energy-efficient in order to save battery. Similarly, Mtibaa et al. [67] perform offloading between end devices in order to maximize the group lifetime. Other works presented by Liu et al. [43], Valancius et al. [34] and Nishio et al. [41] consider energy efficiency in their algorithms, but at a more 4. https://aws.amazon.com/fr/s3/ general level, without battery life considerations. Similarly, Bianzino et al. [65] aim for energy efficiency but use an abstract model of power usage based on the amount of data being shuffled.
Still considering energy but with another perspective, Fan et al. [51] present a virtual machine migration scheme which aims at using as much green energy as possible in the context of green cloudlet networks. Similarly, Borylo et al. [54] classify datacenters in two categories (green and brown depending on which source of energy they use) and then use a latency-aware policy to choose a datacenter for serving a request.
Several works aim to combine analysis of multiple resources. Arkian et al. [40] tackle resource issues in vehicular clouds when considering computation, communication and storage resources. Elsewhere, another application area, namely crowd sensing is tackled with the same resource considerations [49].
When modelling multiple resources, the abstraction used varies in various articles. Qi et al. [66] choose to abstract resources as services, whereas Penner et al. [45] work with device capabilities as an abstraction when proposing resource assignment algorithms. Some works, such as Aazam et al. [33], [50], define new units that correspond to a generic notion of a resource. "Virtual Resource Value" is the unit for any resource, which is then mapped to physical resources according to the type of service and current policies of the cloud service provider. Borgia et al. [63] consider on a high-level data-centric service providers having storage, computing and networking capabilities, and Liu et al. [44] abstract the resources available for sharing in what they call "available resources" (e.g. unused CPU cycles or bandwidth) but they also consider bandwidth separately when two nodes are in contact.
Finally, Yousaf et al. [37] emphasize the fact that different resources should not be considered in isolation as there are interactions between them. Thus, they describe and use the concept of resource affinity in their scheme. When considering energy as a resource, a comprehensive discussion of interaction between multiple actions is mapped to energy apportionment policies by Vergara et al. [68]. However, since this work considers edge/cloud specific apportionments as one among many application areas, i.e. addresses energy sharing in a much wider context, we do not further consider it in our classifications. Table 1 summarises the papers mentioned above in terms of their mapping to the architectural choices in Figure 1. It also shows the resources focused on within various works, either specific or generic.

Summary of resource and architecture choices
Examining the collection of papers above, data as a resource is a potential not extensively explored. Similiarly, energy is underrepresented among resources studied. Resource studies so far seem to focus on computation and communication resources to a greater extent. Furthermore, it is noticeable that storage is not the main focus of attention. It could be due to the fact that the cloud is available as a fallback in many cases. It could also be the case that persistent data storage is not the main focus of most of the applications considered here. That is, the service or completed task is the main purpose. Another point to note is that the first architectural instance (Edge server) is the most predominent structure used in the surveyed papers.

Objective
The second aspect represented in this taxonomy is the objective of resource management. Indeed, resource management at the edge can be decomposed into several areas addressing different problems. In Table 2, we present which surveyed article addresses which problem(s) and we describe those problems in the following subsections. As it can be seen in the table, one surveyed work can address several of the areas.

Resource estimation
One of the first requirements in resource management is the ability to estimate how much resource will be needed to complete a task or to carry a load. This is important, especially for being able to handle fluctuations in resource demand while maintaining a good quality of service (QoS) for the user. In particular, resources at the edge can be mobile, and thus become unreachable, which makes them less reliable than resources in a datacenter. Moreover, user mobility implies that there can be sudden user churn, with the corresponding dynamic requests having to be handled by the edge.
In their work, Liu et al. [43] use the average of historic data in order to predict the characteristics of resource distribution and usage for the next time slot. The term fog is used by Aazam et al. [33], who propose that it can be used to perform future resource consumption estimation as a first step for allocating resources in advance. They formulate an estimation mechanism which takes into account the reliability of the customer, using what they call the relinquish probability. In another article, Aazam et al. [50], present the same idea but with an emphasis on how different customers can be charged for the service. Another work by Mtibaa et al. [67] estimates power consumption in order to maximise device life time.
There are of course many earlier works that use sophisticated prediction mechanisms for estimating future loads in cloud environments (e.g. [69]) but our focus has been on edge-related papers and instances of estimation therein.

Resource discovery
As opposed to the estimation problem which relates to the demand side, resource discovery is about the supply side. A management system needs to know which resources are available for use, where they are located and how long they will be available for use (specially if the resource providing device is moving or it is battery-driven). This area is especially important at the edge where every resource is not under the control of the system at all times, so the supply is volatile.
The collaboration at the edge can take the form of clusters, as advocated by Atwhani et al. [64]. They present an algorithm for forming clusters of devices and performing resource discovery within the cluster. From their evaluation with respect to energy consumption and delay, they conclude that maintaining the cluster consumes extra energy, Liu [47] Confais [48] Aazam [33] Arkian [49] Aazam [50] Fan [51] Oueis [52] Tang [53] Borylo [54] Yousaf [37] Wang [38] Gu [55] Tärneberg [56] Plachy [57] Gomes [11] Fricker [58] Rodrigues [59] Zhang [60] Bittencourt [61] Zamani [62] Valancius [34] Coordinator device Nishio [41] Skarlat [39] Borgia [63] Athwani [64] Arkian [40] Penner [45] Bianzino [65] Device cloud Liu [43] Mascitti [42] Liu [44] Meng [35] Qi [66] Mtibaa [67] especially if the devices are very mobile. Arkian et al. [40] also present a solution using clusters and an algorithm for selecting the cluster head, i.e. the vehicle which will be responsible for maintaining the vehicular cloud resources, using fuzzy logic and a reinforcement learning technique. In order to select the best vehicle, they need to know which vehicles possess which resources, hence performing resource discovery. Another aspect of resource discovery is the fact that at the edge, the resources might not come all the time from the same part of the network. Therefore, Liu et al. [43] propose an algorithm enabling a switch between looking for resources in the neighbouring devices and looking for resources in the cloud. They qualify their strategy as adaptive as it takes into account the current resource situation in the network.
Finally, Zamani et al. [62] use a framework called Comet-Cloud which performs resource discovery for video analysis and compare the benefit gained to a solution in the cloud.

Resource allocation
Resource allocation can be tackled from two different perspectives: where to allocate (both initially, but also where and when to perform a migration if needed), and when and how much to allocate. In what follows we group papers that have a single perspective, and then move on to papers where both perspectives are present.

Placement/migration
Most of the surveyed works emphasize the first perspective, i.e. where should the task be executed and the resource allocated for the best possible execution. The definition of best execution varies depending on the considered system and the focus of the research.
Load distribution to achieve lower latency has attracted attention in some of the surveyed works. Among them, Fricker et al. [58] propose an offloading strategy between edge datacenters under high loads which show the benefit of having a larger datacenter as back-up for a small one. Oueis [52] tackles the issue of load distribution and resource allocation in small cell clusters. She formulates a joint computational and communication resource allocation and optimization problem in a multi-user case with focus on latency and power efficiency. Latency is also the focus of study for Borylo et al. [54] who investigate dynamic resource provisioning. They present a policy in which the edge can use the cloud in compliance with the latency requirements of the edge but enables a better energy efficiency by using resources in datacenters powered by green energy.
Valancius et al. [34] propose a content placement strategy where the content is movies. The focus is first on finding the optimal number of replicas of the data to be stored, and then on placing the replicas on available gateways. Similarly, Qi et al. [66] present an allocation scheme where the resource (either coming from a cloudlet or a cloud) is chosen for each task. The aim is to pick the resource from the most suitable location when the user is moving. Mascitti et al. [42] present an algorithm where an end device can choose to either use a resource from a node it is directly in contact with or to compose different resources from different nodes in order to complete its task. Coming from the same research group, Borgia et al. [63] present a framework where the decision is taken to obtain a service from a local group of end devices or from the cloud based on estimation of the time required to obtain the service.
Confais et al. [48] propose a storage mechanism where the objects to be stored are primarily placed locally at their creation but can then be copied to another location if another edge site is requiring access to it. In vehicular networks where resources are mobile, placement has to take account of changes in location. Meng et al. [35] present a resource allocation scheme which can manage resources from a vehicular cloud, the edge or the cloud. Their focus is on minimizing delays for the users.
When it comes to virtual entities such as services, applications, task and VMs, the focus should be on how they can be moved during execution if the new location is better.
For example, Skarlat et al. [39] present a service placement problem for IoT services that they study as an integer linear programming problem. Similarly, Tärneberg et al. [56] study application-driven placement and present a system model for mobile cloud network with a placement algorithm that guarantees application performance, minimizes cost and tackles resource asymmetry problems. In the application domain of healthcare, Gu et al. [55] include VM placement in their optimization problem for analyzing data. Plachy et al. [57] propose a cooperative and dynamic VM placement algorithm associated with another cooperative algorithm for selecting a suitable communication path. They use VM migration to solve user mobility problems.
Other works focus specifically on the problem of migrating resources, which can be seen as a subproblem of the resource placement problem. For example, Gomes et al. [11] present a content-relocation algorithm for migrating the content of caches present in edge devices. This needs a prediction of the users' mobility.
With respect to virtual computational resources, Fan et al. [51] and Rodrigues et al. [59] focus on virtual machine migration, but with different optimization objectives (increase the use of green energy and minimize delays, respectively). Yousaf et al. [37] propose a VM migration (and VM management) system that takes into account the relationship between resource units when taking migration decisions. They present this work in the context of 5G but it should be applicable to all physical machines hosting VMs.
Penner et al. [45] introduce the term transient cloud and concentrate on task assignment to a given node. They present a collaborative computing platform that allows nearby devices to form an ad-hoc network and provide the ability to balance the assignments among themselves.

Scheduling
While there is a huge body of research available on when and how much resources to allocate within networking and cloud-specific areas, our goal was here to identify examples where scheduling decisions are at the edge level in the sense of our terminology in section 2.
Bittencourt et al. [61] study the impact of three different scheduling strategies (Concurrent, First Come-First Served, and Delay-priority) on application QoS when used in a fog. Wang et al. [38] propose a joint cost-effective resource allocation between the mobile cloud computing infrastructures and the cloud radio access network infrastructure.

Multiple perspectives
A work that tackles both perspectives is by Liu et al. [47]. It presents a multi-resource allocation system which first decides whether the request should be served or rejected (admission control), then where to run it (edge or cloud level), and finally how much bandwidth and computing resources should be allocated for this task. Their aim is to maximize system benefit while guaranteeing QoS for the users. Zamani et al. [62] have studied placement and scheduling in the context of video analysis. Their scheduling is based on identified chunks of video, applying two alternatives: minimising computation time or minimising computation costs. Their placement is done after resource discovery using CometCloud.
How much resources are to be allocated to a given IoT data generator is a topic of discussion by Arkian et al. [49], in which they first mathematically model deployment and communication costs on various fog nodes and then decide on placement of VMs to achieve lowest costs.

Resource sharing
Resources on end devices are heterogeneous and most of the time scarce, and edge devices also have limited resources compared to (almost infinite) resources in the cloud. Sharing resources between devices or between end and edge devices aims at tackling three different issues: not having the needed resource in the device where the task is initiated, not having enough of it, or using other devices' resources in order to get a faster completion of the task.
Sharing resources is typically realised by pooling resources in local vicinity of client nodes. This can extended to the edge domain (clustering edge servers) or remain at end devices. The latter is investigated by Skarlat et al in so called fog colonies [39], by Arkian et al. within vehicular clusters [40], or by Bianzino et al. [65] for uploading data streams in presence of mobility.
Data is a resource that many stakeholders may be interested in sharing, for example when searching for a person with the help of multiple cameras from different owners. In their work, Zhang et al. [60] present a framework for this type of sharing, called Firework.
Some researchers, such as Liu et al. [44], try to exploit opportunistic contacts between the devices, creating a resource sharing mechanism that enables faster task completion. They propose different models for calculating task latencies. Similarly, Mtibaa et al. [67] define three mobile device clusters (one hop, two hop and opportunistic) which can share their resources. Their aim is to share resources in order to get the longest possible network lifetime, i.e. saving as much energy as possible through offloading so that the devices can stay on longer. They identify two important topological factors: number of hops and disconnection rate due to mobility.
However, creating and maintaining a group of devices which can share their resources has a cost, for example shown by Athwani et al. [64] who concluded that maintaining the cluster consumes extra energy, especially if the devices are very mobile. This is why Yu et al. [70] propose in their model to do the resource sharing in two phases, where the first phase is deciding whether the device gains more by working alone or joining what they call a coalition, and the second one is deciding if the device will rent or lease resources. Their aim is to improve resource utilization and user QoS.
Resource sharing can perhaps speed up the execution of a task, but Nishio et al. [41] argue that this is not bringing any advantage for the user if we do not consider how different tasks are related in order to provide a service to the user. They provide the example of a GPS service: if the best route calculation is very fast but the downloading of the map is not, the service to the user won't get faster as both are needed.
Finally, even if resource sharing can bring benefits for a group of end devices, it is not obvious that users will agree to share their resources, especially if they are always on the providing side. Therefore there is a need to develop incentives for resource sharing such as works by Tang et al. and Bianzino et al. [53], [65]. The former proposes a double bidding mechanism for demander and supplier of resources where the focus is on how to encourage mobiles with resources to share them. The latter has a mechanism whereby lending energy to vicinity nodes is rewarded and can be used in future scenarios when the lending node itself needs energy.

Resource optimization
A fifth objective pursued in the surveyed works is to optimize the resource use at the edge. This is usually a joint objective together with one of the previously described objectives. Which aspect should be optimized and the associated constraints varies among the surveyed works but the three main ones are quality of service, energy, and operational cost.
In most of the surveyed works, optimizing quality of service is linked to the notion of latency. This is in line with one of the aims of edge computing being faster services than cloud-provided services. Liu et al [44] propose an optimization problem where a resource seeker can utilize resources from other mobile devices in order to reduce the latency of its tasks, and Meng et al. [35] formulate an optimal problem as a Semi-Markov decision process with a reward system based on latency. Rodrigues et al. [59] try to minimize delay through VM migration and by focusing on both transmission delay and processing delay.
In their work, Liu et al. [47] combine low blocking probability with short service time to define QoS. The optimal solution should give highest QoS but also maximize the system benefit, which they calculate by taking into account the income received by serving a request or the penalty for rejecting it, and the cost of using the resources.
However, quality of service encompasses more than just latency. Zamani et al. [62] want to maximize the overall value obtained from the processed data at the same time as maximizing the throughput of the infrastructure. Finally, to improve quality of service, you first need to be able to provide the service. Therefore, Mtibaa et al. [67] are optimizing the lifetime of the devices so that they can be used as long as possible by taking into consideration the fact that they are battery-driven.
Energy is an important aspect of computing at the edge, and some works focus on optimizing its use. For example, Liu et al. [43] present an algorithm which changes the location of resources (cloud or end device) to be used depending on which alternative provides the most reliable resources at a reasonable energy cost. Other works focus on optimizing the use of green energy. Fan et al. [51] want to optimize grid consumption in order to use more green energy.
Sometimes, energy is optimized jointly with latency. In their framework, Nishio et al. [41] formulate an optimization problem where they want to maximize the sharing gains while keeping the latency and the energy consumption low. Similarly, Oueis [52] tackle the issue of load distribution and resource allocation with the aim of being latency-and energy-efficient.
Another aspect is the operational cost of computing at the edge. For example, if some service provider wants to use edge resources but is not the owner of the edge device, they will have to pay to use the resources, in a similar way to what is done today with cloud resources. This is tackled by Wang et al. [38] who propose optimization with regards to expenditure cost for using resources in a mobile cloud computing solution. Gu et al. [55] and Arkian et al. [49] consider both deployment cost as well as communication cost between the different actors.
Resource cost optimization is also present in the work by Qi et al. [66], in which they present a resource orchestration algorithm to optimize three different (and sometimes contradictory) metrics: energy consumption, resource cost and resource availability. Tärneberg et al. [56] has as optimization objective to minimise operational cost of datacenter resource and minimise network usage.
Optimization in the surveyed works can also consider other aspects. For example, Yousaf et al. [37] want to maximize resource utilization, and Skarlat et al. [39] have as their objective to maximize the use of the fog landscape (i.e. not using the cloud if possible) while satisfying QoS requirements. Valancius et al. [34] want to decide on the number of copies of a movie that have to be stored in order to minimize the number of requests that have to be served by the cloud infrastructure. Finally, some works that use clustering present ways to select the best cluster head, even if not formalized as an optimization problem. Athwani et al. [64], for example, take into account inter-node distance in order to minimize delay and energy consumption when doing resource discovery in the cluster. Similarly, Arkian et al. [40], who seek a stable cluster head, use a fit factor combining average speed, neighborhood degree, and communication link quality.

Summary of objectives in resource management
By far, the most active area of research in the edge resource management is resource allocation, as visible in Table 2. This is followed by optimisation as a goal, where we see a great majority of papers present. Among the objectives depicted in Figure 2, resource estimation and resource discovery are least studied. Resource sharing, to the extent it is used, is well-represented among the second and third type of architectures in Figure 1, i.e. coordinator device, and device clouds, but not in the first type of architecture (edge server).
Somewhat surprisingly, while scheduling is a major topic in cloud systems, the edge-specific literature does not consider it as a main problem, as evident from fewer works addressing scheduling compared to placement and migration. Where autoscaling is mentioned in an edge context, authors typically deal with offloading to the cloud which was not the focus of our work. There are several excellent surveys already covering these. The work by Wang et al. [71], addressing autoscaling and the edge is among few exceptions, so we did not create a special category for this type of work.

Resource location
Computing at the edge differentiates itself from regular cloud computing with the fact that resources used can belong to different levels. It is indeed not uncommon to use resources at the edge level primarily, but also from the cloud level if required. Moreover, end devices, and sometimes edge devices do not have to be static as in a datacenter. Note that here we make a distinction between mobility on the demand side and mobility on the supply side. Even though the demand side clients are invariably mobile, the infrastructure that supplies the adequate resources has been invariably static in the past.

Static resources
Most of the surveyed articles consider resources which are static. This can be because the architecture/application considered does not have mobile resources or for simplification reasons. The latter is found in works where the architecture presented has resources that are theoretically mobile but where this part is ignored in the solution or evaluation presented, e.g. in [43] or [41].
Among the surveyed works, Aazam et al. [33] consider resources located at one location, a fog node, but the majority of the works consider several locations, though in different ways. Resources spread among several physical locations can belong to the same level or not (located at the edge level and the cloud level for example).
The former is considered at the device level, by Tang et al. [53], and Nishio et al. [41] who consider resources present on different end devices.
Finally, some works distinguish different levels between the same architecture level. For example, Wang et al. [38] consider transmission in the access network, and computation in a mobile cloud computing architecture. Tärneberg et al. [56] consider that dataceneters at the edge can have a different distance to the device, and different sizes.

Mobile resources
Having mobile edge devices, and thus mobile resources obviously creates lots of challenges such as how to handle the unreliable connectivity of those resources, how to provide seamless handovers, etc. Thus, having mobile resources introduce another level of complexity in resource management algorithms.
Similar to static resources, mobile resources are mostly considered within a single level in the surveyed articles. Different mobility models are used, for example Mascitti et al. [42] and Liu et al. [44] use the Random Way Point model, Penner et al. [45] model departure and arrival times using statistical models, Arkian et al. [40] consider the speed of the vehicles, and Athwani et al. [64] consider that 10% of the nodes are moving after a request. Finally, Mtibaa et al. [67] consider a mobility model with low disconnection rate based on a dataset (Infocom06) where the mobility of the devices is predictable.

Combination of mobile and static resources
Some works present a combination of mobile and static resources. In the edge level, Aazam et al. [50] consider different type of devices (static or mobile). However, the devices are actually not mobile in their simulations.
Borgia et al. [63] consider the local cloud as mobile and the global cloud as static. They use the Random way point model for mobility. Similarly, Qi et al. [66] have mobile end devices and static infrastructure servers and describe their own mobility model. Meng et al. [35] use a mobile vehicular cloud together with a static local cloud at the edge level and a static remote cloud. The mobility of the vehicles is modelled as a Poisson process.
Similarly, Bianzino et al. [65] use estimated (upload) bandwidth need for the nearest time window per mobile device. This allows handling churn by revising the total resource needs at the edge device. Their algorithm predicts joint resource needs in the cluster during that window and decides how to use interface sharing to achieve a lower overall energy consumption for all uploads.
Finally, some works are considering stationary resources that are allocated for mobile users. For example, Plachy et al. [57] consider that computational resources needed by a user are allocated in a stationary base station in a VM, which can be transferred to another base station if the user is moving. Similar solutions are presented by Tärneberg et al. [56], Gomes et al. [11], Oueis [52], and Fan et al. [51]. Table 3 presents the surveyed articles according to their architecture categories and considered resource locations. A bit more than half of the surveyed works are considering resources that are located in only one level of the architecture and a majority of them consider only static resources.

Summary of edge resource mobility
Despite demand side node mobility that may be present in all architectures, the supply side node mobility, i.e. the notion of mobile resource, is clearly what the edge brings. We see more mobile resources present in the second and third types of architecture (coordinator device and device cloud).

Resource use
The final aspect of resource management considered in this survey is the purpose for which the resource will be used.

Functional properties
Indeed, in most of the surveyed articles, resources are used in order to get the wanted service, i.e. for satisfying functional properties. For example, resources can be used for IoT services [39], video stream services [27], GPS services [41] or crowd sensing applications [49].

Non-functional properties
However, functionalities enabled when using the edge computing paradigm or properties that are a consequence of how the edge is organized also require some additional work to be performed. This additional work is not directly related to the service to obtain, i.e. it is a non-functional property.
For example, Athwani et al. [64] describe a solution which is mostly about functional properties but they are also taking into account that maintaining the cluster demands extra energy. Similarly, Fan et al. [51] consider the cost of VM migration.
Another aspect, the fact that we can't have devices sharing resources without incentives is tackled by Tang et al. [53] and Bianzino et al. [65].
However, it is interesting to note that none of the surveyed papers analyzed the overhead of some properties achieved by using edge computing and which are advocated as good reasons to choose this paradigm, e.g. privacy [2] or context adaptation [72].

CONCLUSION
The past decade has created tremendous expectations on IoT changing the landscape of data-driven services with benefits for multiple societal sectors. Many researchers have contributed to development of technologies and addressed challenges that come with resource scarcity in the end devices. Other researchers with a background in cloud computing have looked at how to carry the data generated by the massive IoT deployments and how to efficiently use the cloud resources. The area of edge computing brings these two ends of the same service together and creates a means to discuss resource adequacy from an end-to-end perspective. In this paper we have tried to provide an overview, not from a cloud perspective, or an IoT device perspective, but with a focus on edge resource management.
Scalability, timeliness, and security were among the underlying reasons to explore the edge paradigm. Although scalability and timeliness are dependent on efficient management of resources, earlier surveys on edge computing have not been focusing on resources. Much less so, those that have surveyed security have not studied how additional mechanisms for additional non-functional properties impact resource use.
End-to-end timeliness requires quantification of latencies from an end device towards the cloud (or somewhere at the edge) all the way back to the same device (or to another device). This means traversing the edge networking services, including what we referred to as resource management services in this paper. Since estimation, discovery, sharing, allocation (including migration) are complex algorithms in such networks, these must also be evaluated in terms of their own resource footprint, and thereby their impact on timeliness and QoS. Our survey indicates that this problem is not extensively studied.
Many works have tried to understand the challenges using a given application focus. So naturally their attention has been drawn to understanding the networkingcomputing joint problem to achieve the functional goals. Hence their characterisation of the resource management has been geared towards functional requirements. We find that few works have characterised resource usage due to non-functional properties, including energy efficiency, storage efficiency, and others. We believe that this survey provides a good overview in order to fill the gaps towards a resource-efficient end-to-end edge-based service.

ACKNOWLEDGMENTS
This work was supported by the Swedish national graduate school in computer science (CUGS).