Unmanned aerial vehicle‐aided edge networks with ultra‐reliable low‐ latency communications: A digital twin approach

A digital twin (DT) framework for Internet ‐ of ‐ thing (IoT) networks is proposed where unmanned aerial vehicles (UAVs) acting as flying mobile edge computing (MEC) servers support the task offloading on the fly. The considered DT model is very well suitable for industrial automation with the strict constraints of mission ‐ critical services' ultra ‐ reliable low ‐ latency communication (URLLC) links. To support low ‐ latency IoT devices, we formulate the end ‐ to ‐ end (e2e) latency minimisation problem of digital twin ‐ aided off-loading UAV ‐ URLLC. Specifically, the minimised latency is obtained by jointly optimising both communication and computation parameters, namely power, offloading factors, and the processing rate of IoT devices and MEC ‐ UAV servers. Due to the highly non ‐ convex optimisation problem, we first consider the K ‐ means clustering algorithm to optimally deploy the on ‐ demand UAVs. Then, an alternative optimisation approach combined with appropriate inner approximations is effectively exploited to tackle this challenge. We demonstrate the effectiveness of the proposed DT framework through representative numerical results.


| INTRODUCTION 1.| Literature review
Digital Twin (DT) is an emerging technology that is able to create virtual twins of physical objects in order to facilitate the processing of control and to manage cyber-physical systems.DT can be exploited in networking and communications for many aspects, such as system modelling, physical data processing, cloud computing, and edge computing [2].Therefore, studies of DT are attracting much attention from active researchers [3][4][5][6].More specifically, in [3], a DT-assisted task offloading in mobile edge computing (MEC) was investigated to address the problem of minimising power and time overhead.Another work in reducing offloading latency for DT edge network was introduced in [4].In [5], DT was proposed for intelligent authorisation in the beyond 5G smart grid applications.DT was exploited in [6,7] to empower edge networks for the industrial Internet of things environment.These representative studies demonstrate the huge potential of DT in various domains, especially in networked systems.
In recent years, unmanned aerial vehicles (UAVs) have been under the spotlight due to their flexible configuration and mobile characteristics [8,9].Numerous research studies have been conducted to enhance the control performance of UAVs.In [10], authors carried out the ground test of UAV, evaluating the performance of its entire flight control system through rotor speed, roll attitude, etc.The flight test [11] and collision avoidance [12] have also been studied to give a stronger control over this smart vehicle.UAVs achieve even better performance in many research areas by combining with other advanced technologies.To integrate with the intelligent reflecting surface (IRS) [13], a well-performed UAV-assisted IRS symbiotic radio system has been formed.The system performs better as data information is transferred via UAV by optimising the UAV trajectory and the IRS phase shifts.Applying a deep reinforcement learning algorithm [14], the decision making of UAVs can be autonomous rather than pre-planned.UAV's low consumption of energy also attracts public attention.Through optimisation algorithms, resource allocation can be optimised, and the total energy consumption in a multi-UAV network framework can be minimised [15,16].Based on their outstanding advantages, UAVs are now being developed and used in military and civil applications [17].These applications are in many fields, such as environment monitoring, traffic control, public safety, damaged buildings detection, and industrial automation [18][19][20].Particularly, with the help of UAV, Yang X et al. [21] develop a method for high-precision geolocation of distant targets, which is more effective than the conventional one-shot localisation way.Search and rescue operations with UAVs participation increase the speed of rescue and thus improve the survival rate of people [22,23].In Ref. [24], UAVs are used as flying base stations to ensure the connectivity of communication networks in unexpected disasters.These aerial vehicles also play a role in smart cities [19].
With the rapid development of the 5G network, ultrareliable and low-latency communication (URLLC) emerges as a promising paradigm to ensure a certain quality of service (QoS).With strict requirements of extremely low latency (from 1 ms to few milliseconds) and ultra-high reliability (over 99.999%) [25], URLLC plays an indispensable role in remote healthcare, autonomous driving, immersive virtual reality, cloud robotics, deterministic communication, and many other areas [26].This novel communication service uses short packet transport, which allows optimising the transmission of control information [27].
Evolved from cloud computing, MEC has been widely considered as a key application in 5G communication.This promising technology extends the capabilities of cloud computing at the network edge [28] and performs excellently in smart manufacturing, industrial Internet of things (IoT) as well as many other areas [29][30][31].In recent years, many efforts have been put into MEC.In Ref. [32], the author demonstrates a wellestablished MEC architecture and integrates an application deployment use case, establishing a proof of concept, which is very similar to the actual deployment of the MEC system in a 5G environment.Combining with the optimising method, MEC is an appropriate solution to improve the quality of service.In Ref. [33], the author proposes a Reinforcement Learning (RL)-based optimisation framework to minimise the cost of delay and energy consumptions for user equipment in a timevariant dynamic MEC system.The considered MEC system outperforms other baseline solutions according to the demonstration, whereas Wu J et al. [34] adopt an offloading strategy in MEC that considers delay and energy consumptions of cost optimisation.Two schemes named optimised OMA and hybrid NOMA are proposed in [35] for solving the problem of joint power and time allocation for MEC offloading.Through these extensive studies of MEC, this evolving technology has been used in various use cases, for example, an audience metre [36].In this particular use case, MEC modules are used to improve the algorithm's performance, detecting the estimated number of participants in an event over the entire time period.
MEC and URLLC techniques are closely related to each other.Under the sufficiently powerful computing of MEC, applications can be processed in real time.By reducing the transmission processing time and reception processing time, MEC can reduce latency in 5G systems significantly.Since finite blocklength (FBL) is adopted to satisfy the latency constraints of URLLC, Yang et al. [37] propose a MEC network where one MEC server is used to minimise the error probability between users under FBL and energy consumption constraints.This FBL scheme is also considered in a MEC-enabled vehicular network to support URLLC [38].The placement of MEC servers that promises URLLC requirements has also been studied.An algorithm called LowMEP has been proposed to find a minimum number of MEC servers, satisfying the quality of service [39].The combination of these two promising techniques is frequently used in various fields.For industrial application, Jia et al. [40] propose a 5G MEC gateway system capable of supporting URLLC to enable communication in factories.For autonomous vehicles application, vehicle's latency and energy cost functions have been established to investigate URLLC resource scheduling for edge computing [41].
Additionally, UAV-assisted communications can be effectively combined with MEC to enable time-sensitive and computation-intensive services for a wide range of Internet of Things (IoT) applications [42][43][44][45].UAV-based edge networks not only be able to minimise the energy consumption of IoT devices with optimal offloading decisions [46,47] but also reduce the latency by providing edge caching solutions [48].More importantly, ultra-reliable and low-latency communications (URLLC) recently emerged as a promising technology for mission-critical applications [49].Combining UAV-enabled MEC with URLLC opens many opportunities as well as challenges for the next generation of IoT applications [50].

| Motivations and main contributions
Recently, combined merging technologies, including MEC, DT, UAV, and URLLC, are attracting many active research groups [3,4,42].In particular, the DT-assisted task offloading based on edge collaboration has been investigated in [3] with a DRL-based approach.The edge selection and offloading optimisation have been addressed in this paper; however, the communication resource optimisation has not taken into consideration.Similarly, in [4,42], the joint computation and communication resources have not been fully addressed to obtain the optimal latency of task offloading.More importantly, the combination of DT, MEC with URLLC for industrial scenarios is a promising research direction, which still has many open issues that can be further explored and contributed to this research area.
Moving beyond above background, this paper addresses the problem of combining these important technologies, namely DT, MEC, and URLLC, in UAV-assisted IoT systems.Both communication and computation factors, including transmit power, the processing rate of IoT devices or user equipment (UE), edge servers, and offloading policies, are carefully taken into consideration to reduce the end-to-end (e2e) latency.Main contributions of this paper are summarised as follows: The rest of the article is structured as follows: Section 2 presents the system model and problem formulation for attaining minimum e2e latency.Section 3 resolves the problem formed in the previous section using an algorithmic solution.Numerical results are discussed in Section 4. Finally, Section 5 concludes the article.

| SYSTEM MODEL AND PROBLEM FORMULATION
This section leads towards the problem to be solved for minimising the e2e latency by expressing the basic network model, transmission model, DT-empowered offloading, associated energy and power consumption model, and the UAV deployment.

| DT-empowering URLLC-based edge networks model
Figure 1 presents a DT-enabled UAV-based edge network architecture with URLLC.The physical layer consists of IoT devices (IoT), also known as UEs, and UAVs.These physical devices connect via URLLC links to ensure stringent reliability and low-latency communications in mission-critical applications.
Let M ¼ f1; 2; …; Mg be the set of M IoT devices and K ¼ f1; 2; …; Kg be the set of K UAVs.There are K UAV-IoT groups, in which the kth UAV serves M k IoT devices in each group.Each UAV can act as an access point (AP) with the capability to perform as an edge server (ES).We assume that the UAV deployment and network planning are performed in advance.

| Channel model
The air-to-ground (ATG) channels between UAVs and UEs are also dominant by light-of-sight (LoS) propagation but these are more complex due to the effects of propagation attenuation by blockage geometry and shadowing [51].As such, the path loss of the link between the kth UAV and the (m, k)-th UE can be written as where η LoS and η NLoS are the average additional losses for LoS and non LoS (NLoS), respectively.The path loss in respect to the distance (PL mk ) is given by where f c is carrier frequency (Hz), c is the speed of light (m/s), , d mk is the Euclidean distance between the mth UE and the kth UAV, and Z k is the antenna height of the kth UAV.The probability of LoS and NLoS can be shown as An exemplary illustration of the DT-enabled UAV-based edge networks with ultra-reliable low-latency communications (URLLC) [1] where the constants a and b depend on the specific arrangement of the environment.Each UAV is equipped with L antennas to serve M k singleantenna IoT devices.Let h mk ∈ C L�1 be the channel vector between the kth UAV and the mth IoT, which can be modelled as h mk ¼ ffi ffi ffi ffi ffi ffi ffi g mk p h mk .Here, g mk denotes the large-scale channel coefficient defined in (1), and h mk is the small-scale fading following the distribution of CN ð0; IÞ.Let H k ∈ C L�M k be the channel matrix from M k devices to the kth UAV with . Under the shared wireless medium, the L � 1 received signal vector at the kth UAV is given by y k ¼ P M k m¼1 h mk ffi ffi ffi ffi ffi ffi ffi p mk p s mk þ n k , where p mk is the payload power of the (m, k)-th device, s mk is the zero mean and unit variance Gaussian information message from the (m, k)-th IoT, and n k ∼ CN 0; N 0 I L ð Þ is the additive white Gaussian noise (AWGN) during the data transmission with N 0 being the noise power.
In this paper, we consider the uplink transmission from IoTs to UAVs to perform task offloading.We apply the maximum-ratio combining (MRC) at the UAV to improve the performance gain.Moreover, to guarantee fairness among all IoT devices and further improve wireless transmission performance, we additionally adopt the matched filter and successive interference cancellation (MF-SIC) technique at the UAVs.In particular, by using MF-SIC, we assume that the decoding order follows IoTs' index by arranging the channel vector as kh 1k k 2 ≥ kh 2k k 2 … ≥ kh M k k k 2 ; 8k.Consequently, the signal-to-interference-plus-noise (SINR) at the kth UAV of the signal from the (m, k)-th IoT device can be expressed as where I mk ðpÞ ¼ P M n>m p nk kh nk k 2 is the interference power caused by IoT devices n > m and p ¼ p m f g 8m

| URLLC-based uplink transmission rate
The approximation of achievable transmission rate (bit/s) in URLLC finite blocklength is [52,53]: ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi where ω k = M k /N, 8k, N is the blocklength, which can be written as N = δB with B as the bandwidth and δ as the transmission time interval; ϵ is the decoding error probability, À � dt, and V is the channel dispersion given by When the blocklength N approaches to infinity, the data rate Þ, which is the classic Shannon's equation.

| Digital twin empowered task offloading model
A particular task from the (m, k)-th IoT device is represented by a tuple J mk = {D mk , C mk , T mk }, where D mk is data size (bits), C mk is required computation resource (cycles), and T mk (s) is the minimum required latency for task J mk .Let α ¼ α mk f g 8m;k be the amount of the task processed locally, which satisfies 0 ≤ α m ≤ 1.

| Local processing
For the (m, k)-th IoT device, its DT (DT mk ) can be expressed as where f loc mk is the estimated processing rate of the physical IoT device, and f loc mk is the deviation between the real device and its DT.
The DT layer has the estimated processing rate f loc mk to replicate the behaviours of IoT devices and trigger decisions on optimising physical devices configuration.
The (m, k)-th IoT executes α mk portion of task J mk with the estimated processing rate f loc mk , and the estimated time required to execute the task locally is given by Assuming that the deviation between the physical IoT ðMÞ and M in DT can be acquired in advance, the computing latency gap between real value and DT estimation is computed as The actual time for local computing is expressed as

| Edge processing
Given the estimated processing rate of the kth ES for executing the offloaded task from the (m, k)-th IoT device is f es mk , the estimated latency of the kth ES to execute task J m is given by Then, the latency gap ΔT es mk between the real value and DT estimation can be expressed as As a result, the actual latency for executing at edge DT can be expressed as The total DT latency in the system can be expressed as follows: The latency comprises three main components, namely local processing latency T loc mk � � , uplink transmission latency , and edge processing latency T es mk À � .Since the response messages from UAVs to IoTs are typically small (e.g.control packets), the downlink transmission latency is negligible [54,55].

| Energy and power consumption model
Total energy consumption of the (m, k)-th IoT includes energy for transmission and computation: where θ m /2 represents the average switched capacitance and the average activity factor of the mth IoT [56].
The power consumption of the kth UAV for processing the uploaded tasks is modelled as follows [48]: where θ k represents the average switched capacitance and the average activity factor of the kth UAV.

| UAV deployment
In this section, we present the clustering algorithm for UAV deployment by considering an efficient QoS-constrained K-means clustering approach [57][58][59].In particular, the constrained clustering method considers whether the mth UE can be grouped in the kth cluster based on two types of pairwise constraints, namely must-link constraints and cannot-link constraints [58], which represent the satisfied QoS constraints and the violated QoS constraints, respectively.The constrained K-means clustering algorithm can be briefly described as follows.At the initial stage, the locations of UAVs are randomly set within the deployment area as the centroid location.First, based on the Euclidean distance between the UEs and the UAVs, the mth UE is assigned to an appropriate cluster with the smallest distance.Then, if QoS constraints are not satisfied for any UEs, the altitude of the corresponding UAV must be adjusted.Finally, the centroid location for each cluster is updated.Such procedure repeats until the cluster members are stable or the number of iterations exceeds a predefined threshold.
As an illustrative case, we set the number of UEs randomly located in a critical area at M = 6 and the number of UAVs at K = 2 with the path loss threshold corresponding to the QoS requirement γ QoS = 110 dB. Figure 2 shows the clustering result after implementing the QoS-constrained K-means clustering algorithm.

| Problem formulation
Here, the worst case of the total DT latency is minimised by optimising offloading policies, transmit power, and estimated processing rates of IoT and ESs.By defining the following notations D ≜ α mk ; 8m; kj0 ≤ α mk ≤ 1; 8m; k f g, P ≜ p mk ; f 8m; kj0 ≤ p mk ≤ P max mk ; 8mg, and F ≜ f loc mk ; f es mk ; 8m; kj0 n ≤ f loc mk ≤ F loc max ; 8m; 0 ≤ f es mk ≤ F es max ; 8kg as the set constraints of offloading decisions, uplink transmission power, processing rates, respectively, the problem is formulated as follows: where constraint (17b) presents maximum latency constraint for every incoming task.Constraints (17c) and (17d) are the minimum transmission rate requirements for uplink transmission and the maximum energy consumption requirement of IoT, respectively.Finally, constraints (17e) and (17f) refer to maximum available computation resource and power budget of the UAVs.
As we can observe from (17), the objective function is nonconcave and non-smooth, while other constraints (17c), (17d), (17e), and (17f) are also highly complex non-convex constraints.This results in solving the problem directly, which is computationally challenging.Therefore, to solve the problem (17) ) is the optimal solution to problem (18), then (α ⋆ , p ⋆ , f ⋆ , t ⋆ ) is also the optimal solution to problem (17) and vice versa.
Proof To prove Lemma 1, we show that the constraints (18d)-(18f) must hold with equality at optimum.Firstly, assuming that the equality of (18e) does not hold at the optimum for some m, that is, existing . There exists a positive constant Δt lc > 0, which is defined as . As a result, t lc − Δt loc is also feasible to problem (18), but results in a strictly lower latency.This contradicts the original assumption that the set (α ⋆ , p ⋆ , f ⋆ , t) is the optimal solution to problem (18).Other constraints (18e), (17f) are followed similarly.□ Due to the complexity of the non-convex problem (18), we decompose (18) into two sub-problems and solve the problem in the fashion of alternating optimisation (AO) approach and inner approximation (AO-IA) framework [60,61].The following subsections fully present the development of our proposed solution.

| Transmit power and computation resource optimisation
In this subsection, we solve (18) for given (α (i) ) to obtain the next optimal values of (p (i+1) , f As we can observe from the sub-problem ( 19), the constraints (17c), (17d), and (18e) are non-convex.We are now in the position to approximate these constraints.
Convexify of (17c): To address constraint (17c), we first rewrite that γ mk p ð Þ ¼ p mk q mk p ð Þ , where q mk p ð Þ is defined as Following the Appendix, we have under the trusted regions.
where G ðiÞ mk p ð Þ, and W ðiÞ mk p ð Þ are defined as in the Appendix, ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi As a results, we innerly approximate constraint (17e) as Convexify of (17d): By variables r ≜ r mk f g 8m;k that satisfy r mk ≥ 1/R mk , 8m, k.We can equivalently express (17d) as follows The constraint (25b) is now convex, while (25a) is still nonconvex so we apply the following inequality which is now a convex constraint.Convexify of (18e): By using r defined in Equation (25b), Equation (18e) can be convexified as This is a convex program so that we can solve it efficiently with the CVX package.For complexity analysis, the convex problem (29) comprises 11 KM k + 2K linear or quadratic constraints and 5KM k + K scalar decision variables, which leads to the periteration computational complexity of O ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi
This is obviously a convex program and can be solved effectively with standard solvers, such as CVX [63].The per-iteration of solving this convex program is O ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi

| Proposed algorithm
Let us denote S ðiÞ 1 ≜ p ðiÞ ; f ðiÞ ; r ðiÞ À � and S ðiÞ 2 ≜ α ðiÞ À � , and at the ith iteration, respectively.We now proceed by proposing Algorithm 1 to solve the problem (18).LI ET AL. -7 This section expresses the performance metrics and impacts studied through numerical simulations.

| Simulations setup
To understand the performance and resolution of the problem (17) for minimising the e2e latency, we relied on numerical simulations.The values of parameters used for attaining these results are summarised in Table 1.

| Convergence of the proposed algorithm
Figure 3 clearly demonstrates the convergence of the proposed algorithm in reducing the worse-case e2e latency.In particular, with the model of M k = 4 UEs, K = 2 UAVs, the optimising process converges at six iterations.The figure additionally illustrates the impact of the required computation resource on the e2e latency.Unsurprisingly, when the required computation resource increases, the e2e latency of computational tasks gradually increases.For instance, the observed scenario witnesses considerable rise in the worst-case e2e latency from 0.6 s to approximate 0.87 s when Cm rises to 1100 megacycles.

| Impact of required computation resource
Figure 4 plots the impacts of required computing resources of the tasks (Cm) in the e2e latency of UEs under the proposed algorithm and other benchmark schemes.Particularly, when the tasks are more complicated, which require more computation resources, the e2e latency gradually increases.For instance, in the model of M k = 4, K = 2 as observed in Figure 4, when C m rises from 800 to 1, 200 megacycles, the worst-case latency obtained by using Algorithm 1 increases by approximate 300 ms.In addition, Figure 4 clearly demonstrates that our proposed algorithm is far better than all benchmark schemes.These results prove that joint optimisation of both communication and computation resources significantly improves the performance of MEC-based systems in reducing the e2e latency of UEs.

| Impact of UEs' processing rate
To demonstrate the impact of UEs' processing rate on obtaining the minimised latency and adjusting optimal offloading decisions, Figure 6 presents the numerical results of experiments with a range values of F loc max .Unsurprisingly, when the processing capacity of UEs increases from 1 to 1.3 GHz, the e2e latency of UEs significantly reduces in both models.This is because the UEs are more powerful in processing tasks locally.Additionally, due to the constraints on UEs energy consumption in (17d), the average offloading portions of UEs have to increase to satisfy the energy budget when UEs have higher processing rate.

| Impact of ESs' processing rate
Figure 7 illustrates the impact of the total computation resource of ESs, F es max on the e2e latency of computational tasks coming from UEs.The figure clearly shows that the more powerful the ESs are, the less e2e latency can be achieved both examined scenarios of M k = {4, 6} UEs, K = 2 UAVs. Figure 7 also indicates under the same computation resource budget of ESs, the model that has more UEs (M k = 6) obtains higher e2e latency than that in the smaller size model (M k = 4).
These results definitely demonstrate the effectiveness of the proposed offloading design.

| CONCLUSION
In conclusion, this paper has tackled the problem of minimising e2e latency of DT-assisted UAV-based edge networks.The latency minimisation problem has been successfully resolved through the AO-IA framework that is comprised of two sub-problems, namely optimal resource allocation and optimal tasks offloading policies.The numerical results have clearly demonstrated the effectiveness of the proposed solution.In the future, the model can be extended to include a multi-UAVs scenario with heterogeneous requests and offloading operations.

�
We first formulate the latency minimisation problem of UAV-based edge network with URLLC in the DT regime.The addressed problem fully considers both communication and computation factors in reducing the task offloading latency.� In order to solve the problem, we propose the AO-IA algorithm with two sub-problems, namely transmit power and computation resources optimisation, offloading portions of optimisation.� Finally, intensive simulations have been conducted to demonstrate the effectiveness of the proposed solution.

2
An illustrative system model with M = 6 UEs and K = 2 unmanned aerial vehicles (UAVs) after the implementation of QoSconstrained K-means clustering algorithm

F I G U R E 3 2 F I G U R E 4 2 4. 2 . 3 |
Convergence of the proposed algorithm with different values of required computation resource (C m ) in the scenarios of M k = 4, K = The worst-case latency among different values of required computation resource (C m ≜ C ) in the scenarios of M k = 4, K = Impact of UE transmit power budgetTo investigate the impact of UEs transmit power in reducing the e2e latency, we have conducted experiments among different values of UE power budget in three models of M k = {4, 5, 6} and K = 2 UAVs.Figure5clearly states that when the transmit power budget of UEs increases, the worstcase e2e latency of UEs gradually reduces.For instance, the model of M k = 4 UEs, K = 2 UAVs witnesses a considerable decline in latency by around 100 ms when the maximum of UEs transmit power reaches 23 dBm.

F I G U R E 5 2 F I G U R E 6 2 F
The worst-case latency among different values of user equipment (UE) transmit power budgetP max mk À � in the scenarios of M k = 4, K = 2, M k = 5, K = 2, M k = 6, K =The worst-case latency among the maximum values of UEs' processing rate F loc max À � in the scenarios of M k = 4, K = 2 and M k = 5, K = I G U R E 7 The worst-case latency among different values of total ESs' processing rate F es max À � in the scenarios of M k = 4, K = 2 and M k = 6, K = 2, P max k ¼ 8 W LI ET AL.How to cite this article: Li, Y., et al.: Unmanned aerial vehicle-aided edge networks with ultra-reliable lowlatency communications: a digital twin approach.IET Signal Process.1-12 (2022).https://doi.org/10.1049/sil2.12128APPENDIXWefirst rewrite the SINR of UE (m, k) as γ mk (p) = p mk / q mk (p).By applying the inequality[63, Equation (72)] for x = p mk , y = q mk (p), x ¼ p ðiÞ mk , and y ¼ q mk p ðiÞ upper bounding convex function approximation of W mk (p), we apply the inequality [63, Eq. (75)] forx ¼ 1 − 1= 1 þ γ mk ðpÞ ð Þ 2 and x ¼ 1 − 1= 1 þ γ mk p ðiÞ À