| 1. |
- Jasim, Al-Hussein Hameed, et al.
(författare)
-
Packet Probing Study to Assess Sustainability in Available Bandwidth Measurements : Case of High-Speed Cellular Networks
- 2020
-
Ingår i: Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications. - Seoul, Republic of Korea : Innovative Information Science & Technology Research Group (ISYOU). - 2093-5374 .- 2093-5382. ; 11:2, s. 106-125
-
Tidskriftsartikel (refereegranskat)abstract
- Facts and figures indicate that the latest generations of cellular networks are likely to become the dominant medium of the global data exchange. This may pose a challenge to service providers trying to improve the Quality of Service (QoS) provided that is usually specified in Service Level Agreement (SLA), which, in turn, has to be practically verified. It requires methods and tools to measure the adopted QoS metrics such as bandwidth and Round-Trip Time (RTT). For this research, the InfoVista’s proprietary tool (Blixt™) was used. The research discusses how the probing packet parameters play a vital role in determining the accuracy of the measurements, the level of intrusiveness in a shared-resources network, and its implications for sustainability for this application. The experiments were carried out in a live commercial network and the performance was also compared with other cutting-edge available bandwidth measurement tools in a multi-carrier scenario.
|
|
| 2. |
- Minovski, Dimitar, 1990-, et al.
(författare)
-
Analysis and Estimation of Video QoE in Wireless Cellular Networks using Machine Learning
- 2019
-
Ingår i: 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX). - : IEEE. - 9781538682128
-
Konferensbidrag (refereegranskat)abstract
- The use of video streaming services are increasing in the cellular networks, inferring a need to monitor video quality to meet users' Quality of Experience (QoE). The so-called no-reference (NR) models for estimating video quality metrics mainly rely on packet-header and bitstream information. However, there are situations where the availability of such information is limited due to tighten security and encryption, which necessitates exploration of alternative parameters for conducting video QoE assessment. In this study we collect real-live in-smartphone measurements describing the radio link of the LTE connection while streaming reference videos in uplink. The radio measurements include metrics such as RSSI, RSRP, RSRQ, and CINR. We then use these radio metrics to train a Random Forrest machine learning model against calculated video quality metrics from the reference videos. The aim is to estimate the Mean Opinion Score (MOS), PSNR, Frame delay, Frame skips, and Blurriness. Our result show 94% classification accuracy, and 85% model accuracy (R 2 value) when predicting the MOS using regression. Correspondingly, we achieve 89%, 84%, 85%, and 82% classification accuracy when predicting PSNR, Frame delay, Frame Skips, and Blurriness respectively. Further, we achieve 81%, 77%, 79%, and 75% model accuracy (R 2 value) regarding the same parameters using regression.
|
|
| 3. |
- Minovski, Dimitar, 1990-, et al.
(författare)
-
Anomaly Detection for Discovering Performance Degradation in Cellular IoT Services
- 2021
-
Ingår i: Proceedings of the IEEE 46th Conference on Local Computer Networks (LCN 2021). - : IEEE. ; , s. 99-106
-
Konferensbidrag (refereegranskat)abstract
- Connected and automated vehicles (CAVs) are envisioned to revolutionize the transportation industry, enabling autonomous processes and real-time exchange of information among vehicles and infrastructure. To safely navigate the roadways, CAVs rely on sensor readings and data from the surrounding vehicles. Hence, a fault or anomaly arising from the hardware, software, or the network can lead into devastating consequences regarding safety. This study investigates potential performance degradation caused by anomalies, by analyzing real-life vehicles’ sensory and network-related data. The aim is to utilize unsupervised learning for anomaly detection, with a goal to describe the cause and effect of the detected anomalies from a performance perspective. The results show around 93% F1-score when detecting anomalies imposed by the cellular network and the vehicle’s sensors. Moreover, with approximately 90% F1-score we can detect anomalous predictions from a deployed network-related ML model predicting cellular throughput and describe the root-causes behind the detected anomalies.
|
|
| 4. |
- Minovski, Dimitar, 1990-, et al.
(författare)
-
Defining Quality of Experience for the Internet of Things
- 2020
-
Ingår i: IT Professional Magazine. - USA : IEEE. - 1520-9202 .- 1941-045X. ; 22:5, s. 62-70
-
Tidskriftsartikel (refereegranskat)abstract
- The Internet of Things (IoT) brings a set of unique and complex challenges to the field of Quality of Experience (QoE) evaluation. The state-of-the-art research in QoE mainly targets multimedia services, such as voice, video, and the Web, to determine quality perceived by end-users. Therein, main evaluation metrics involve subjective and objective human factors and network quality factors. Emerging IoT may also include intelligent machines within services, such as health-care, logistics, and manufacturing. The integration of new technologies such as machine-to-machine communications and artificial intelligence within IoT services may lead to service quality degradation caused by machines. In this article, we argue that evaluating QoE in the IoT services should also involve novel metrics for measuring the performance of the machines alongside metrics for end-users' QoE. This article extends the legacy QoE definition in the area of IoT and defines conceptual metrics for evaluating QoE using an industrial IoT case study.
|
|
| 5. |
- Minovski, Dimitar, 1990-, et al.
(författare)
-
Modeling Quality of IoT Experience in Autonomous Vehicles
- 2020
-
Ingår i: IEEE Internet of Things Journal. - Canada : IEEE Computer Society Digital Library. - 2327-4662. ; 7:5, s. 3833-3849
-
Tidskriftsartikel (refereegranskat)abstract
- Today's research on Quality of Experience (QoE) mainly addresses multimedia services. With the introduction of the Internet of Things (IoT), there is a need for new ways of evaluating the QoE. Emerging IoT services, such as autonomous vehicles (AVs), are more complex and involve additional quality requirements, such as those related to machine-to-machine communication that enables self-driving. In fully autonomous cases, it is the intelligent machines operating the vehicles. Thus, it is not clear how intelligent machines will impact end-user QoE, but also how end users can alter and affect a self-driving vehicle. This article argues for a paradigm shift in the QoE area to cover the relationship between humans and intelligent machines. We introduce the term Quality of IoT-experience (QoIoT) within the context of AV, where the quality evaluation, besides end users, considers quantifying the perspectives of intelligent machines with objective metrics. Hence, we propose a novel architecture that considers Quality of Data (QoD), Quality of Network (QoN), and Quality of Context (QoC) to determine the overall QoIoT in the context of AVs. Finally, we present a case study to illustrate the use of QoIoT.
|
|
| 6. |
|
|
| 7. |
- Minovski, Dimitar, 1990-
(författare)
-
Quality of Experience in Industrial Internet of Things
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Quality of Experience (QoE) is a research domain that measures the ”degree of de-light or annoyance of the users of an application or service”. Today’s research on QoE mainly addresses multimedia services, where the user’s subjective perception is the prime factor of determining the QoE. However, with the proliferation of the Internet of Things (IoT) in industrial services, this thesis argues for extending the conventional QoE defi-nition, architecture, and evaluation methods. Emerging IoT services, such as industrial transportation and manufacturing, are more complex, with different quality requirements than multimedia services. For instance, evaluating the machine-to-machine (M2M) com-munication that enables autonomous operations. Some IoT services closely engage with human lifestyle and privacy; hence, delivering and measuring quality is of utmost impor-tance. Consider a self-driving vehicle, making multiple real-time decisions as a result of automated predictive models. Therein, measuring QoE faces two key challenges. The first challenge is about measuring the QoE of an autonomous service. Traditional QoE is typically measured through subjective tests - an approach that doe not apply to intelli-gent machines. The second challenge is about understanding and measuring the impact of intelligent machines and their autonomous decisions. Quality degradation caused by the machines and M2M communication can affect the business, environment, and even raise life-threatening concerns.This thesis argues for a paradigm shift within the area of QoE in the direction of understanding the relationships between humans and intelligent machines, as well as among the machines. Thus, the main outcome of the thesis is the introduction of Quality of IoT experience (QoIoT), which extends the traditional QoE definition in covering IoT services. Within QoIoT, we consider a quality evaluation from the perspective of users, as well as from intelligent machines. The user’s perception is captured by following the traditional QoE models, while intelligent machines are evaluated throughput objective metrics describing their experiences and performance.As a result of the extended QoE definition, this thesis presents a QoIoT architecture consisting of a methodology and measurable parameters in emerging IoT services. The QoIoT architecture models low-level objective metrics from four layers of the IoT service: physical, network, application, and virtual. Then, the architecture argues for automated approaches in determining the quality of network (QoN) and quality of data (QoD) based on machine learning methods. In the next step of the architecture, we map the measured low-level objective metrics into high-level contextual metrics as part of a quality of context (QoC) layer. Finally, we aggregate the QoC metrics by considering the stakeholder’s requirements to form a unified metric measuring the QoIoT. The last part of the thesis validates the QoIoT architecture in a commercial industrial IoT service, involving autonomous mining vehicles. First, we devise experiments and propose methods for measuring the objective metrics within the QoN and QoD layers. Key results within the QoN layer include estimations of latency of IoT vehicular data, with the highest achieved of R2 accuracy of 90%, and cellular link throughput per vehicle, with R2 accuracy of 92%. Regarding QoD, we detect anomalous behavior within the data generated by the vehicles, including network and sensory data. The results show on average 0.90 F1 score when detecting network anomalies while driving the vehicle in the mining, rural, highway, and city environment, whereas 0.92 F1 anomaly detection score on average regarding speed, positioning, and heading directions in the same environments. Based on the mining stakeholder requirements, we then combine the QoN and QoD data with utility theory to calculate QoC metrics - driver’s perception, reliability, and safety of the vehicle’s operations; Finally, we aggregate the contextual metrics to measure the productivity of the vehicle, which is the ultimate proposed QoIoT metric for the mining use-case.
|
|
| 8. |
|
|
| 9. |
- Minovski, Dimitar, 1990-, et al.
(författare)
-
Throughput Prediction using Machine Learning in LTE and 5G Networks
- 2023
-
Ingår i: IEEE Transactions on Mobile Computing. - : IEEE. - 1536-1233 .- 1558-0660. ; 22:3, s. 1825-1840
-
Tidskriftsartikel (refereegranskat)abstract
- The emergence of novel cellular network technologies, within 5G, are envisioned as key enablers of a new set of use-cases, including industrial automation, intelligent transportation, and tactile internet. The critical nature of the traffic requirements ranges from ultra-reliable communications, massive connectivity, and enhanced mobile broadband. Thus, the growing research on cellular network monitoring and prediction aims for ensuring a satisfied user-base and fulfillment of service level agreements. The scope of this study is to develop an approach for predicting the cellular link throughput of end-users, with a goal to benchmark the performance of network slices. First, we report and analyze a measurement study involving real-life cases, such as driving in urban, sub-urban, and rural areas, as well as tests in large crowded areas. Second, we develop machine learning models using lower-layer metrics, describing the radio environment, to predict the available throughput. The models are initially validated on the LTE network and then applied to a non-standalone 5G network. Finally, we suggest scaling the proposed model into the future standalone 5G network. We have achieved 93% and 84% R^2 accuracy, with 0.06 and 0.17 mean squared error, in predicting the end-user's throughput in LTE and non-standalone 5G network, respectively.
|
|
| 10. |
- Minovski, Dimitar, 1990-
(författare)
-
Towards Quality of Experience for Industrial Internet of Things
- 2020
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Today’s research on Quality of Experience (QoE) mainly addresses multimedia services, where the end-users' subjective perception is the prime factor of determining the QoE. With the introduction of the Internet of Things (IoT), there is a need for new ways of evaluating the QoE. Emerging IoT services, such as remotely-controlled operations, autonomous vehicles (AVs), and energy management, are more complex, creating additional quality requirements emerging from the machine-to-machine (M2M) communication and autonomous processes. One challenge, as an extension of the legacy QoE concept, is understanding the perception of end-users QoE in the context of IoT services. For instance, within the current state of the art in QoE it is not clear how intelligent machines can impact end-users' QoE, but also how end-users can alter or affect an intelligent machine. Another challenge is the quality evaluation of the M2M and systems that enable the machines to run by themselves. Consider a self-driving vehicle, where multiple autonomous decisions are simultaneously made as a result of predictive models that reason on the vehicle's generated data. An evaluation of the predictive models is inevitable due to abundance of the potential sources of failures. A quality degradation of the IoT hardware, the software enabling autonomous decision, and the M2M communication can raise life-threatening concerns, directly impacting the end-users' QoE. In this thesis, we argue for a paradigm shift in the QoE area that understands the relationships between humans and intelligent machines, as well as within the machines. Our contributions are as follows: first, we introduce the term Quality of IoT-experience (QoIoT) to extend the conventional QoE approaches in covering IoT services. Within QoIoT, we consider a quality evaluation from the perspective of the end-users, as well as from intelligent machines. The end-user's perception is captured by following the conventional QoE approaches, while regarding intelligent machine we propose the usage of objective metrics to describe their experiences and performance. As our second contribution, we propose a novel QoIoT architecture that consists of a layered methodology in order to determine the overall QoIoT. The QoIoT architecture, firstly, models the data-sources of an IoT service, classified within four layers: physical, network, application, and virtual. Secondly, the architecture proposes three layers for measuring the QoIoT by considering Quality of Data (QoD), Quality of Network (QoN), and Quality of Context (QoC), with QoC being the prime layer in measuring the objective performance metrics. Finally, the third contribution considers a case-study of cellular IoT, involving autonomous mining vehicles, which we utilize to achieve a preliminary results that validate the proposed QoIoT architecture.
|
|
