| 1. |
- Bampa, Maria, et al.
(författare)
-
EpidRLearn : Learning Intervention Strategies for Epidemics with Reinforcement Learning
- 2022
-
Ingår i: Artificial Intelligence in Medicine. - Cham : Springer Nature. - 9783031093425 ; , s. 189-199
-
Konferensbidrag (refereegranskat)abstract
- Epidemics of infectious diseases can pose a serious threat to public health and the global economy. Despite scientific advances, containment and mitigation of infectious diseases remain a challenging task. In this paper, we investigate the potential of reinforcement learning as a decision making tool for epidemic control by constructing a deep Reinforcement Learning simulator, called EpidRLearn, composed of a contact-based, age-structured extension of the SEIR compartmental model, referred to as C-SEIR. We evaluate EpidRLearn by comparing the learned policies to two deterministic policy baselines. We further assess our reward function by integrating an alternative reward into our deep RL model. The experimental evaluation indicates that deep reinforcement learning has the potential of learning useful policies under complex epidemiological models and large state spaces for the mitigation of infectious diseases, with a focus on COVID-19.
|
|
| 2. |
- Beikmohammadi, Ali, 1995-, et al.
(författare)
-
Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge
- 2024
-
Ingår i: Information Sciences. - 0020-0255 .- 1872-6291. ; 661
-
Tidskriftsartikel (refereegranskat)abstract
- Despite the huge success of reinforcement learning (RL) in solving many difficult problems, its Achilles heel has always been sample inefficiency. On the other hand, in RL, taking advantage of prior knowledge, intentionally or unintentionally, has usually been avoided, so that, training an agent from scratch is common. This not only causes sample inefficiency but also endangers safety –especially during exploration. In this paper, we help the agent learn from the environment by using the pre-existing (but not necessarily exact or complete) solution for a task. Our proposed method can be integrated with any RL algorithm developed based on policy gradient and actor-critic methods. The results on five tasks with different difficulty levels by using two well-known actor-critic-based methods as the backbone of our proposed method (SAC and TD3) show our success in greatly improving sample efficiency and final performance. We have gained these results alongside robustness to noisy environments at the cost of just a slight computational overhead, which is negligible.
|
|
| 3. |
- Beikmohammadi, Ali, 1995-, et al.
(författare)
-
Comparing NARS and Reinforcement Learning : An Analysis of ONA and Q-Learning Algorithms
- 2023
-
Ingår i: Artificial General Intelligence. - : Springer. - 9783031334696 - 9783031334689 ; , s. 21-31
-
Konferensbidrag (refereegranskat)abstract
- In recent years, reinforcement learning (RL) has emerged as a popular approach for solving sequence-based tasks in machine learning. However, finding suitable alternatives to RL remains an exciting and innovative research area. One such alternative that has garnered attention is the Non-Axiomatic Reasoning System (NARS), which is a general-purpose cognitive reasoning framework. In this paper, we delve into the potential of NARS as a substitute for RL in solving sequence-based tasks. To investigate this, we conduct a comparative analysis of the performance of ONA as an implementation of NARS and Q-Learning in various environments that were created using the Open AI gym. The environments have different difficulty levels, ranging from simple to complex. Our results demonstrate that NARS is a promising alternative to RL, with competitive performance in diverse environments, particularly in non-deterministic ones.
|
|
| 4. |
- Beikmohammadi, Ali, 1995-, et al.
(författare)
-
TA-Explore : Teacher-Assisted Exploration for Facilitating Fast Reinforcement Learning: Extended Abstract
- 2023
-
Ingår i: AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. - : The International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). - 9781450394321 ; , s. 2412-2414
-
Konferensbidrag (refereegranskat)abstract
- Reinforcement Learning (RL) is crucial for data-driven decision-making but suffers from sample inefficiency. This poses a risk to system safety and can be costly in real-world environments with physical interactions. This paper proposes a human-inspired framework to improve the sample efficiency of RL algorithms, which gradually provides the learning agent with simpler but similar tasks that progress toward the main task. The proposed method does not require pre-training and can be applied to any goal, environment, and RL algorithm, including value-based and policy-based methods, as well as tabular and deep-RL methods. The framework is evaluated on a Random Walk and optimal control problem with constraint, showing good performance in improving the sample efficiency of RL-learning algorithms.
|
|
| 5. |
- Benalcazar, Diego R., et al.
(författare)
-
Average Consensus With Error Correction
- 2024
-
Ingår i: IEEE Control Systems Letters. - 2475-1456. ; 8, s. 115-120
-
Tidskriftsartikel (refereegranskat)abstract
- We propose a novel method for achieving the average consensus in a distributed manner while dealing with communication compression. While it is widely recognized that distributed consensus algorithms with compression can falter due to compression-error-induced divergences, our approach integrates an error correction step to guarantee convergence towards an approximate average consensus across any bounded compression function. Significantly, with our error correction mechanism, we can achieve convergence to a solution of arbitrarily high accuracy, irrespective of how crude the compression is in a fully distributed setting. Additionally, we quantify the convergence rate and provide upper bounds for the estimation error based on the spectral properties of the underlying communication network. Simulation results validate the scalability and efficacy of our proposed algorithm.
|
|
| 6. |
- Berglund, Erik, et al.
(författare)
-
Distributed Newton Method Over Graphs : Can Sharing of Second-Order Information Eliminate the Condition Number Dependence?
- 2021
-
Ingår i: IEEE Signal Processing Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 1070-9908 .- 1558-2361. ; 28, s. 1180-1184
-
Tidskriftsartikel (refereegranskat)abstract
- One of the main advantages of second-order methods in a centralized setting is that they are insensitive to the condition number of the objective function's Hessian. For applications such as regression analysis, this means that less pre-processing of the data is required for the algorithm to work well, as the ill-conditioning caused by highly correlated variables will not be as problematic. Similar condition number independence has not yet been established for distributed methods. In this paper, we analyze the performance of a simple distributed second-order algorithm on quadratic problems and show that its convergence depends only logarithmically on the condition number. Our empirical results indicate that the use of second-order information can yield large efficiency improvements over first-order methods, both in terms of iterations and communications, when the condition number is of the same order of magnitude as the problem dimension.
|
|
| 7. |
- Berglund, Erik, et al.
(författare)
-
Revisiting the Curvature-aided IAG : Improved Theory and Reduced Complexity
- 2023
-
Ingår i: IFAC-PapersOnLine. - : Elsevier BV. - 2405-8963. ; 56:2, s. 5221-5226, s. 5221-5226
-
Tidskriftsartikel (refereegranskat)abstract
- The curvature-aided IAG (CIAG) algorithm is an efficient asynchronous optimization method that accelerates IAG using a delay compensation technique. However, existing step-size rules for CIAG are conservative and hard to implement, and the Hessian computation in CIAG is often computationally expensive. To alleviate these issues, we first provide an easy-to-implement and less conservative step-size rule for CIAG. Next, we propose a modified CIAG algorithm that reduces the computational complexity by approximating the Hessian with a constant matrix. Convergence results are derived for each algorithm on both convex and strongly convex problems, and numerical experiments on logistic regression demonstrate their effectiveness in practice.
|
|
| 8. |
- Chaliane Junior, Guilherme Dinis, et al.
(författare)
-
Policy Evaluation with Delayed, Aggregated Anonymous Feedback
- 2022
-
Ingår i: Discovery Science. - Cham : Springer Nature. - 9783031188398 - 9783031188404 ; , s. 114-123
-
Konferensbidrag (refereegranskat)abstract
- In reinforcement learning, an agent makes decisions to maximize rewards in an environment. Rewards are an integral part of the reinforcement learning as they guide the agent towards its learning objective. However, having consistent rewards can be infeasible in certain scenarios, due to either cost, the nature of the problem or other constraints. In this paper, we investigate the problem of delayed, aggregated, and anonymous rewards. We propose and analyze two strategies for conducting policy evaluation under cumulative periodic rewards, and study them by making use of simulation environments. Our findings indicate that both strategies can achieve similar sample efficiency as when we have consistent rewards.
|
|
| 9. |
- Devagiri, Vishnu Manasa
(författare)
-
Clustering Techniques for Mining and Analysis of Evolving Data
- 2021
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The amount of data generated is on rise due to increased demand for fields like IoT, smart monitoring applications, etc. Data generated through such systems have many distinct characteristics like continuous data generation, evolutionary, multi-source nature, and heterogeneity. In addition, the real-world data generated in these fields is largely unlabelled. Clustering is an unsupervised learning technique used to group, analyze and interpret unlabelled data. Conventional clustering algorithms are not suitable for dealing with data having previously mentioned characteristics due to memory and computational constraints, their inability to handle concept drift, distributed location of data. Therefore novel clustering approaches capable of analyzing and interpreting evolving and/or multi-source streaming data are needed. The thesis is focused on building evolutionary clustering algorithms for data that evolves over time. We have initially proposed an evolutionary clustering approach, entitled Split-Merge Clustering (Paper I), capable of continuously updating the generated clustering solution in the presence of new data. Through the progression of the work, new challenges have been studied and addressed. Namely, the Split-Merge Clustering algorithm has been enhanced in Paper II with new capabilities to deal with the challenges of multi-view data applications. A multi-view or multi-source data presents the studied phenomenon/system from different perspectives (views), and can reveal interesting knowledge that is not visible when only one view is considered and analyzed. This has motivated us to continue in this direction by designing two other novel multi-view data stream clustering algorithms. The algorithm proposed in Paper III improves the performance and interpretability of the algorithm proposed in Paper II. Paper IV introduces a minimum spanning tree based multi-view clustering algorithm capable of transferring knowledge between consecutive data chunks, and it is also enriched with a post-clustering pattern-labeling procedure. The proposed and studied evolutionary clustering algorithms are evaluated on various data sets. The obtained results have demonstrated the robustness of the algorithms for modeling, analyzing, and mining evolving data streams. They are able to adequately adapt single and multi-view clustering models by continuously integrating newly arriving data.
|
|
| 10. |
- Du, Rong, 1989-, et al.
(författare)
-
The Internet of Things as a Deep Neural Network
- 2020
-
Ingår i: IEEE Communications Magazine. - : Institute of Electrical and Electronics Engineers (IEEE). - 0163-6804 .- 1558-1896. ; 58:9, s. 20-25
-
Tidskriftsartikel (refereegranskat)abstract
- An important task in the Internet of Things (IoT) is field monitoring, where multiple IoT nodes take measurements and communicate them to the base station or the cloud for processing, inference, and analysis. When the measurements are high-dimensional (e.g., videos or time-series data), IoT networks with limited bandwidth and low-power devices may not be able to support such frequent transmissions with high data rates. To ensure communication efficiency, this article proposes to model the measurement compression at IoT nodes and the inference at the base station or cloud as a deep neural network (DNN). We propose a new framework where the data to be transmitted from nodes are the intermediate outputs of a layer of the DNN. We show how to learn the model parameters of the DNN and study the trade-off between the communication rate and the inference accuracy. The experimental results show that we can save approximately 96 percent transmissions with only a degradation of 2.5 percent in inference accuracy, which shows the potentiality to enable many new IoT data analysis applications that generate a large amount of measurements.
|
|
