| 1. |
|
|
| 2. |
- Bosch, David, 1997, et al.
(författare)
-
Double Descent in Feature Selection: Revisiting LASSO and Basis Pursuit
- 2021
-
Ingår i: Thirty-eighth International Conference on Machine Learning, ICML 2021.
-
Konferensbidrag (refereegranskat)abstract
- We present a novel analysis of feature selection in linear models by the convex framework of least absolute shrinkage operator (LASSO) and basis pursuit (BP). Our analysis pertains to a general overparametrized scenario. When the numbers of the features and the data samples grow proportionally, we obtain precise expressions for the asymptotic generalization error of LASSO and BP. Considering a mixture of strong and weak features, we provide insights into regularization trade-offs for double descent for l1 norm minimization. We validate these results with numerical experiments.
|
|
| 3. |
- Bosch, David, 1997, et al.
(författare)
-
Precise Asymptotic Analysis of Deep Random Feature Models
- 2023
-
Ingår i: Proceedings of Machine Learning Research. - 2640-3498. ; 195, s. 4132-4179
-
Konferensbidrag (refereegranskat)abstract
- We provide exact asymptotic expressions for the performance of regression by an L−layer deep random feature (RF) model, where the input is mapped through multiple random embedding and non-linear activation functions. For this purpose, we establish two key steps: First, we prove a novel universality result for RF models and deterministic data, by which we demonstrate that a deep random feature model is equivalent to a deep linear Gaussian model that matches it in the first and second moments, at each layer. Second, we make use of the convex Gaussian Min-Max theorem multiple times to obtain the exact behavior of deep RF models. We further characterize the variation of the eigendistribution in different layers of the equivalent Gaussian model, demonstrating that depth has a tangible effect on model performance despite the fact that only the last layer of the model is being trained.
|
|
| 4. |
- Bosch, David, 1997, et al.
(författare)
-
Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves
- 2023
-
Ingår i: Proceedings of Machine Learning Research. - 2640-3498. ; 206, s. 11371-11414
-
Konferensbidrag (refereegranskat)abstract
- We compute precise asymptotic expressions for the learning curves of least squares random feature (RF) models with either a separable strongly convex regularization or the ℓ1 regularization. We propose a novel multi-level application of the convex Gaussian min max theorem (CGMT) to overcome the traditional difficulty of finding computable expressions for random features models with correlated data. Our result takes the form of a computable 4-dimensional scalar optimization. In contrast to previous results, our approach does not require solving an often intractable proximal operator, which scales with the number of model parameters. Furthermore, we extend the universality results for the training and generalization errors for RF models to ℓ1 regularization. In particular, we demonstrate that under mild conditions, random feature models with elastic net or ℓ1 regularization are asymptotically equivalent to a surrogate Gaussian model with the same first and second moments. We numerically demonstrate the predictive capacity of our results, and show experimentally that the predicted test error is accurate even in the nonasymptotic regime.
|
|
| 5. |
- Ghanem, Sally, et al.
(författare)
-
Robust Group Subspace Recovery: A New Approach for Multi-Modality Data Fusion
- 2020
-
Ingår i: IEEE Sensors Journal. - 1558-1748 .- 1530-437X. ; 20:20, s. 12307-12316
-
Tidskriftsartikel (refereegranskat)abstract
- Robust Subspace Recovery (RoSuRe) algorithm was recently introduced as a principled and numerically efficient algorithm that unfolds underlying Unions of Subspaces (UoS) structure, present in the data. The union of Subspaces (UoS) is capable of identifying more complex trends in data sets than simple linear models. We build on and extend RoSuRe to prospect the structure of different data modalities individually. We propose a novel multi-modal data fusion approach based on group sparsity which we refer to as Robust Group Subspace Recovery (RoGSuRe). Relying on a bi-sparsity pursuit paradigm and non-smooth optimization techniques, the introduced framework learns a new joint representation of the time series from different data modalities, respecting an underlying UoS model. We subsequently integrate the obtained structures to form a unified subspace structure. The proposed approach exploits the structural dependencies between the different modalities data to cluster the associated target objects. The resulting fusion of the unlabeled sensors' data from experiments on audio and magnetic data has shown that our method is competitive with other state of the art subspace clustering methods. The resulting UoS structure is employed to classify newly observed data points, highlighting the abstraction capacity of the proposed method.
|
|
| 6. |
- Huang, Yuming, et al.
(författare)
-
Community Detection and Improved Detectability in Multiplex Networks
- 2020
-
Ingår i: IEEE Transactions on Network Science and Engineering. - 2327-4697. ; 7:3, s. 1697-1709
-
Tidskriftsartikel (refereegranskat)abstract
- Belief propagation is a technique to optimize probabilistic graphical models, and has been used to solve the community detection problem for networks described by the stochastic block model. In this work, we investigate the community detection problem in multiplex networks with generic community label constraints using the belief propagation algorithm. Our main contribution is a generative model that does not assume consistent communities between layers and allows a potentially heterogeneous community structure, suitable in many real world multiplex networks, such as social networks. We show by numerical experiments that in the presence of consistent communities between different layers, consistent communities are matched, and the detectability is improved over single layers. We compare it with a "correlated model" which has the prior knowledge of community correlation between layers. Similar detectability improvement is obtained, even though our model has much milder assumptions than the "correlated model". When the network has heterogeneous community structures, our model is shown to yield a better detection performance over a certain parameter range.
|
|
| 7. |
- Huang, Yuming, et al.
(författare)
-
Fusion of Community Structures in Multiplex Networks by Label Constraints
- 2018
-
Ingår i: 26th European Signal Processing Conference (EUSIPCO). ; , s. 887-891
-
Konferensbidrag (refereegranskat)abstract
- We develop a Belief Propagation algorithm for community detection problem in multiplex networks, which more accurately represents many real-world systems. Previous works have established that real world multiplex networks exhibit redundant structures/communities, and that community detection performance improves by aggregating (fusing) redundant layers which are generated from the same Stochastic Block Model (SBM). We introduce a probability model for generic multiplex networks, aiming to fuse community structure across layers, without assuming or seeking the same SBM generative model for different layers. Numerical experiment shows that our model finds out consistent communities between layers and yields a significant detectability improvement over the single layer architecture. Our model also achieves a comparable performance to a reference model where we assume consistent communities in prior. Finally we compare our method with multilayer modularity optimization in heterogeneous networks, and show that our method detects correct community labels more reliably.
|
|
| 8. |
- Jiang, Bo, et al.
(författare)
-
Dynamic graph learning: A structure-driven approach
- 2021
-
Ingår i: Mathematics. - : MDPI AG. - 2227-7390. ; 9:2, s. 1-20
-
Tidskriftsartikel (refereegranskat)abstract
- The purpose of this paper is to infer a dynamic graph as a global (collective) model of time-varying measurements at a set of network nodes. This model captures both pairwise as well as higher order interactions (i.e., more than two nodes) among the nodes. The motivation of this work lies in the search for a connectome model which properly captures brain functionality across all regions of the brain, and possibly at individual neurons. We formulate it as an optimization problem, a quadratic objective functional and tensor information of observed node signals over short time intervals. The proper regularization constraints reflect the graph smoothness and other dynamics involving the underlying graph’s Laplacian, as well as the time evolution smoothness of the underlying graph. The resulting joint optimization is solved by a continuous relaxation of the weight parameters and an introduced novel gradient-projection scheme. While the work may be applicable to any time-evolving data set (e.g., fMRI), we apply our algorithm to a real-world dataset comprising recorded activities of individual brain cells. The resulting model is shown to be not only viable but also efficiently computable.
|
|
| 9. |
- Khanzadi, M Reza, 1983, et al.
(författare)
-
A model-based analysis of phase jitter in RF oscillators
- 2012
-
Ingår i: 2012 IEEE International Frequency Control Symposium, IFCS 2012, Proceedings. - 9781457718199 ; , s. 508-511
-
Konferensbidrag (refereegranskat)abstract
- The closed-form autocorrelation function of the phase jitter accumulation process in presence of 1/f 3 and 1/f 2 shape noises is derived from the single-sideband (SSB) phase noise (PN) measurements. Exploiting the calculated autocorrelation function, a lower bound for the minimum achievable mean square error (MSE) of the PN prediction in a typical single-input singleoutput communication system is computed. This bound links the performance of a communication system suffering from the PN directly to the SSB PN measurements.
|
|
| 10. |
- Khanzadi, M Reza, 1983, et al.
(författare)
-
A Novel Cognitive Modulation Method Considering the Performance of Primary User
- 2010
-
Ingår i: Wireless Advanced (WiAD), 2010 6th Conference on. - 9781424470693 ; , s. 1 - 6
-
Konferensbidrag (refereegranskat)abstract
- This paper proposes a new modulation method foran uncoded cognitive transmission (secondary user transmission)in presence of a Primary User (PU) for the AWGN channel.Interference of the PU is assumed to be known at the transmitterof Cognitive User (CU) non-causally. Based on this knowledge,for the design of the modulator and demodulator of the CU,a symbol by symbol approach is studied which can fulfill thecoexistence conditions of the CU and the PU of the band. In thisscheme, the modulator and demodulator of CU are designedjointly by solving an optimization problem to mitigate theinterference of the PU and minimize the symbol error probability(Pe) in CU’s communication link without increasing the symbolerror probability (Pe) of the PU. The proposed method is amodulation approach in a single (complex-valued) dimensionrather than a high dimensional coding scheme. Although thisone-dimensional method is not capacity achieving, we show it stillhas a remarkable performance with low amount of complexity.An implementation algorithm for our modulation method is alsopresented and the performance of this method is evaluated byexperimental results.
|
|
