| 1. |
- Ning, Xin, et al.
(författare)
-
DILF : Differentiable rendering-based multi-view Image–Language Fusion for zero-shot 3D shape understanding
- 2024
-
Ingår i: Information Fusion. - Amsterdam : Elsevier. - 1566-2535 .- 1872-6305. ; 102, s. 1-12
-
Tidskriftsartikel (refereegranskat)abstract
- Zero-shot 3D shape understanding aims to recognize “unseen” 3D categories that are not present in training data. Recently, Contrastive Language–Image Pre-training (CLIP) has shown promising open-world performance in zero-shot 3D shape understanding tasks by information fusion among language and 3D modality. It first renders 3D objects into multiple 2D image views and then learns to understand the semantic relationships between the textual descriptions and images, enabling the model to generalize to new and unseen categories. However, existing studies in zero-shot 3D shape understanding rely on predefined rendering parameters, resulting in repetitive, redundant, and low-quality views. This limitation hinders the model's ability to fully comprehend 3D shapes and adversely impacts the text–image fusion in a shared latent space. To this end, we propose a novel approach called Differentiable rendering-based multi-view Image–Language Fusion (DILF) for zero-shot 3D shape understanding. Specifically, DILF leverages large-scale language models (LLMs) to generate textual prompts enriched with 3D semantics and designs a differentiable renderer with learnable rendering parameters to produce representative multi-view images. These rendering parameters can be iteratively updated using a text–image fusion loss, which aids in parameters’ regression, allowing the model to determine the optimal viewpoint positions for each 3D object. Then a group-view mechanism is introduced to model interdependencies across views, enabling efficient information fusion to achieve a more comprehensive 3D shape understanding. Experimental results can demonstrate that DILF outperforms state-of-the-art methods for zero-shot 3D classification while maintaining competitive performance for standard 3D classification. The code is available at https://github.com/yuzaiyang123/DILP. © 2023 The Author(s)
|
|
| 2. |
- Ran, Hang, et al.
(författare)
-
Learning optimal inter-class margin adaptively for few-shot class-incremental learning via neural collapse-based meta-learning
- 2024
-
Ingår i: Information Processing & Management. - London : Elsevier. - 0306-4573 .- 1873-5371. ; 61:3
-
Tidskriftsartikel (refereegranskat)abstract
- Few-Shot Class-Incremental Learning (FSCIL) aims to learn new classes incrementally with a limited number of samples per class. It faces issues of forgetting previously learned classes and overfitting on few-shot classes. An efficient strategy is to learn features that are discriminative in both base and incremental sessions. Current methods improve discriminability by manually designing inter-class margins based on empirical observations, which can be suboptimal. The emerging Neural Collapse (NC) theory provides a theoretically optimal inter-class margin for classification, serving as a basis for adaptively computing the margin. Yet, it is designed for closed, balanced data, not for sequential or few-shot imbalanced data. To address this gap, we propose a Meta-learning- and NC-based FSCIL method, MetaNC-FSCIL, to compute the optimal margin adaptively and maintain it at each incremental session. Specifically, we first compute the theoretically optimal margin based on the NC theory. Then we introduce a novel loss function to ensure that the loss value is minimized precisely when the inter-class margin reaches its theoretically best. Motivated by the intuition that “learn how to preserve the margin” matches the meta-learning's goal of “learn how to learn”, we embed the loss function in base-session meta-training to preserve the margin for future meta-testing sessions. Experimental results demonstrate the effectiveness of MetaNC-FSCIL, achieving superior performance on multiple datasets. The code is available at https://github.com/qihangran/metaNC-FSCIL. © 2024 The Author(s)
|
|
| 3. |
- Tian, Songsong, et al.
(författare)
-
A survey on few-shot class-incremental learning
- 2024
-
Ingår i: Neural Networks. - Oxford : Elsevier. - 0893-6080 .- 1879-2782. ; 169, s. 307-324
-
Forskningsöversikt (refereegranskat)abstract
- Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta learning-based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective. © 2023 The Author(s)
|
|
| 4. |
- Yu, Zaiyang, et al.
(författare)
-
MV-ReID : 3D Multi-view Transformation Network for Occluded Person Re-Identification
- 2024
-
Ingår i: Knowledge-Based Systems. - Amsterdam : Elsevier. - 0950-7051 .- 1872-7409. ; 283
-
Tidskriftsartikel (refereegranskat)abstract
- Re-identification (ReID) of occluded persons is a challenging task due to the loss of information in scenes with occlusions. Most existing methods for occluded ReID use 2D-based network structures to directly extract representations from 2D RGB (red, green, and blue) images, which can result in reduced performance in occluded scenes. However, since a person is a 3D non-grid object, learning semantic representations in a 2D space can limit the ability to accurately profile an occluded person. Therefore, it is crucial to explore alternative approaches that can effectively handle occlusions and leverage the full 3D nature of a person. To tackle these challenges, in this study, we employ a 3D view-based approach that fully utilizes the geometric information of 3D objects while leveraging advancements in 2D-based networks for feature extraction. Our study is the first to introduce a 3D view-based method in the areas of holistic and occluded ReID. To implement this approach, we propose a random rendering strategy that converts 2D RGB images into 3D multi-view images. We then use a 3D Multi-View Transformation Network for ReID (MV-ReID) to group and aggregate these images into a unified feature space. Compared to 2D RGB images, multi-view images can reconstruct occluded portions of a person in 3D space, enabling a more comprehensive understanding of occluded individuals. The experiments on benchmark datasets demonstrate that the proposed method achieves state-of-the-art results on occluded ReID tasks and exhibits competitive performance on holistic ReID tasks. These results also suggest that our approach has the potential to solve occlusion problems and contribute to the field of ReID. The source code and dataset are available at https://github.com/yuzaiyang123/MV-Reid. © 2023 Elsevier B.V.
|
|
| 5. |
- Chen, Guang, et al.
(författare)
-
NeuroIV : Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations
- 2022
-
Ingår i: IEEE transactions on intelligent transportation systems (Print). - : Institute of Electrical and Electronics Engineers (IEEE). - 1524-9050 .- 1558-0016. ; 23:2, s. 1171-1183
-
Tidskriftsartikel (refereegranskat)abstract
- Neuromorphic vision sensors such as the Dynamic and Active-pixel Vision Sensor (DAVIS) using silicon retina are inspired by biological vision, they generate streams of asynchronous events to indicate local log-intensity brightness changes. Their properties of high temporal resolution, low-bandwidth, lightweight computation, and low-latency make them a good fit for many applications of motion perception in the intelligent vehicle. However, as a younger and smaller research field compared to classical computer vision, neuromorphic vision is rarely connected with the intelligent vehicle. For this purpose, we present three novel datasets recorded with DAVIS sensors and depth sensor for the distracted driving research and focus on driver drowsiness detection, driver gaze-zone recognition, and driver hand-gesture recognition. To facilitate the comparison with classical computer vision, we record the RGB, depth and infrared data with a depth sensor simultaneously. The total volume of this dataset has 27360 samples. To unlock the potential of neuromorphic vision on the intelligent vehicle, we utilize three popular event-encoding methods to convert asynchronous event slices to event-frames and adapt state-of-the-art convolutional architectures to extensively evaluate their performances on this dataset. Together with qualitative and quantitative results, this work provides a new database and baseline evaluations named NeuroIV in cross-cutting areas of neuromorphic vision and intelligent vehicle.
|
|
| 6. |
- Feng, Zhenhua, et al.
(författare)
-
Multicore-Fiber-Enabled WSDM Optical Access Network With Centralized Carrier Delivery and RSOA-Based Adaptive Modulation
- 2015
-
Ingår i: IEEE PHOTONICS JOURNAL. - : Institute of Electrical and Electronics Engineers (IEEE). - 1943-0655. ; 7:4
-
Tidskriftsartikel (refereegranskat)abstract
- We proposed and experimentally demonstrated a wavelength-space division multiplexing (WSDM) optical access network architecture with centralized optical carrier delivery utilizing multicore fibers (MCFs) and adaptive modulation based on reflective semiconductor amplifier (RSOA). In our experiment, five of the outer cores are used for undirectional downstream (DS) transmission only, whereas the remaining outer core is utilized as a dedicated channel to transmit upstream (US) signals. Optical carriers for US are delivered from the optical line terminal (OLT) to the optical network unit (ONU) via the inner core and then transmitted back to the OLT after amplification and modulation by the RSOA in the colorless ONU side. The mobile backhaul (MB) service is also supported by the inner core. Wavelengths used in US transmission should be different from that of the MB in order to avoid the Rayleigh backscattering effect in bidirectional transmission. With quadrature phase-shift keying-orthogonal frequency-division multiplexing (QPSK-OFDM) modulation format, the aggregation DS capacity reaches 250 Gb/s using five outer cores and ten wavelengths, and it can be further scaled to 1 Tb/s using 20 wavelengths modulated with 16 QAM-OFDM. For US transmission, 2.5 Gb/s QPSK-OFDM transmission can be achieved just using a low-bandwidth RSOA, and adaptive modulation is applied to the RSOA to further enhance the US data rate to 3.12 Gb/s. As an emulation of high-speed MB transmission, 48 Gb/s inphase and quadrature (IQ) modulated popularization division multiplexing (PDM)-QPSK signal is transmitted in the inner core of MCF and coherently detected in the OLT side. Both DS and US optical signals exhibit acceptable performance with sufficient power budget.
|
|
| 7. |
- Kerrebrouck, Joris Van, et al.
(författare)
-
High-speed PAM4-based Optical SDM Interconnects with Directly Modulated Long-wavelength VCSEL
- 2019
-
Ingår i: Journal of Lightwave Technology. - : IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. - 0733-8724 .- 1558-2213. ; 37:2, s. 356-362
-
Tidskriftsartikel (refereegranskat)abstract
- This paper reports the demonstration of high-speed PAM-4 transmission using a 1.5-μm single-mode vertical cavity surface emitting laser (SM-VCSEL) over multicore fiber with 7 cores over different distances. We have successfully generated up to 70 Gbaud 4-level pulse amplitude modulation (PAM-4) signals with a VCSEL in optical back-to-back, and transmitted 50 Gbaud PAM-4 signals over both 1-km dispersion-uncompensated and 10-km dispersion-compensated in each core, enabling a total data throughput of 700 Gbps over the 7-core fiber. Moreover, 56 Gbaud PAM-4 over 1-km has also been shown, whereby unfortunately not all cores provide the required 3.8 × 10$^-3$bit error rate (BER) for the 7% overhead-hard decision forward error correction (7% OH HDFEC). The limited bandwidth of the VCSEL and the adverse chromatic dispersion of the fiber are suppressed with pre-equalization based on accurate end-to-end channel characterizations. With a digital post-equalization, BER performance below the 7% OH-HDFEC limit is achieved over all cores. The demonstrated results show a great potential to realize high-capacity and compact short-reach optical interconnects for data centers.
|
|
| 8. |
|
|
| 9. |
- Ning, Tiao, et al.
(författare)
-
Local origin or external input : modern horse origin in East Asia
- 2019
-
Ingår i: BMC Evolutionary Biology. - : BMC. - 1471-2148. ; 19:1
-
Tidskriftsartikel (refereegranskat)abstract
- Background Despite decades of research, the horse domestication scenario in East Asia remains poorly understood. Results The study identified 16 haplogroups with fine-scale phylogenetic resolution using mitochondrial genomes of 317 horse samples. The time to the most recent common ancestor of the 16 haplogroups ranges from [0.8-3.1] thousand years ago (KYA) to [7.9-27.1] KYA. With combined analyses of the mitochondrial control region for 35 extant Przewalski's horses, 3544 modern and 203 ancient horses across the world, researchers provide evidence for that East Asian prevalent haplogroups Q and R were indigenously domesticated or they were involved in numerous distinct genetic components from wild horses in the southern part of East Asia. These events of haplotypes Q and R occurred during 4.7 to 16.3 KYA and 2.1 to 11.5 KYA, respectively. The diffusion of preponderant European haplogroups L from west to East Asia is consistent with the external gene input. Furthermore, genetic differences were detected between northern East Asia and southern East Asia cohorts by Principal Component Analysis, Analysis of Molecular Variance test, the chi(2) test and phylogeographic analyses. Conclusions All results suggest a complex picture of horse domestication, as well as geographic pattern in East Asia. Both local origin and external input occurred in East Asia horse populations. And besides, there are at least two different domestication or hybridization centers in East Asia.
|
|
| 10. |
- Ning, Xin, et al.
(författare)
-
ICGNet : An intensity-controllable generation network based on covering learning for face attribute synthesis
- 2024
-
Ingår i: Information Sciences. - New York : Elsevier. - 0020-0255 .- 1872-6291. ; 660
-
Tidskriftsartikel (refereegranskat)abstract
- Face-attribute synthesis is a typical application of neural network technology. However, most current methods suffer from the problem of uncontrollable attribute intensity. In this study, we proposed a novel intensity-controllable generation network (ICGNet) based on covering learning for face attribute synthesis. Specifically, it includes an encoder module based on the principle of homology continuity between homologous samples to map different facial images onto the face feature space, which constructs sufficient and effective representation vectors by extracting the input information from different condition spaces. It then models the relationships between attribute instances and representational vectors in space to ensure accurate synthesis of the target attribute and complete preservation of the irrelevant region. Finally, the progressive changes in the facial attributes by applying different intensity constraints to the representation vectors. ICGNet achieves intensity-controllable face editing compared to other methods by extracting sufficient and effective representation features, exploring and transferring attribute relationships, and maintaining identity information. The source code is available at https://github.com/kllaodong/-ICGNet.•We designed a new encoder module to map face images of different condition spaces into face feature space to obtain sufficient and effective face feature representation.•Based on feature extraction, we proposed a novel Intensity-Controllable Generation Network (ICGNet), which can realize face attribute synthesis with continuous intensity control while maintaining identity and semantic information.•The quantitative and qualitative results showed that the performance of ICGNet is superior to current advanced models.© 2024 Elsevier Inc.
|
|
| 11. |
- Ran, Hang, et al.
(författare)
-
3D human pose and shape estimation via de-occlusion multi-task learning
- 2023
-
Ingår i: Neurocomputing. - Amsterdam : Elsevier. - 0925-2312 .- 1872-8286. ; 548
-
Tidskriftsartikel (refereegranskat)abstract
- Three-dimensional human pose and shape estimation is to compute a full human 3D mesh given a single image. The contamination of features caused by occlusion usually degrades its performance significantly. Recent progress in this field typically addressed the occlusion problem implicitly. By contrast, in this paper, we address it explicitly using a simple yet effective de-occlusion multi-task learning network. Our key insight is that feature for mesh parameter regression should be noiseless. Thus, in the feature space, our method disentangles the occludee that represents the noiseless human feature from the occluder. Specifically, a spatial regularization and an attention mechanism are imposed in the backbone of our network to disentangle the features into different channels. Furthermore, two segmentation tasks are proposed to supervise the de-occlusion process. The final mesh model is regressed by the disentangled occlusion-aware features. Experiments on both occlusion and non-occlusion datasets are conducted, and the results prove that our method is superior to the state-of-the-art methods on two occlusion datasets, while achieving competitive performance on a non-occlusion dataset. We also demonstrate that the proposed de-occlusion strategy is the main factor to improve the robustness against occlusion. The code is available at https://github.com/qihangran/De-occlusion_MTL_HMR. © 2023
|
|
| 12. |
- Tian, Liting, et al.
(författare)
-
Lead concentration in plasma as a biomarker of exposure and risk, and modification of toxicity by delta-aminolevulinic acid dehydratase gene polymorphism
- 2013
-
Ingår i: Toxicology Letters. - : Elsevier BV. - 1879-3169 .- 0378-4274. ; 221:2, s. 102-109
-
Tidskriftsartikel (refereegranskat)abstract
- Blood lead concentration (B-Pb), the main biomarker of lead exposure and risk, is curvi-linearily related to exposure. We assessed plasma lead (P-Pb) as a marker for both lead exposure and toxic effects. We examined claims that delta-aminolevulinic acid dehydratase genotype (ALAD) can modify lead toxicity. In 290 lead-exposed and 91 unexposed Chinese workers, we determined P-Pb, B-Pb, urinary lead (U-Pb), AMD polymorphism (rs1800435, ALAD112; TaqMan assay), and also toxic effects on heme synthesis (blood zinc protoporphyrin and hemoglobin, urinary delta-aminolevulic acid), on the kidneys (urinary albumin, beta(2)microglobulin and N-acetyl-beta-D-glucosaminidase) and on the peripheral nervous system (sensory and motor conduction velocities). In exposed workers, median P-Pb was 4.10 (range 0.35-27) mu g/L, B-Pb 401 (110-950) mu g/L, and U-Pb 188 (22-590) mu g/g creatinine. P-Pb had a higher ratio between exposed and unexposed workers (median 39, range 18-110) than B-Pb (19, 15-36; p<0.001) and U-Pb (28, 15-36; p<0.001). All three biomarkers were associated with all toxic effects (P-Pb: r(s)= -0.10 to 0.79; B-Pb: r(s) = -0.08 to 0.75; all p <0.05). In the exposed workers, B-Pb and U-Pb were significantly higher (p = 0.04) in AIAD2 carriers (7% in the exposed population) than in ALAD1 homozygotes. P-Pb values were similar; ALAD1 homozygotes suffered higher kidney toxicity at the same P-Pb. Conclusions: (i) P-Pb has advantages over B-Pb as a biomarker of high Pb exposure, but it was not significantly better as an index of risk of toxicity. (ii) The ALAD genotype modifies toxicokinetics and toxicodynamics. (C) 2013 Elsevier Ireland Ltd. All rights reserved.
|
|
| 13. |
- Tian, Songsong, et al.
(författare)
-
Continuous transfer of neural network representational similarity for incremental learning
- 2023
-
Ingår i: Neurocomputing. - Amsterdam : Elsevier. - 0925-2312 .- 1872-8286. ; 545
-
Tidskriftsartikel (refereegranskat)abstract
- The incremental learning paradigm in machine learning has consistently been a focus of academic research. It is similar to the way in which biological systems learn, and reduces energy consumption by avoiding excessive retraining. Existing studies utilize the powerful feature extraction capabilities of pre-trained models to address incremental learning, but there remains a problem of insufficient utilization of neural network feature knowledge. To address this issue, this paper proposes a novel method called Pre-trained Model Knowledge Distillation (PMKD) which combines knowledge distillation of neural network representations and replay. This paper designs a loss function based on centered kernel alignment to transfer neural network representations knowledge from the pre-trained model to the incremental model layer-by-layer. Additionally, the use of memory buffer for Dark Experience Replay helps the model retain past knowledge better. Experiments show that PMKD achieved superior performance on various datasets and different buffer sizes. Compared to other methods, our class incremental learning accuracy reached the best performance. The open-source code is published athttps://github.com/TianSongS/PMKD-IL. © 2023 The Author(s)
|
|
| 14. |
- Yu, Hua, et al.
(författare)
-
Organic coating on sulfate and soot particles during late summer in the Svalbard Archipelago
- 2019
-
Ingår i: Atmospheric Chemistry And Physics. - : Copernicus GmbH. - 1680-7316 .- 1680-7324. ; 19:15, s. 10433-10446
-
Tidskriftsartikel (refereegranskat)abstract
- Interaction of anthropogenic particles with radiation and clouds plays an important role in Arctic climate change. The mixing state of aerosols is a key parameter to influence aerosol radiation and aerosol-cloud interactions. However, little is known of this parameter in the Arctic, preventing an accurate representation of this information in global models. Here we used transmission electron microscopy with energy-dispersive X-ray spectrometry, scanning electron microscopy, nanoscale secondary ion mass spectrometry, and atomic forces microscopy to determine the size and mixing state of individual sulfate and carbonaceous particles at 100 nm to 2 mu m collected in the Svalbard Archipelago in summer. We found that 74% by number of non-sea-salt sulfate particles were coated with organic matter (OM); 20% of sulfate particles also had soot inclusions which only appeared in the OM coating. The OM coating is estimated to contribute 63% of the particle volume on average. To understand how OM coating influences optical properties of sulfate particles, a Mie core-shell model was applied to calculate optical properties of individual sulfate particles. Our result shows that the absorption cross section of individual OM-coated particles significantly increased when assuming the OM coating as light-absorbing brown carbon. Microscopic observations here suggest that OM modulates the mixing structure of fine Arctic sulfate particles, which may determine their hygroscopicity and optical properties.
|
|
