The synchronization of K-valued Fuzzy cognitive maps

Abstract: Fuzzy cognitive maps (FCMs) with the abilities of knowledge representation and causal reasoning have been widely applied in the control-related fields of multi-agent systems (MAS), such as the mobile robots and unmanned aerial vehicles (UAVs). From the perspective of system applications, the cooperative control of group behaviors is a meaningful topic in this community, where the synchronization of agents is one of fundamental issues. However, how to achieve the synchronization of FCM-based control mechanism is still an open problem. In this article, two finite state fuzzy cognitive map systems with unidirectional coupling in the drive-response configuration are introduced. By utilizing matrix semi-tensor product (STP), the coupling dynamical systems are transformed into algebraic forms, based on which the necessary and sufficient conditions for complete synchronization are presented. The corresponding theoretical proofs are given and quantitative analysis are implemented. Finally, several experiments are given to prove the effectiveness of the proposed methods.


GradDiff: Gradient-based membership inference attacks against federated distillation with differential comparison

Abstract: Membership inference attacks (MIAs) has demonstrated a great threat to federated learning (FL) and its extension federated distillation (FD). However, existing research on MIAs against FD is insufficient. In this paper, we propose a novel membership inference attack named GradDiff, which is a passive gradient-based MIA employing differential comparison. Additionally, to make full use of the federated training process, we also design the gradient drift attack (GradDrift), an active version of GradDiff, in which the attacker modifies the target model by gradient tuning and is able to obtain more information about membership privacy. We conduct extensive experiments on three real-world datasets to evaluate the effectiveness of the proposed attacks. The results show that our proposed attacks can outperform the existing baseline methods in terms of precision and recall. Besides, we perform a thorough investigation of the factors that may influence the performance of MIAs against FD.


Enhancing deep feature representation in self-knowledge distillation via pyramid feature refinement

Abstract: In recent years, various self-knowledge distillation approaches have been proposed to reduce the cost of training teacher networks. However, these methods often overlook the significance of deep features. To address this limitation and strengthen the capability of deep features while preserving the ability of shallow features, we propose performing Self-Knowledge Distillation via Pyramid Feature Refinement (PR-SKD). Inspired by the representation learning characteristics of deep neural networks, PR-SKD builds a cohort of sub-networks with a pyramid architecture to hierarchically transfer refined information to the target network. According to the different contributions and functions between deep and shallow feature maps, our PR-SKD fully utilizes feature information to improve deep feature representation ability without compromising the capability of shallow feature maps. Extensive experiments on various image classification datasets demonstrate the superiority of our proposed method over widely used state-of-the-art knowledge distillation methods. The code is available at: https://github.com/wo16pao/PR-SKD.


LiDAR video object segmentation with dynamic kernel refinement

Abstract: In this paper, we formalize memory- and tracking-based methods to perform the LiDAR-based Video Object Segmentation (VOS) task, which segments points of the specific 3D target (given in the first frame) in a LiDAR sequence. LiDAR-based VOS can directly provide target-aware geometric information for practical application scenarios like behavior analysis and anticipating danger. We first construct a LiDAR-based VOS dataset named KITTI-VOS based on SemanticKITTI, which acts as a testbed and facilitates comprehensive evaluations of algorithm performance. Next, we provide two types of baselines, i.e., memory-based and tracking-based baselines, to explore this task. Specifically, the first memory-based pipeline is built on a space–time memory network equipped with the non-local spatiotemporal attention-based memory bank. We further design a more potent variant to introduce the locality into the spatiotemporal attention module by local self-attention and cross-attention modules. For the second tracking-based baseline, we modify two representative 3D object tracking methods to adapt to LiDAR-based VOS tasks. Finally, we propose a refine module that takes mask priors and generates object-aware kernels, which could boost all the baselines’ performance. We evaluate the proposed methods on the dataset and demonstrate their effectiveness.


Adaptive trajectory-constrained exploration strategy for deep reinforcement learning

Abstract: Deep reinforcement learning (DRL) faces significant challenges in addressing hard-exploration tasks with sparse or deceptive rewards and large state spaces. These challenges severely limit the practical application of DRL. Most previous exploration methods relied on complex architectures to estimate state novelty or introduced sensitive hyperparameters, resulting in instability. To mitigate these issues, we propose an efficient adaptive trajectory-constrained exploration strategy for DRL. The proposed method guides the agent’s policy away from suboptimal solutions by regarding previous offline demonstrations as references. Specifically, this approach gradually expands the exploration scope of the agent and strives for optimality in a constrained optimization manner. Additionally, we introduce a novel policy-gradient-based optimization algorithm that utilizes adaptive clipped trajectory-distance rewards for both single- and multi-agent reinforcement learning. We provide a theoretical analysis of our method, including a deduction of the worst-case approximation error bounds, highlighting the validity of our approach for enhancing exploration. To evaluate the effectiveness of the proposed method, we conducted experiments on two large 2D grid world mazes and several MuJoCo tasks. The extensive experimental results demonstrated the significant advantages of our method in achieving temporally extended exploration and avoiding myopic and suboptimal behaviors in both single- and multi-agent settings. Notably, the specific metrics and quantifiable results further support these findings. The code used in the study is available at https://github.com/buaawgj/TACE.


Unsupervised feature selection with high-order similarity learning

Abstract: Graph-based unsupervised feature selection methods have successfully processed high-dimensional data since they can effectively preserve data structure information. However, most existing methods are facing two challenges: (1) it is unreliable to exploit the intrinsic spatial structure from raw data because of the existence of redundant information and noise; (2) they only consider the first-order similar information but ignore the high-order similar information (i.e., samples with similar neighborhood network structure should be similar to each other). This study proposes an unsupervised feature selection with high-order similarity learning (HSL) to tackle the above problems. The projection matrix, first-order similar information, and high-order similar information are simultaneously learned in a unified framework, such that the intrinsic structure information can be exploited in the clean data that come from a latent low-dimensional embedding subspace. Specifically, we project raw data into a low-dimensional embedding subspace to effectively eliminate redundant information and noise. Moreover, we simultaneously consider the first-order similarity and the high-order similarity in the low-dimensional embedding space so that a better graph Laplacian matrix can be learned to fully preserve the intrinsic structure information of raw data and simultaneously utilize the label information. Furthermore, we design an effective optimization algorithm to tackle the proposed HSL. Comprehensive experiments on real-world datasets verify the superiority and effectiveness of HSL.


Self-attention deep ConvLSTM with sparse-learned channel dependencies for wearable sensor-based human activity recognition

Abstract: In this study, we propose a novel deep-learning architecture with sparse learning for human activity recognition. The proposed model contains 1D CNNs and LSTM layers with a self-attention mechanism to enhance a substantial number of time points in time-series data for human activity recognition systems. Based on the recent success of squeeze-and-excite (SE) networks, the proposed deep learning model utilizes the SE module to enhance channel-wise interdependencies, which in turn leads to a boost in performance. In addition, we utilized sparse learning to retrain only weak nodes and freeze stronger nodes in a fully connected layer prior to classification layer. Furthermore, we utilized an entropy-inspired formula to find sparsely located weaker nodes and validated our model on various datasets, including Opportunity, UCI-HAR, and WISDM. Herein, we present an extensive analysis and survey of state-of-the-art studies, in addition to our proposed research. For a fair comparison, we evaluated our deep learning architecture using various performance metrics and achieved better results; the proposed model outperformed state-of-the-art algorithms for human activity recognition.


Parallel private information retrieval protocol with index anonymity for untrusted databases

Abstract: Private information retrieval (PIR) enables a client to search data from a single (or multiple) untrusted server, without revealing which entry was queried. PIR can be divided into two categories: information-theoretic PIR (IT-PIR) and computational PIR (CPIR). However, there is a deployment challenge with IT-PIR because the non-collusion assumption is difficult to implement in practice. Meanwhile, due to the fact that CPIR involves performing cryptographic operations on each element, its performance significantly decreases as the size and number of database entries increase. To overcome these problems, based on homomorphic encryption, this paper presents a parallel PIR (ParPIR) with index anonymity for untrusted databases, which uses the batch encoding to compress request size. Furthermore, our ParPIR eliminates computationally expensive ciphertext multiplication operations, which improves the response time. Moreover, our ParPIR packs multiple responses through rotation column operations on ciphertext, thereby reducing the response size. Compared to previous PIR protocols, the offline time in our ParPIR has almost 1∼63× improvement; the online time in our ParPIR decreases by at least 49.9∼98.9%. Meanwhile, although the request size of our ParPIR has a 0∼31× increase, the response size of our ParPIR has a 0∼4095× improvement.


Subfield subcodes of projective Reed-Muller codes

Abstract: Explicit bases for the subfield subcodes of projective Reed-Muller codes over the projective plane and their duals are obtained. In particular, we provide a formula for the dimension of these codes. For the general case over the projective space, we generalize the necessary tools to deal with this case as well: we obtain a universal Gröbner basis for the vanishing ideal of the set of standard representatives of the projective space and we show how to reduce any monomial with respect to this Gröbner basis. With respect to the parameters of these codes, by considering subfield subcodes of projective Reed-Muller codes we obtain long linear codes with good parameters over a small finite field.


Multi-objective reinforcement learning for autonomous drone navigation in urban areas with wind zones

Abstract: Drones can be used for tasks such as data collection and logistics in civil engineering. Current research on autonomous drones mainly focuses on planning a safe path and avoiding obstacles in a static environment. However, navigating a drone in complex environments like urban areas involves many dynamic constraints, such as building layout, winds, and signal coverage, which are interdependent. The wind factor is the most important among these environmental factors, which may cause a drone to lose control or even crash. This paper presents a multi-objective navigation reinforcement learning algorithm (MONRL) for the drone to navigate and avoid obstacles in an unknown environment when dynamic wind zones present, with only imagery data about the building layouts. Based on a deep reinforcement learning and memory architecture, the drone develops policies to prioritize navigation decisions, optimizing the path while minimizing negative impact of winds with only sparse sensor data, in our case, camera inputs. By leveraging the advantages of the proposed method in estimating the environmental factors from previous trials, no aerodynamic force sensors are needed for the drone to develop effective strategies to navigate to target while counteracting to milder winds and dodging aways from stronger winds. The proposed method was tested in a virtual environment, and a real model of New York City. The method is expected to contribute to new automation algorithms for urban aerial logistics and future automated civil infrastructure inspection.


Personalized Federated Learning with Layer-Wise Feature Transformation via Meta-Learning

Abstract: Federated learning enables multiple clients to collaboratively learn machine learning models in a privacy-preserving manner. However, in real-world scenarios, a key challenge encountered in federated learning is the statistical heterogeneity among clients. Existing work mainly focused on a single global model shared across the clients, making it hard to generalize well to all clients due to the large discrepancy in the data distributions. To address this challenge, we propose pFedLT , a novel approach that can adapt the single global model to different data distributions. Specifically, we propose to perform a pluggable layer-wise transformation during the local update phase based on scaling and shifting operations. In particular, these operations are learned with a meta-learning strategy. By doing so, pFedLT can capture the diversity of data distribution among clients, therefore, can generalize well even when the data distributions among clients exhibit high statistical heterogeneity. We conduct extensive experiments on synthetic and real-world datasets (MNIST, Fashion_MNIST, CIFAR-10, and Office+Caltech10) under different Non-IID settings. Experimental results demonstrate that pFedLT significantly improves the model accuracy by up to 11.67% and reduces the communication costs compared with state-of-the-art approaches.


A Novel Neural Ensemble Architecture for On-the-fly Classification of Evolving Text Streams

Abstract: We study on-the-fly classification of evolving text streams in which the relation between the input data and target labels changes over time—i.e., “concept drift.” These variations decrease the model’s performance, as predictions become less accurate over time and they necessitate a more adaptable system. While most studies focus on concept drift detection and handling with ensemble approaches, the application of neural models in this area is relatively less studied. We introduce Adaptive Neural Ensemble Network ( AdaNEN ), a novel ensemble-based neural approach, capable of handling concept drift in data streams. With our novel architecture, we address some of the problems neural models face when exploited for online adaptive learning environments. Most current studies address concept drift detection and handling in numerical streams, and the evolving text stream classification remains relatively unexplored. We hypothesize that the lack of public and large-scale experimental data could be one reason. To this end, we propose a method based on an existing approach for generating evolving text streams by introducing various types of concept drifts to real-world text datasets. We provide an extensive evaluation of our proposed approach using 12 state-of-the-art baselines and 13 datasets. We first evaluate concept drift handling capability of AdaNEN and the baseline models on evolving numerical streams; this aims to demonstrate the concept drift handling capabilities of our method on a general spectrum and motivate its use in evolving text streams. The models are then evaluated in evolving text stream classification. Our experimental results show that AdaNEN consistently outperforms the existing approaches in terms of predictive performance with conservative efficiency.


Leveraging Dual Gloss Encoders in Chinese Biomedical Entity Linking

Abstract: Entity linking is the task of assigning a unique identity to named entities mentioned in a text, a sort of word sense disambiguation that focuses on automatically determining a pre-defined sense for a target entity to be disambiguated. This study proposes the DGE (Dual Gloss Encoders) model for Chinese entity linking in the biomedical domain. We separately model a dual encoder architecture, comprising a context-aware gloss encoder and a lexical gloss encoder, for contextualized embedding representations. DGE are then jointly optimized to assign the nearest gloss with the highest score for target entity disambiguation. The experimental datasets consist of a total of 10,218 sentences that were manually annotated with glosses defined in the BabelNet 5.0 across 40 distinct biomedical entities. Experimental results show that the DGE model achieved an F1-score of 97.81, outperforming other existing methods. A series of model analyses indicate that the proposed approach is effective for Chinese biomedical entity linking.


Improving the Detection of Multilingual South African Abusive Language via Skip-gram Using Joint Multilevel Domain Adaptation

Abstract: The distinctiveness and sparsity of low-resource multilingual South African abusive language necessitate the development of a novel solution to automatically detect different classes of abusive language instances using machine learning. Skip-gram has been used to address sparsity in machine learning classification problems but is inadequate in detecting South African abusive language due to the considerable amount of rare features and class imbalance. Joint Domain Adaptation has been used to enlarge features of a low-resource target domain for improved classification outcomes by jointly learning from the target domain and large-resource source domain. This article, therefore, builds a Skip-gram model based on Joint Domain Adaptation to improve the detection of multilingual South African abusive language. Contrary to the existing Joint Domain Adaptation approaches, a Joint Multilevel Domain Adaptation model involving adaptation of monolingual source domain instances and multilingual target domain instances with high frequency of rare features was executed at the first level and adaptation of target-domain features and first-level features at the next level. Both surface-level and embedding word features were used to evaluate the proposed model. In the evaluation of surface-level features, the Joint Multilevel Domain Adaptation model outperformed the state-of-the-art models with accuracy of 0.92 and F1-score of 0.68. In the evaluation of embedding features, the proposed model outperformed the state-of-the-art models with accuracy of 0.88 and F1-score of 0.64. The Joint Multilevel Domain Adaptation model significantly improved the average information gain of the rare features in different language categories and reduced class imbalance.


Optimal Hadamard Gate Count for Clifford+ <i>T</i> Synthesis of Pauli Rotations Sequences

Abstract: The Clifford+ T gate set is commonly used to perform universal quantum computation. In such setup the T gate is typically much more expensive to implement in a fault-tolerant way than Clifford gates. To improve the feasibility of fault-tolerant quantum computing it is then crucial to minimize the number of T gates. Many algorithms, yielding effective results, have been designed to address this problem. It has been demonstrated that performing a pre-processing step consisting of reducing the number of Hadamard gates in the circuit can help to exploit the full potential of these algorithms and thereby lead to a substantial T -count reduction. Moreover, minimizing the number of Hadamard gates also restrains the number of additional qubits and operations resulting from the gadgetization of Hadamard gates, a procedure used by some compilers to further reduce the number of T gates. In this work we tackle the Hadamard gate reduction problem, and propose an algorithm for synthesizing a sequence of π /4 Pauli rotations with a minimal number of Hadamard gates. Based on this result, we present an algorithm which optimally minimizes the number of Hadamard gates lying between the first and the last T gate of the circuit.


NNC-GCN: Neighbours-to-Neighbours Contrastive Graph Convolutional Network for Semi-Supervised Classification

Abstract: Contrastive learning (CL) is a popular learning paradigm in deep learning, which uses contrastive principle to learn low-dimensional embeddings, and has been applied in Graph Neural Networks (GNNs) successfully. Existing works of contrastive multi-view GNNs usually focus on point-to-point contrastive learning strategies. However, they neglect the local information in neighbors, which brings isolated positive samples. The quality of selected positive samples is hard to evaluate, and these samples may lead to invalid contrastiveness. Therefore, we propose a simple and efficient neighbors-to-neighbors contrastive graph neural network (NNC-GCN), which constructs a consistent multi-view by using the topologies of original input graphs. Moreover, we raise a new learning problem of unlabeled data base on these constructed multi-view topologies and propose a loss function NNC-InfoNCE to guide its learning process. The NNC-InfoNCE is an improved version of InfoNCE, which can be adapted to neighborhood-level contrast learning. Specifically, the neighborhoods and the remaining nodes of the selected anchor are weighted and treated as positive and negative sample sets. The experimental results show that our method is effective on public benchmark datasets.


Ibn-Ginni: An Improved Morphological Analyzer for Arabic

Abstract: Arabic is a morphologically rich language, which means that the Arabic language has a complicated system of word formation and structure. The affixes in the Arabic language (i.e., prefixes and suffixes) can be added to root words to generate different meanings and grammatical functions. These affixes can indicate aspects such as tense, gender, number, case, person, and more. In addition, the meaning and function of words can be modified in Arabic using an internal structure known as morphological patterns. Computational morphological analyzers of Arabic are vital to developing Arabic language processing toolkits. In this article, we introduce a new morphological analyzer (Ibn-Ginni) that inherits the speed and quality of the Buckwalter Arabic Morphological Analyzer (BAMA). The BAMA has poor coverage of the classical Arabic language. Hence, the coverage of classical Arabic is improved by using the Alkhalil analyzer. Although it is slow, it was used to generate a huge number of solutions for 3 million unique Arabic words collected from different resources. These word form-based solutions were converted to stem-based solutions, refined manually, and added to the database of BAMA, resulting in substantial improvements in the quality of the analysis. Hence, Ibn-Ginni is a hybrid system between BAMA and Alkhalil analyzers and may be considered an efficient large-scale analyzer. The Ibn-Ginni analyzer analyzed 0.6 million more words than the BAMA analyzer. Therefore, our analyzer significantly improves the coverage of the Arabic language. Besides, the Ibn-Ginni analyzer is high speed at providing solutions, the average time to analyze a word is 0.3 ms. Using a corpus designed for benchmarking Arabic morphological analyzers, our analyzer was able to find all solutions for 72.72% of the words. Moreover, the analyzer did not provide all possible morphological solutions for 24.24% of the words. The analyzer and its morphological database are publicly available on GitHub. 1


Bridging Formal Methods and Machine Learning with Model Checking and Global Optimisation

Abstract: Formal methods and machine learning are two research fields with drastically different foundations and philosophies. Formal methods utilise mathematically rigorous techniques for software and hardware systems' specification, development and verification. Machine learning focuses on pragmatic approaches to gradually improve a parameterised model by observing a training data set. While historically, the two fields lack communication, this trend has changed in the past few years with an outburst of research interest in the robustness verification of neural networks. This paper will briefly review these works, and focus on the urgent need for broader and more in-depth communication between the two fields, with the ultimate goal of developing learning-enabled systems with excellent performance and acceptable safety and security. We present a specification language, MLS2, and show that it can express a set of known safety and security properties, including generalisation, uncertainty, robustness, data poisoning, backdoor, model stealing, membership inference, model inversion, interpretability, and fairness. To verify MLS2 properties, we promote the global optimisation-based methods, which have provable guarantees on the convergence to the optimal solution. Many of them have theoretical bounds on the gap between current solutions and the optimal solution.


CLUE: Contrastive language-guided learning for referring video object segmentation

Abstract: Referring video object segmentation (R-VOS), the task of separating the object described by a natural language query from the video frames, has become increasingly critical with recent advances in multi-modal understanding. Existing approaches are mainly visual-dominant in both representation-learning and decision-making process, and are less sensitive to fine-grained clues in text description. To address this, we propose a language-guided contrastive learning and data augmentation framework to enhance the model sensitivity to the fine-grained textual clues (i.e., color, location, subject) in the text that relate heavily to the video information. By substituting key information of the original sentences and paraphrasing them with a text-based generation model, our approach conducts contrastive learning through automatically building diverse and fluent contrastive samples. We further enhance the multi-modal alignment with a sparse attention mechanism, which can find the most relevant video information by optimal transport. Experiments on a large-scale R-VOS benchmark show that our method significantly improves strong Transformer-based baselines, and further analysis demonstrates the better ability of our model in identifying textual semantics.


Multi-supervision transformer combining bounding box and mask for data-limited pose estimation

Abstract: In scenarios where acquiring accurate annotated pose data is difficult, obtaining sufficient training samples becomes a challenge. Given these data constraints, it is challenging to provide optimal end-to-end supervision with limited pose information. To address this limitation, we propose a novel approach, named MSPose, which combines diverse data sources from human instances to provide multi-level supervision in human pose estimation. This approach mitigates the impact of scarce training data and enhances pose estimation precision by synergistically incorporating human bounding box and mask information. Our proposed method achieves multi-level model activation for the human body region, structure, and joints. This hierarchical pixel activation is facilitated by integrating human bounding box and mask signals as supplementary supervision during training. Importantly, we exclude auxiliary branches to ensure complete freedom during inference. Our experimental validation on a quarter of the COCO and MPII datasets demonstrates the competitiveness of our method compared to state-of-the-art methods. Compared to TokenPose-T, our MSPose-T achieves a 6.1 points improvement in average precision and a corresponding 3.1 points increase on the full COCO validation set. On the MPII dataset, MSPose achieves the highest Mean score of 1.9 points. Our approach achieves these improvements while maintaining consistent parameters and GFLOPs, highlighting the advantage of our approach.


FAUC-S: Deep AUC maximization by focusing on hard samples

Abstract: Deep AUC maximization (DAM) is a popular method to deal with complex imbalanced classification problems. It learns a deep neural network by minimizing a surrogate AUC loss. The mostly used surrogates are the pairwise square loss and its variants. However, they are usually used in regression problems and are sensitive to noise samples. The easy samples suffer larger losses than those without correct classification, which is unreasonable in optimization problems. In this paper, we propose a focus AUC loss based on samples (FAUC-S) by constructing a differentiable weight function that identifies hard and easy samples, which makes easy samples have small losses, and hard samples have large losses. It is more reasonable than the traditional AUC loss while having the same advantages on large-scale datasets. In experiments, we use an end-to-end compositional DAM to investigate the effect of different weight functions on the AUC loss and compare it with other methods. Experimental results on several benchmark datasets demonstrate that FAUC-S achieves superior performance in terms of AUC compared to existing state-of-the-art methods.


Three-way evidence theory-based density peak clustering with the principle of justifiable granularity

Abstract: Clustering by fast search and find of density peaks (DPC) is an effective clustering approach that can find all the cluster centers at once with just one parameter and without iterative processing. However, the cutoff distance, a key parameter of density measurement in the DPC approach, affects the quality of the final clustering results. Its selection relies on experimental experience and lacks of a semantic explanation. Furthermore, the allocation strategy of the traditional DPC approach may cause several points to be assigned incorrectly, leading to subsequent points being assigned incorrectly and ultimately forming continuous allocation errors. To overcome the deficiencies, this paper proposes a novel three-way evidence theory-based density peak clustering with the principle of justifiable granularity (3 W-PEDP). First, the computation of the cutoff distance is converted into the search for nearest neighbors. From the perspective of granular computing, 3 W-PEDP transforms the neighbor selection issue into the construction of justifiable granularity. And the optimal neighbors can be achieved with the construction of coverage and specificity criteria. Second, inspired by three-way clustering, we adopt a two-stage method for sample allocation. On the one hand, for core point allocation, a two-layer nearest neighbor is constructed based on the achieved optimal neighbors. On the other hand, we designed a new evidence mass function to guide us in assigning the remaining points. In this novel evidence mass function, not only the labels of the assigned samples are considered, but also the information of the neighborhoods around the unassigned samples is fused. Finally, we assess the effectiveness of 3 W-PEDP on numerous public synthetic datasets and UCI real-world datasets. Then, detail comparing results with several popular clustering methods are presented. In addition, experimental studies verify the effectiveness of constructing justifiable granularity in selecting the optimal neighbors. The experimental results demonstrate 3 W-PEDP has good adaptability and robustness, which can achieve better clustering performance. Our source code is available at https://github.com/Luyangabc/3W-PEDP.


Lightweight image super-resolution for IoT devices using deep residual feature distillation network

Abstract: The 5th industrial revolution is characterized by an extensive interconnection of embedded devices, which offer a range of services, including the monitoring of their environments. Images captured from remote cameras require enhancements for effective analysis. Despite recent progress in single-image super-resolution techniques by yielding impressive results through deep convolutional neural networks, the complexity of these advanced models renders them impractical for use on miniaturized Internet of Things (IoT) devices, primarily due to their limited computational capabilities and memory constraints. Furthermore, the rapid evolution of IoT devices necessitates efficient image super-resolution techniques, while existing advanced methods, based on deep convolutional neural networks, are too resource-intensive for these devices, and this gap highlights the need for a more suitable solution. In this study, we introduce a lightweight, efficient super-resolution model specifically designed for IoT devices. This model incorporates a novel deep residual feature distillation block (DRFDB), which leverages a depthwise-separable convolution block (DCB) for effective feature extraction. The focus is on reducing computational and memory demands without compromising on image quality. The proposed DCB extracts coarse features from given input features as calculation units, using two operations, depthwise and pointwise convolutions. These two operations are able to significantly reduce the number of parameters and floating-point operations while maintaining a PSNR value higher than the 90% threshold. We modify the proposed DCB and introduce a multi-kernel depthwise-separable convolution block (MKDCB) to fine-tune the model. The experiments, conduct on various standard datasets such as DIV2K, Set5, Set14, Urban100, and Manga109 by demonstrating that our model significantly outperforms existing methods in terms of both image quality and computational efficiency. The model shows improved performance metrics like PSNR, while requiring fewer parameters and less memory usage, making it highly suitable for IoT applications. This study presents a breakthrough in super-resolution for IoT devices, balancing high-quality image reconstruction with the limited resources of these devices.


Class incremental learning via dynamic regeneration with task-adaptive distillation

Abstract: Class Incremental Learning (CIL) is a paradigm that excels in efficiently training models on an expanding set of classes, while preserving performance in classes learned earlier. In the context of smart city development, CIL plays a crucial role in addressing the need for Networking Systems of Artificial Intelligence (NSAI) to continuously adapt to ever-evolving data. In recent years, the incremental network structure widely adopted in CIL suffers from poor knowledge distillation and rapid model parameter growth. In this paper, we introduce a novel strategy named dynamic regeneration with task-adaptive distillation (DRTAD), which dynamically adapts to new tasks, and sustains good performance even as the class set continues to grow. DRTAD adopts a two-stage training strategy: dynamic regeneration and dynamic retention. During dynamic regeneration, DRTAD enhances feature representation by creating a new feature extraction module that extracts features from new classes, while also utilizing features from previously learned classes. Additionally, DRTAD introduces task-adaptive distillation to improve the poor knowledge distillation, further mitigating catastrophic forgetting. During the dynamic retention phase, DRTAD achieves a higher pruning rate through RM operation. Comprehensive experiments on CIFAR-100 and ImageNet-100 datasets demonstrate DRTAD’s superior performance compared to existing CIL methods. Notably, in the CIFAR100-B50 5 steps incremental setting, DRTAD increases the last-phase accuracy from 65.51% to 70.54% (＋5.03%), while maintaining fewer parameters (−11.1%). Similarly, in the ImageNet100-B50 10 steps setting, the last-phase accuracy rises from 70.04% to 72.50% (＋2.46%). These results indicate DRTAD’s efficacy in mitigating catastrophic forgetting in incremental structure.


Multiple integration model for single-source domain generalizable person re-identification

Abstract: Domain generalizable (DG) person re-identification (re-ID) aims to train a model on labeled source domains which can perform well on invisible target domains. Because of the distribution shifts between different domains, it is a challenging task. Existing methods address this challenge by using multiple source domains to train a model which requires more data, manual labor, and computation. In contrast, we pay attention to the single-source DG re-ID task, that is, only one source domain data is accessible for training. However, due to the limited availability of training data, this task is more difficult. In this paper, a novel MulTiple Integration (MTI) model is introduced for single-source DG person re-ID. By integrating multiple reliable perturbations, the generalization performance can be improved. Specifically, MTI model contains two types of integration modules, one is shallow-level compensation (SLC) and the other is deep-level integration (DLI). For SLC, according to the idea of continual learning, the shallow-level information of the ImageNet pre-trained ResNet-50 branch is introduced and fused with the shallow-level information of our backbone network. In this way, massive information in ImageNet can be used to prevent the disastrous forgetting of the pre-trained information, and information compensation can be provided for backbone network. Additionally, we propose a hybrid integrated normalization layer to fuse information and improve the model’s generalization performance. For DLI, a wave transformer block is introduced in the deep layer of the backbone, which can integrate the information of a batch images and contain reliable disturbance, so that the robustness of the model can be promoted. Extensive experimental results demonstrate the superiority of our model.


TMBL: Transformer-based multimodal binding learning model for multimodal sentiment analysis

Abstract: Multimodal emotion analysis is an important endeavor in human–computer interaction research, as it enables the accurate identification of an individual’s emotional state by simultaneously analyzing text, video, and sound features. Although current emotion recognition algorithms have performed well using multimodal fusion strategies, two key challenges remain. The first challenge is the efficient extraction of modality-invariant and modality-specific features prior to fusion, which requires deep feature interactions between the different modalities. The second challenge concerns the ability to distinguish high-level semantic relations between modality features. To address these issues, we propose a new modality-binding learning framework and redesign the internal structure of the transformer model. Our proposed modality binding learning model addresses the first challenge by incorporating bimodal and trimodal binding mechanisms. These mechanisms handle modality-specific and modality-invariant features, respectively, and facilitate cross-modality interactions. Furthermore, we enhance feature interactions by introducing fine-grained convolution modules in the feedforward and attention layers of the transformer structure. To address the second issue, we introduce CLS and PE feature vectors for modality-invariant and modality-specific features, respectively. We use similarity loss and dissimilarity loss to support model convergence. Experiments on the widely used MOSI and MOSEI datasets show that our proposed method outperforms state-of-the-art multimodal sentiment classification approaches, confirming its effectiveness and superiority. The source code can be found at https://github.com/JackAILab/TMBL.


Mobility and dependency-aware task offloading for intelligent assisted driving in vehicular edge computing networks

Abstract: Intelligent assisted driving is an important application in vehicular edge computing networks (VECNs). In the intelligent transportation system (ITS), a group of moving vehicle users need to be coordinated to complete complex vehicular applications. A number of dependent, latency-sensitive, and computation-intensive tasks are generated. However, the existing works have given less consideration to the dependencies among both vehicle users and the subtasks in vehicle, which makes it a huge challenge to complete tasks timely. When interdependent tasks come from different vehicle users, a special task preparation time is needed, which can disrupt the ongoing task processing. Furthermore, the high mobility of vehicles directly affects the data transmission rate. To address the mentioned challenges, we design an efficient mobility and dependency-aware task offloading strategy in VECNs. The objective is to minimize both the overall system task completion delay and the economic cost. We take into account the real-time locations and task preparation time of vehicle users. Additionally, we propose a multi-decision-making offloading algorithm (MDOA) that primarily analyzes the processing priorities for both vehicle users and subtasks. In order to integrate practical applications, the financial expenses of vehicle users are also considered as an indispensable part. As a result, we propose an efficient two-step task offloading algorithm. Through numerous simulation examples, we demonstrate the efficiency and high performance of the proposed task offloading strategies in VECNs when compared to existing algorithms.


Neural ordinary differential equation for irregular human motion prediction

Abstract: Human motion prediction often assumes that the input sequence is of fixed frame rates. However, in real-world applications, the motion capture system may work unstably sometimes and miss some frames, which leads to inferior performance. To solve this problem, this paper leverages neural Ordinary Differential Equations and proposes a human Motion Prediction method named MP-ODE to handle irregular-time human pose series. First, a Difference Operator and a Positional Encoding are proposed to explicitly provide the kinematic and time information for the model. Second, we construct the encoder–decoder model with ODE-GRU unit, which enables us to learn continuous-time dynamics of human motion. Third, a Quaternion Loss transforms exponential maps to quaternion to train MP-ODE. The Quaternion Loss can avoid the discontinuities and singularities of exponential maps, boosting the convergence of the model. Comprehensive experiments on Human3.6 m and CMU-Mocap datasets demonstrate that the proposed MP-ODE achieves promising performance in both normal and irregular-time conditions.


Energy-efficient resource allocation in cloud infrastructure using L3F-MGA and E-ANFIS

Abstract: In industry along with academia, Task Scheduling (TS) along with Resources Allocation (RA) remains a challenging issue even though lots of work was carried out in the Cloud Computing (CC) field. CC uses massive virtualized Data Centers (DC). Still, it took an enormous quantity of energy on account of the constant demand by users for resources in addition to inefficient RA techniques. The higher Energy Consumption (EC) because of improper RA inside DC can be overcome using energy-aware RA techniques. Therefore, this paper proposes a Levy flight Firefly Modified Genetic Algorithm (L3F-MGA) as well as an Entropy-based Adaptive Neuros-Fuzzy Inferences System (E-ANFIS) for energy-efficient Resources Allocation (EE-RA) in cloud structure that manages the user demands and also lessens the EC of Cloud Data Center (CDC). In the proposed work, centred on task features together with cloud resource features, the input tasks are arranged using L3F-MGA. Conversely, the Virtual Machine (VM) status is identified. After that, centred on the task feature and VM status details, the E-ANFIS allows the active VM to complete the tasks effectively. The experiments compare the proposed method's performance for effective RA with other associated works. The proposed RA technique shows EC by 11 % lesser than the existing CM-GA. The Processing Time of E-ANFIS is 8 % lesser than CM-GA, 14 % less than ANFIS, 20 % less than GA, and 16 % less than PSO.


A survey on personalized itinerary recommendation: From optimisation to deep learning

Abstract: The tourism industry is a significant contributor to the global economy, responsible for generating nearly 10% of the world’s GDP and employing around 9% of the global workforce. A crucial aspect of this industry is personalised itinerary recommendation, where visitors’ preferences and constraints are taken into account to create customised travel plans. This task involves selecting the best points of interests (POIs) for visitors in various cities and then schedule these POIs as an itinerary considering numerous constraints. However, due to the varied ways in which researchers have defined the itinerary recommendations, it can be challenging for new researchers to locate up-to-date literature on the topic. As a result, this paper aims to review existing research in this area and provide a taxonomy of the works based on problem formulations, proposed techniques, constraints, and features used. We divide the study into two directions: user satisfaction and provider satisfaction, where user satisfaction is derived non–personalised and personalised POI/ Itinerary recommendations. We also discuss the data sources, techniques ranging from optimization approaches to deep learning and evaluation methodologies commonly used in this field. Finally, we highlight the importance of personalised itinerary recommendation and identify areas for future research to address the current challenges.


Prompted representation joint contrastive learning for aspect-based sentiment analysis

Abstract: As a fine-grained and challenging subtask in the natural language processing (NLP) community, aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarity towards a given aspect term. In previous ABSA research, most works utilized the pre-trained language model (PLM) as the backbone of their proposed methods, without any specific task-related instructions. Besides, some works focused on learning the dependency information or the external knowledge-enhanced dependency information separately, which lacked the exploitation of the mutual interaction between the normal dependency and knowledge-enhanced dependency. Therefore, we propose a novel ABSA method namely prompted representation joint contrastive learning enhanced graph convolutional networks (PRCL-GCN) to strengthen the robustness of the ABSA model. Specifically, to achieve the task-oriented contextual representation, we design the task-specific prompt template to guide the fine-tuning process of PLM in the ABSA task. And a biaffine attention mechanism is employed to further extract the essential sentiment feature from the prompted representation. Moreover, we introduce the syntax dependency graph as prior knowledge, and construct an affective syntactic dependency graph by injecting the affective knowledge from SenticNet into the graph. Then, we utilize the multi-layer GCNs to process the above two syntactic graphs independently, which aims to learn multi-granularity syntactic features. Subsequently, a novel designed attention variant is leveraged to integrate these syntax features with the guided contextual representation, separately. Eventually, through designing a Kullback–Leibler divergence-based contrastive learning to encourage the model’s learning, we improve the model’s accuracy in modeling contextual representation by integrating the designed dual-ways information. Extensive experiments are conducted on five benchmark datasets, and the outstanding experiment results validate the effectiveness of our proposed model.


An Efficient GCN Accelerator Based on Workload Reorganization and Feature Reduction

Abstract: The irregular adjacency matrix and the mismatched computation patterns of Aggregation and Combination phases make Graph Neural Networks (GNNs) challenging to compute efficiently. This paper proposes a software and hardware co-design system to reduce computational latency and memory access based on workload reorganization and feature reduction. In software, the adjacency matrix is preprocessed, and the workload in both feature and node dimensions is concentrated to optimize memory access and hardware utilization. The interlayer nodes are analyzed using Principal Component Analysis (PCA) to explore the minimum feature vector length based on information redundancy, and a unique weight initialization is utilized for retraining to trim the feature vector to the minimum length. In hardware, an efficient GCN accelerator is designed to fully support the reorganized workload by reconfigurable output node computation. The hardware accelerator is implemented using 28-nm CMOS technology. It achieves 3.3 TOPS peak throughput and 2.6 TOPS/W energy efficiency. Compared with HyGCN, this result shows that the proposed method can improve the overall performance by 5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> with a negligible accuracy loss of less than 0.5%.


LABANet: A Lightweight Asymmetrical Bottleneck and Attention-Based Network for Cloud Detection

Abstract: Recently, on-board satellite cloud detection has attracted extensive interest due to its ability to reduce invalid data, saving satellite storage and downlink resources. As the demand for accuracy has increased, deep neural networks with encoder-decoder architectures have been widely utilized for cloud detection. Most networks mainly focus on extracting contextual features and fusing more information derived from multilevel features to achieve high accuracy. However, convolutional layers with contextual feature extraction abilities in networks always contain many computations and parameters, making them difficult to deploy in processors with limited computing and memory resources. Additionally, fusing features acquired from different levels easily introduces redundant noise and decreases accuracy. To address these issues, a novel lightweight asymmetrical bottleneck and attention-based network (LABANet) is proposed. In the encoder, a multiscale context-based asymmetrical bottleneck (MCAB) is designed to extract more contextual features in an efficient manner. The numbers of computations and parameters required by the bottleneck can be effectively reduced via asymmetric channel compression and grouping for different pointwise convolutions. Multiscale contextual features can be extracted by dilated depthwise asymmetric convolution. In the decoder, a spatial-channel cascade attention (SCCA) module is proposed to fuse multilevel features with less redundant noise by highlighting spatial and channel correlation information. Experimental results obtained on public datasets demonstrate that the proposed LABANet can exceed 95% overall accuracy (OA) with a computational cost of 1.20 giga-floating-point operations (GFLOPs) and a parameter count of 1.27×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">6</sup> ; thus, it achieves competitive accuracy with fewer computations and parameters than other baseline cloud detection networks.


Differential private knowledge trading in vehicular federated learning using contract theory

Abstract: In vehicular differentially private federated learning, the personalized differential privacy mechanism allows vehicles to customize their local privacy budget. However, it leads to uneven utility across datasets. Therefore, model owners must consider the unevenness in utility within the incentive mechanisms to enhance learning performance. Due to limitations in system resources, servers in the Internet of Vehicles should prioritize the selection of high-utility datasets before initiating the learning process. Since existing utility evaluation schemes are costly and inefficient, we propose a knowledge trading framework based on prior utility evaluation and contract theory. Specifically, we first derive a utility evaluation function that captures the functional relationship between utility and privacy elements, such as privacy cost, sensitivity, and learning rounds. These elements are shared as hyperparameters among vehicles and servers for utility evaluation. In the presence of information asymmetry, we utilize the self-revealing property of contract theory to mitigate dishonest behavior. The learning process is modeled as an optimization function that incorporates utility and privacy costs. The complexity is further reduced by utilizing the monotonicity of the utility function. Experimental results based on multiple models and datasets illustrate that our scheme outperforms existing schemes in terms of convergence and accuracy.


Hybrid cross-modal interaction learning for multimodal sentiment analysis

Abstract: Multimodal sentiment analysis (MSA) predicts the sentiment polarity of an unlabeled utterance that carries multiple modalities, such as text, vision and audio, by analyzing labeled utterances. Existing fusion methods mainly focus on establishing the relationship of characteristics among different modalities to enhance their emotion recognition abilities. However, they always ignore the all-round interaction between different modalities, especially the cross-modal interaction, which is critical to the sentiment decision of multimodal data. To address these issues, we propose a novel hybrid cross-modal interaction learning (HCIL) framework for hybrid learning of intra-modal, inter-modal, interactive-modal and cross-modal interactions, with which the model can fully utilize the sentiment information of multimodalities and enhance the sentiment assistance between modalities. Specifically, we propose two core substructures to learn discriminative multimodal features. One is the comparative learning interaction structure that can track the class dynamics in the intra-modal, reduce the modal gap in the inter-modal and establish emotional communication in the interactive-modal; the other is the cross-modal prediction structure, which can build the sentiment relationship between cross-modal pairs, especially exploring the auxiliary sentiment effect of audio on the vision and text. Furthermore, we adopt a hierarchical feature fusion structure to generate the multimodal feature for the final sentiment prediction. Extensive experiments on three benchmark datasets showed that our HCIL approach can obtain significant performance on the MSA task and that the design of a cross-modal interaction structure can directly promote the improvement of sentiment classification performance.


Towards Unsupervised Referring Expression Comprehension with Visual Semantic Parsing

Abstract: Referring Expression Comprehension (REC) is a task that involves grounding a specific object in an image based on a given referring query in the form of bounding boxes. Existing fully-supervised or weakly-supervised REC methods rely on either manually annotated regional coordinates or query texts. In this paper, we propose an unsupervised training paradigm for the REC task that does not require any manual annotated data. Specifically, we introduce a Visual-Semantic-Parsing-based Unsupervised Referring Expression Comprehension framework (VUREC), which leverages a Visual Semantic Parser (VSP) as its core module to recognize the rich semantic information in images and construct pseudo-region-query pairs as the training supervision for REC. The VSP comprises a Scene Graph Parser (SGP) and a Visual Concept Detector (VCD) that can detect the locations, categories, attributes of objects, and visual relationships among them in images. Furthermore, we present a Referring Expression Reasoning (RER) model that utilizes a Multi-Modal Cascade Attention Decoder (MCAD) for fine-grained multi-modality fusion and regresses the four coordinates of the referential object directly. The experimental results on three benchmark datasets of Refcoco, Refcoco+ and Refcocog demonstrate the effectiveness of our proposed method.


PU-Detector: A PU Learning-based Framework for Real Money Trading Detection in MMORPG

Abstract: Massive multiplayer online role-playing games (MMORPG) have been becoming one of the most popular and exciting online games. In recent years, a cheating phenomenon called real money trading (RMT) has arisen and damaged the fantasy world in many ways. RMT is the sale of in-game items, currency, or even characters to earn real money, breaking the balance of the game economy ecosystem and damaging the game experience. Therefore, some studies have emerged to address the problem of RMT detection. However, they cannot well handle the label uncertainty problem in practice, where there are only labeled RMT samples (positive samples) and unlabeled samples, which could either be RMT samples or normal transactions (negative samples). Meanwhile, the trading relationship between RMTers is modeled in a simple way, leading to some normal transactions being falsely classified as RMT. In this article, we propose PU-Detector, a novel framework based on PU learning (learning from positive and unlabeled data) for RMT detection, considering the fact that there are only labeled RMT samples and other unlabeled transactions. We first automatically estimate the likelihood of one transaction being RMT by developing an improved PU learning method and proposing an assessment rule. Sequentially, we use the estimated likelihood as edge weight to construct a trading graph to learn trader representation. Then, with the trader representations and basic trading features, we detect RMT samples by the improved PU learning method. PU-Detector is evaluated on a large-scale real world dataset consisting of 33,809,956 transaction logs generated by 43,217 unique players. Compared with other approaches, it achieves the state-of-the-art performance and demonstrates its advantages in detecting underlying RMT samples.


Explanation Guided Knowledge Distillation for Pre-trained Language Model Compression

Abstract: Knowledge distillation is widely used in pre-trained language model compression, which can transfer knowledge from a cumbersome model to a lightweight one. Though knowledge distillation based model compression has achieved promising performance, we observe that explanations between the teacher model and the student model are not consistent. We argue that the student model should study not only the predictions of the teacher model but also the internal reasoning process. To this end, we propose Explanation Guided Knowledge Distillation (EGKD) in this article, which utilizes explanations to represent the thinking process and improve knowledge distillation. To obtain explanations in our distillation framework, we select three typical explanation methods rooted in different mechanisms, namely gradient-based , perturbation-based , and feature selection methods. Then, to improve computational efficiency, we propose different optimization strategies to utilize the explanations obtained by these three different explanation methods, which could provide the student model with better learning guidance. Experimental results on GLUE demonstrate that leveraging explanations can improve the performance of the student model. Moreover, our EGKD could also be applied to model compression with different architectures.


DSISA: A New Neural Machine Translation Combining Dependency Weight and Neighbors

Abstract: Most of the previous neural machine translations (NMT) rely on parallel corpus. Integrating explicitly prior syntactic structure information can improve the neural machine translation. In this article, we propose a Syntax Induced Self-Attention (SISA) which explores the influence of dependence relation between words through the attention mechanism and fine-tunes the attention allocation of the sentence through the obtained dependency weight. We present a new model, Double Syntax Induced Self-Attention (DSISA), which fuses the features extracted by SISA and a compact convolution neural network (CNN). SISA can alleviate long dependency in sentence, while CNN captures the limited context based on neighbors. DSISA utilizes two different neural networks to extract different features for richer semantic representation and replaces the first layer of Transformer encoder. DSISA not only makes use of the global feature of tokens in sentences but also the local feature formed with adjacent tokens. Finally, we perform simulation experiments that verify the performance of the new model on standard corpora.


Active contrastive coding reducing label effort for sensor-based human activity recognition

Abstract: Human activity recognition (HAR) plays a crucial role in remotely monitoring the health of the elderly. Human annotation is time-consuming and expensive, especially for abstract sensor data. Contrastive learning can extract robust features from weakly annotated data to promote the development of sensor-based HAR. However, current research mainly focuses on the exploration of data augmentation methods and pre-trained models, disregarding the impact of data quality on label effort for fine-tuning. This paper proposes a novel active contrastive coding model that focuses on using an active query strategy to evenly select small, high-quality samples in downstream tasks to complete the update of the pre-trained model. The proposed uncertainty-based balanced query strategy mines the most indistinguishable hard samples according to the data posterior probability in the unlabeled sample pool, and imposes class balance constraints to ensure equilibrium in the labeled sample pool. Extensive experiments have shown that the proposed method consistently outperforms several state-of-the-art baselines on four mainstream HAR benchmark datasets (UCI, WISDM, MotionSense, and USCHAD). With approximately only 10% labeled samples, our method achieves impressive F1-scores of 98.54%, 99.34%, 98.46%, and 87.74%, respectively.


Evolving masked low-rank transformer for long text understanding

Abstract: Long sequence text processing is time-consuming owing to the ultra-large-scale self-attention computing. Recent advances demonstrate the attention in transformer can be accelerated by redundancy removal, and there are various sparse variants for attention in large sequences are proposed, which leads to state-of-the-art performance on language and vision task. Low-rank method achieve outstanding success in the field of efficient transformer. The dynamic token sparsification is efficiently time-saving and cost-saving, which can be easily extended to prune redundant spans and to yield semantic features. Evolutionary algorithm is attractive for selecting hyperparameter which is of significant importance in effectiveness. Motivated by these works, we propose an efficient transformers model, termed EMLT, to alleviate time and cost without sacrificing the accuracy. EMLT effectively combines strengths of Low-rank transformers, dynamic token sparsification and evolutionary algorithm to ulteriorly cut redundant token and meanwhile maintains the original precision, which can achieve a linear memory and time complexity. We compress transformer in three stages. Firstly, sliding window is validated as local attention to capture fine-grained dependency semantics. After that, low-rank approximation of attention matrix is applied as global attention to store long-range dependency semantics, and aggregated with local attention. On this basis, we consistently prune redundant token in accordance with importance score to further sparse the attention operation. Finally, Evolutionary algorithm is utilized to optimize the hyper-parameters of every layer. The results of comprehensive experiments and analysis show that our method can rival others on accuracy, and outperforms others on efficiency by a large margin in terms of the computational complexity.


Intensive vision-guided network for radiology report generation

Abstract: Abstract Objective. Automatic radiology report generation is booming due to its huge application potential for the healthcare industry. However, existing computer vision and natural language processing approaches to tackle this problem are limited in two aspects. First, when extracting image features, most of them neglect multi-view reasoning in vision and model single-view structure of medical images, such as space-view or channel-view. However, clinicians rely on multi-view imaging information for comprehensive judgment in daily clinical diagnosis. Second, when generating reports, they overlook context reasoning with multi-modal information and focus on pure textual optimization utilizing retrieval-based methods. We aim to address these two issues by proposing a model that better simulates clinicians perspectives and generates more accurate reports. Approach. Given the above limitation in feature extraction, we propose a globally-intensive attention (GIA) module in the medical image encoder to simulate and integrate multi-view vision perception. GIA aims to learn three types of vision perception: depth view, space view, and pixel view. On the other hand, to address the above problem in report generation, we explore how to involve multi-modal signals to generate precisely matched reports, i.e. how to integrate previously predicted words with region-aware visual content in next word prediction. Specifically, we design a visual knowledge-guided decoder (VKGD), which can adaptively consider how much the model needs to rely on visual information and previously predicted text to assist next word prediction. Hence, our final intensive vision-guided network framework includes a GIA-guided visual encoder and the VKGD. Main results. Experiments on two commonly-used datasets IU X-RAY and MIMIC-CXR demonstrate the superior ability of our method compared with other state-of-the-art approaches. Significance. Our model explores the potential of simulating clinicians perspectives and automatically generates more accurate reports, which promotes the exploration of medical automation and intelligence.


Strengthening Cooperative Consensus in Multi-Robot Confrontation

Abstract: Multi-agent reinforcement learning (MARL) has proven effective in training multi-robot confrontation, such as StarCraft and robot soccer games. However, the current joint action policies utilized in MARL have been unsuccessful in recognizing and preventing actions that often lead to failures on our side. This exacerbates the cooperation dilemma, ultimately resulting in our agents acting independently and being defeated individually by their opponents. To tackle this challenge, we propose a novel joint action policy, referred to as the consensus action policy (CAP). Specifically, CAP records the number of times each joint action has caused our side to fail in the past and computes a cooperation tendency, which is integrated with each agent’s Q -value and Nash bargaining solution to determine a joint action. The cooperation tendency promotes team cooperation by selecting joint actions that have a high tendency of cooperation and avoiding actions that may lead to team failure. Moreover, the proposed CAP policy can be extended to partially observable scenarios by combining it with Deep Q network or actor-critic–based methods. We conducted extensive experiments to compare the proposed method with seven existing joint action policies, including four commonly used methods and three state-of-the-art methods, in terms of episode rewards, winning rates, and other metrics. Our results demonstrate that this approach holds great promise for multi-robot confrontation scenarios.


TSAGNN: Temporal link predict method based on two stream adaptive graph neural network

Abstract: Temporal link prediction based on graph neural networks has become a hot spot in the field of complex networks. To solve the problems of the existing temporal link prediction methods based on graph neural networks do not consider the future time-domain features and spatial-domain features are limited used, this paper proposes a novel temporal link prediction method based on two streams adaptive graph neural networks. Firstly, the network topology features are extracted from the micro, meso, and middle perspectives. Combined with the adaptive mechanism of convolution and self-attention, the preprocessing of the feature extraction is more effective; Secondly, an extended bi-directional long short-term memory network is proposed, which uses graph convolution to process topological features, and recursively learns the state vectors of the target snapshot by using the future time-domain information and the past historical information; Thirdly, the location coding is replaced by the time-coding for the transformer mechanism, so that past information and future information can be learned from each other, and the time-domain information of the network can be further mined; Finally, a novel two-stream network framework is proposed, which combines the processing results of point features and edge features. The experimental results on 9 data sets show that the proposed method has a better prediction effect and better robustness than the classical graph neural network methods.


Digital mammogram based robust feature extraction and selection for effective breast cancer classification in earlier stage

Abstract: Breast cancer poses a significant health risk for women, demanding early detection to mitigate its mortality impact. Leveraging the power of Deep Learning (DL) in medical imaging, this paper introduces a hybrid model that integrates YOLOv7 and Half UNet for feature extraction. YOLOv7 identifies and localizes potential cancerous regions, while Half UNet focuses on extracting pertinent features with its encoder-decoder structure. The fusion of these discriminative features, coupled with feature selection via Coati Optimization, ensures a comprehensive and optimized dataset. The selected features then feed into the CatBoost classification algorithm, refining parameters iteratively for precise predictions and minimizing the loss function. Evaluation metrics, including precision, recall, specificity, and accuracy, demonstrate the model’s superior performance. Notably, the proposed model surpasses existing methods in early-stage breast cancer detection. Beyond numerical metrics, its significance lies in the potential to positively impact patient outcomes and increase survival rates. By amalgamating cutting-edge DL techniques, the model excels in identifying intricate patterns crucial for early cancer detection. The efficient fusion of YOLOv7 and Half UNet, coupled with feature optimization through Coati Optimization, sets this model apart. This research contributes to the evolving landscape of medical imaging and DL applications, emphasizing the potential for enhanced breast cancer diagnosis and improved patient prognoses.


Exploiting classifier inter-level features for efficient out-of-distribution detection

Abstract: Deep learning approaches have achieved state-of-the-art performance in a wide range of applications. Most often, however, it is falsely assumed that samples at inference follow a similar distribution as the training data. This assumption impairs models' ability to handle Out-of-Distribution (OOD) data during deployment. While several OOD detection approaches mostly focus on outputs of the last layer, we propose a novel mechanism that exploits features extracted from intermediate layers of a deep classifier. Specifically, we train an off-the-shelf auxiliary network using features of early layers to learn distinctive representations that improve OOD detection. The proposed network can be appended to any classification model without imposing any modification to its original architecture. Additionally, the mechanism does not require access to OOD data during training. We evaluate the performance of the mechanism on a variety of backbone architectures and datasets for near-OOD and far-OOD scenarios. The results demonstrate improvements in OOD detection compared to other state-of-the-art approaches. In particular, our proposed mechanism improves AUROC by 14.2% and 8.3% in comparison to the strong OOD baseline method, and by 3.2% and 3.9% in comparison to the second-best performing approach, on CIFAR-10 and CIFAR-100 datasets respectively.


Protocol choice and iteration for the free cornering

Abstract: We extend the free cornering of a symmetric monoidal category, a double categorical model of concurrent interaction, to support branching communication protocols and iterated communication protocols. We validate our constructions by showing that they inherit significant categorical structure from the free cornering, including that they form monoidal double categories. We also establish some elementary properties of the novel structure they contain. Further, we give a model of the free cornering in terms of strong functors and strong natural transformations, inspired by the literature on computational effects.


A no-reference image quality assessment model based on neighborhood component analysis and Gaussian process

Abstract: In this paper, a human visual system (HVS) based no-reference image quality assessment model (NR-IQA) is proposed. This is an objective NR-IQA model that uses perceptual features such as structure and orientation to compute the loss of naturalness in the image. An adaptive feature extraction process is modeled to capture the underlying significant features of the images along with the noise. Moreover, the optimal subset of features is derived using the neighborhood component analysis and further used as inputs to the Gaussian process for regression. The various experimentations are carried out on LIVE, TID2008, and TID2013 databases to test the effectiveness of the proposed approach. It is observed that the presented model exhibits a competitive performance in comparison with the existing IQA approaches. Moreover, the computation complexity and run-time show the effectuality of the proposed approach. Further, the experiments show that the predicted score matches with human perceptions. Thus, the proposed model is more accurate, less complex, independent of distortions, and well-suited for real-time applications.


Identification of the Original Images

Abstract: In social networks, people usually communicate online by posting and passing images, people will find that there are many similar or even visually identical images. When the images determine whether the published information is true or false, original images became crucial, and at the same time, deep learning excelled in image identification. Combining the original image identification and deep learning methods, this paper proposes the original image identification method based on the deep network models, aiming to use the deep network models to identify the original images between the original images and a bunch of variant images of the original images, and obtain satisfactory results. In this way, through the trained network models, the original images can be identified and can help the viewers to determine the correctness of the information. Experimental results verify the feasibility, effectiveness and security of the proposed method.


An efficient two-stage evolutionary algorithm for multi-robot task allocation in nuclear accident rescue scenario

Abstract: With the growing maturity of multi-robot system technology, its applications have expanded across various domains. This paper addresses the critical issue of task allocation in nuclear accident rescue scenario, which plays a pivotal role in the success of such operations. The problem is formulated as a multi-objective optimization problem, taking into account three key indicators: execution time, radiation accumulation, and waiting cost. To effectively tackle this problem, an two-stage evolutionary algorithm is proposed. Firstly, a solution encoding method and a crossover mutation method is devised tailored to the problem’s characteristics. Secondly, a two-stage search strategy is designed. In the first stage, a fixed population size and shift-based density estimation method are used to quickly converge the solution set to the Pareto front. The latter stage uses an infinite size population to find as many Pareto solutions as possible. Finally, a local search strategy is introduced to improve the quality of solution set. In the experimental section, our proposed method is compared with five state-of-the-art algorithms on nine instances of varying scales. Across five evaluation metrics, the proposed algorithm demonstrates competitive performance on all instances. These results underscore the efficacy and competitiveness of our approach in tackling the task allocation problem in multi-robot systems within nuclear accident rescue.


Multi-objective cost-aware bag-of-tasks scheduling optimization model for IoT applications running on heterogeneous fog environment

Abstract: Recently introduced by CISCO, the fog computing paradigm extends cloud-based computing services near the Internet of Things (IoT) devices to serve their applications, aiming to increase performance. This new computing paradigm tries to meet the requirements of emerging IoT applications, which mostly are geographically distributed and location-aware, low-cost, and delay-sensitive. One of the frequently applied applications is a bag-of-tasks (BoT) of IoT applications, each of which includes several independent tasks. However, inefficient task scheduling approaches for parallel executions may degrade the delivered quality of service (QoS) to users and also increase the providers’ total cost of ownership (TCO). The reliable execution commensurate with users’ service level agreement (SLA) and affordable services tailored to their budget at the same time is favorable for both users and fog service providers because the highly reliable execution is a reputational metric that increases the users’ trust and adherence. To this end, this paper formulates the task scheduling for the execution of BoT in IoT applications to a multi-objective optimization problem with execution time, costs and reliability perspectives. To measure total costs, both the execution cost and monetary cost models are presented. To measure the resource reliability, the new scheduling failure factor (SFF) metric is also presented, which is a kind of cost parameter. To solve this combinatorial NP-Hard problem, a multi-objective cost-aware discrete grey wolf optimization-based algorithm (MoDGWA) is proposed. To verify the performance of MoDGWA, diverse scenarios were conducted and the performance of the proposed algorithm was compared against other state-of-the-arts. The extensive simulation results prove the average dominance of MoDGWA, and it has 0.55 %, 7.28 %, 10.20 %, and 45.83 % reduction against other state-of-the-art in terms of makespan, ToC, TSFF, and the cost score respectively. Note that, the implementation of the MoDGWA is available on https://github.com/mirsaeid123/Mirsaeid.


Lens imaging opposition-based learning for differential evolution with cauchy perturbation

Abstract: Opposition-based learning (OBL) is an effective optimization strategy that enhances the performance of various global optimization algorithms. Among these algorithms, differential evolution (DE) based on OBL has received the attention of many researchers. However, OBL is a strategy with relative symmetry between the target point and the opposite point, which does not contribute significantly to the population in the later stage of the algorithm, and tends to disregard more meaningful points around the opposite points. Therefore, we propose a novel opposition-based differential evolution based on the principle of convex lens (LensOBLDE), which utilizes the distance relationship between the target point and the focal length of the lens, and dynamically adjusts the search radius by controlling the parameters, thus further improving the search ability. In addition, to avoid the algorithm from falling into a local optimum, a small-scale and low-probability cauchy perturbation strategy is applied to the elite individuals in the population, which ensures the exploration and exploitation of the algorithm. Finally, LensOBLDE evaluates the performance of several advanced algorithms on the CEC2017 benchmark suite and conducts a statistical test to evaluate the difference between algorithms. The results indicate that LensOBLDE outperforms several algorithms with a significant difference between them. This verifies the effectiveness of the strategies newly added in this study.


MIMR: Modality-Invariance Modeling and Refinement for unsupervised visible-infrared person re-identification

Abstract: Compared to visible modality person re-identification that handles only the intra-modality discrepancy, visible-infrared person re-identification (VI-ReID) aims to achieve the match between visible modality and infrared modality. However, supervised VI-ReID methods are limited in flexibility due to their dependence on labeled data. Although several unsupervised VI-ReID methods have been developed, they usually ignore the impact of hard samples and noisy labels. In this paper, we propose the modality-invariance modeling and refinement (MIMR) framework for unsupervised VI-ReID. For the first issue, we first conduct cross-modality image translation, then enable the encoder to extract modality-invariant feature representation by aligning translated versions with original samples, and finally use cross-modality clustering and hard sample contrastive loss to handle the hard samples. For the second issue, we believe that ambiguous samples are usually assigned noisy labels, so we design ambiguity-oriented pseudo label refinement (APLR), which evaluates ambiguity from both the sample itself and the corresponding translated version, rather than discards hard samples as existing methods. Extensive experiments demonstrate that MIMR achieves superior performance compared to state-of-the-art unsupervised methods and even surpasses early supervised methods.


NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

Abstract: Federated learning (FL), as a potential machine learning framework for privacy preservation, has gained significant attention. However, the considerable communication overhead associated with FL remains a prominent challenge. To mitigate this issue, a nonuniform quantization scheme based on Lloyd-Max algorithm is introduced in this letter. By employing this approach, less communication resources are consumed to achieve the same performance. Through performance analysis and numerical simulations, we verify the convergence and effectiveness of the proposed algorithm. It demonstrates the potential of our approach in reducing communication overhead while maintaining reliable performance in FL systems.


Enhancing Gesture Recognition in Low-Light Conditions Using Laser-Induced Graphene Flexible Strain Sensors

Abstract: Gesture recognition using machine learning methods is valuable in the development of healthcare and human-computer interaction, and typically relies on images or video. To improve recognition accuracy, these visual data can be combined with data from other sensors, but this approach, known as data fusion, is limited by the quality of sensor data and incompatible datasets. Here, we propose a flexible strain sensor based on laser-induced graphene technology, which is easy to manufacture and has superior performance. The sensor is attached to the finger and effectively captures finger bending information as an additional gesture information channel, constructing a bimodal gesture interaction interface for strain and vision. In particular, a deep convolutional neural network based on machine learning is used to acquire features in monocular camera image data. The data fusion method of feature layer fusion is selected to construct a bimodal gesture recognition system based on image data and strain sensor data. Experimental tests show that the strain and vision bimodal gesture interaction interface developed based on this fusion recognition model can still achieve more than 90% real-time gesture recognition accuracy under an illumination level of 20 lux, which effectively solves the problem of the traditional monocular vision’s gesture interaction interface with a significant decrease in recognition accuracy under dark light conditions, and also further expands the application of LIG flexible sensor devices in human-computer interaction system.


Enhancing Life Still Sketch Skills Through Virtual Reality Technology

Abstract: This study addresses a significant gap in research concerning the utilization of virtual reality (VR) technology within higher education in China. Focusing on Mianyang Teachers' College in Sichuan, the research explores the potential integration of VR into the practice of life still sketching. Using a qualitative case study approach, ten experts, comprising five in life still sketching and five in VR technology, were interviewed. Thematic analysis of the data revealed that life still sketching is viewed as a highly subjective mode of artistic expression, emphasizing adherence to fundamental aesthetic principles. The study underscores the importance of factors such as the art's current status, exploratory nature, instructional objectives, and technical aspects for effective life still sketching. VR technology, with devices like PICO and QUEST2, emerges as a promising tool to enhance this artistic practice, offering innovative solutions to its challenges. This research contributes valuable insights to the intersection of VR and art education in the Chinese higher education context.


Progress, achievements, and challenges in multimodal sentiment analysis using deep learning: A survey

Abstract: Sentiment analysis is a computational technique that analyses the subjective information conveyed within a given expression. This encompasses appraisals, opinions, attitudes or emotions towards a particular subject, individual, or entity. Conventional sentiment analysis solely considers the text modality and derives sentiment by identifying the semantic relationship between words within a sentence. Despite this, certain expressions, such as exaggeration, sarcasm and humor, pose a challenge for automated detection when conveyed only through text. Multimodal sentiment analysis incorporates various forms of data, such as visual and acoustic cues, in addition to text. By utilizing fusion analysis, this approach can more precisely determine the implied sentiment polarity, which includes positive, neutral, and negative sentiments. Thus, the recent advancements in deep learning have boosted the domain of multimodal sentiment analysis to new heights. The research community has also shown significant interest in this topic due to its potential for both practical application and educational research. In light of this fact, this paper aims to present a thorough analysis of recent ground-breaking research studies conducted in multimodal sentiment analysis, which employs deep learning models across various modalities such as text, audio, image, and video. Furthermore, the article dives into a discussion of the multiple categories of multimodal data, diverse domains in which multimodal sentiment analysis can be applied, a range of operations that are integral to multimodal sentiment analysis, deep learning architectures, a variety of fusion methods, challenges associated with multimodal sentiment analysis, and the benchmark datasets in addition to the state-of-the-art approaches. The ultimate goal of this survey is to indicate the success of deep learning architectures in tackling the complexities associated with multimodal sentiment analysis.


Asynchronous evolution of deep neural network architectures

Abstract: Many evolutionary algorithms (EAs) take advantage of parallel evaluation of candidates. However, if evaluation times vary significantly, many worker nodes (i.e., compute clients) are idle much of the time, waiting for the next generation to be created. Evolutionary neural architecture search (ENAS), a class of EAs that optimizes the architecture and hyperparameters of deep neural networks, is particularly vulnerable to this issue. This paper proposes a generic asynchronous evaluation strategy (AES) that is then adapted to work with ENAS. AES increases throughput by maintaining a queue of up to K individuals ready to be sent to the workers for evaluation and proceeding to the next generation as soon as M<<K individuals have been evaluated. A suitable value for M is determined experimentally, balancing diversity and efficiency. To showcase the generality and power of AES, it was first evaluated in eight-line sorting network design (a single-population optimization task with limited evaluation-time variability), achieving an over two-fold speedup. Next, it was evaluated in 11-bit multiplexer design (a single-population discovery task with extended variability), where a 14-fold speedup was observed. It was then scaled up to ENAS for image captioning (a multi-population open-ended-optimization task), resulting in an over two-fold speedup. In all problems, a multifold performance improvement was observed, suggesting that AES is a promising method for parallelizing the evolution of complex systems with long and variable evaluation times, such as those in ENAS.


Universal domain adaptation from multiple black-box sources

Abstract: Black-box domain adaptation (B2DA) is a practical unsupervised domain adaptation (UDA) setting, in which only an interface to source model is available due to privacy or security concerns. It aims to transfer source knowledge with no source data or model, but only queries for target samples from the source interface. Previous B2DA methods commonly assume a single source interface and shared label space across domains. Whereas in real tasks, there are usually multiple source interfaces with label shift from the target domain, how to effectively combine them without target ground-truth has not been fully investigated yet so far. We term such setting as Universal Domain Adaptation from Multiple Black-box sources (Uni-MBDA), and propose a new Universal multi-source Black-Box method (Um2B) through Dual-Attention on both source domains and classes. In Um2B, adaptive domain attention is first introduced to seek the optimal combination of source interfaces, whose performance is no worse than the single best one. Second, to address the label shift across domains, adaptive class attention is adopted in adaptation to suppress the private source classes, then automatically identify target unknowns and separate them apart from the known classes. Empirical results over benchmark datasets show the effectiveness of Um2B, and in most cases, it performs no worse than the best single-source.


Pragmatic degradation learning for scene text image super-resolution with data-training strategy

Abstract: Super-resolution of scene text images represents a formidable computational problem, marred by a myriad of intricate challenges. This paper focuses on the specific hurdles that have impeded significant advancements in this domain, and introduces the Higher-Order Degradation-Based Super-Resolution Network (HDSN) as a novel solution to address these intricate issues. The challenges in super-resolving scene text images are manifold. Firstly, the semantic ambiguity inherent to text in natural scenes often leads to degraded results, as standard super-resolution techniques struggle to preserve meaningful textual content. Additionally, the uncertainty surrounding font variability exacerbates this issue, as different fonts require distinct treatment for optimal super-resolution. Furthermore, scene text images often exhibit long trailing shadows, artifacts, and strong noise, rendering conventional methods inadequate in producing satisfactory results. To tackle these intricate challenges, we propose a pragmatic higher-order degradation modeling process. This process takes into account the nuanced characteristics of scene text images, including the diverse forms of noise such as Gaussian, Poisson, speckle, and JPEG compression noise, as well as varying levels of blurring. By meticulously considering these real-world scenarios, our approach significantly enhances the robustness and adaptability of super-resolution for scene text images. In addition to addressing these challenges, we recognize the issues arising from sparse datasets and the lack of corresponding paired images for training. To surmount this limitation, we introduce a text image pre-training strategy, which proves to be highly effective in improving recognition accuracy. The experimental results on TextZoom affirm the effectiveness of our approach, demonstrating substantial improvements over existing methods. Notably, our HDSN achieves average recognition rates of 67.2% on ASTER, 63.2% on MORAN, and 58.0% on CRNN, surpassing the performance of available approaches. Our source code is available at https://github.com/syyang2022/HDSN.


HFA-GTNet: Hierarchical Fusion Adaptive Graph Transformer network for dance action recognition

Abstract: Dance action recognition is a hot research topic in computer vision. However, current skeleton-based action recognition methods face difficulties in capturing the adequate spatial structure and temporal variations of dance actions, resulting in lower recognition accuracy. In this paper, we propose a Hierarchical Fusion Adaptive Graph Transformer network (HFA-GTNet) for dance action recognition. A Hierarchical Spatial Attention (HSAtt) module is designed to extract different levels of spatial feature information from joint to parts to group, it can effectively learn high-order dependency relationships from local joints to global poses in dance actions. Secondly, to extract the joint variations in dance actions at different speeds, we have designed a Temporal Fusion Attention (TFAtt) module. This module learns the short-term and long-term temporal dependencies among joints across frames. Additionally, to capture the variations in motion patterns and dance styles among different dancers, we introduce an Adaptive Component (AdaptC). Finally, we evaluate our model on two self-built dance datasets, MSDanceAction and InDanceAction, and demonstrate its superior performance compared to other state-of-the-art methods in dance action recognition.


Locality sensitive hashing scheme based on online-learning

Abstract: Locally Sensitive Hashing (LSH) algorithms are classical algorithms commonly used on the c-Approximate Nearest Neighbor (c-ANN) search problem. When using Euclidean distance to measure sample similarity and solve the c-ANN problem, the traditional approach is to utilize the Exact Euclidean Locality Sensitive Hashing (E2LSH) algorithm based on the p-stable distribution. However, the uncertainty of the p-stable distribution causes the hash buckets constructed by the E2LSH algorithm to vary in queries. Therefore, this paper proposes the OLLSH algorithm based on the Weighted Majority algorithm in the Online-Learning framework, which selects the hash buckets with more stable query accuracy by weighted voting on the hash buckets generated by the E2LSH algorithm. Then, we conduct simulation experiments on synthetic dataset and four real data sets and conclude that the proposed OLLSH algorithm improves the accuracy compared to the original algorithm with the same memory usage.


Evolution of the random subset feature selection algorithm for classification problem

Abstract: Datasets often include excessive or irrelevant data that affect the performance and complexity of the machine learning model. Feature selection is one of the most effective dimension-reduction technics in datasets to solve the problem. Feature reduction can lead to increased classification accuracy and decreased computational costs. Wrapper feature selection algorithms are a popular and effective category of these methods which considering the learning method's feedback. One of the well-known algorithms in this area is the random subset feature selection algorithm (RSFS). This study proposes an improved version that has higher convergence speed, lower feature selection rate, and higher classification accuracy. Improvements include considering the features' interaction in the selective subsets, continuous evaluation of the selected features to avoid being stuck in the local optima and enhancing the method's evolution phase. To comprehensively evaluate the proposed algorithm, its performance was applied to 20 standard and well-known datasets from public resources and compared with various recent related methods. Experimental results demonstrate that the proposed method reduced classification errors using fewer features compared to other methods, achieving the highest ranking in the Friedman and Wilcoxon rank-sum tests.


SDCS-CF: Saliency-driven localization and cascade scale estimation for visual tracking

Abstract: Discriminative correlation filtering (DCF)-based trackers have demonstrated remarkable results in the field of visual tracking. Nevertheless, in most DCF-based trackers: (i) they apply correlation operations across the entire search area features without discriminative weights, rendering them highly susceptible to background interference; and (ii) the fixed aspect ratio scale search strategy is inadequate for accurate scale estimation under irregular scale variations. To overcome these challenges, this study proposes a new correlation filter through saliency-driven localization and cascaded scale estimation (SDCS-CF). Specifically, a U-like network is devised to generate a pixel-wise saliency map. This map is then multiplied element-wise with search area features to accentuate the target attribute, mitigating distractors and increasing localization confidence. Furthermore, a cascaded scale estimation model consisting of three one-dimensional filters is designed to refine the scale estimation process and improving the robustness. Extensive experimental results on three public datasets demonstrate that the proposed SDCS-CF outperforms most DCF-based trackers.