Mutual Information Guided Financial Report Generation With Domain Adaption

Abstract: The conditional long text generation is an important yet challenging task especially for some domain-specific applications such as financial report generation. Generally, this emerging financial report generation is to generate long and informative text provided by a piece of short news, where the domain knowledge obviously plays an important role. Apparently, such domain knowledge dynamically varies but could be learned from reports written by human specialists to address multiple viewpoints contained in the original news. To address this issue, this article proposes the mutual information guided financial report generation approach with domain adaption. Particularly, we first extract the underlying domain spaces from the input news as well as their corresponding reports, respectively. Then, these domains are aligned with each other in a mutual manner via the proposed domain adaption component. To further enhance the model performance, the mutual information loss is designed to guarantee the semantic meaning of the generated report is close to that of the input news. Extensive experiments are conducted on three public datasets and the proposed approach achieves the state-of-the-art model performance.


Quantum protection scheme for privacy data based on trusted center

Abstract: At present, the protection of data privacy has become one of the research hotspots. However, the classic data privacy protection scheme is based on the classic cryptosystem. The classical cryptosystem is based on the computational complexity of mathematical intractable problems. With the improvement of computer computing power, the security of the classical cryptosystem is seriously threatened. Based on the physical properties of quantum mechanics, using quantum key distribution and quantum teleportation technology, a quantum protection scheme for private data based on trusted center is designed. In theory, this scheme has the unconditional security of quantum cryptography and is independent of the computing power of the attacker.


Semi-supervised class-conditional image synthesis with Semantics-guided Adaptive Feature Transforms

Abstract: Generative Adversarial Networks (GANs) have become the mainstream models for class-conditional synthesis of high-fidelity images. To reduce the demand for labeled data, we propose a class-conditional GAN with Semantic-guided Adaptive Feature Transforms, which is referred to as SAFT-GAN for semi-supervised image synthesis. Instead of simply incorporating a classifier to infer the class labels of unlabeled data, the key idea behind SAFT-GAN is to incorporate class-semantic guidance in real-fake discrimination. More specifically, we adopt a two-head architecture for a discriminator: A label-embedded head identifies real and fake instances, conditioned on class label. To focus more on class-related regions, we exploit class-aware attention information to regularize this head via regional feature transforms. On the other hand, to make better use of unlabeled data, we design a label-free head, on which channel-adaptive feature transforms are imposed to fuse the discriminator and classifier features, such that the class semantics of synthesized images can be improved. Extensive experiments are performed to demonstrate how class-conditional image synthesis can benefit from the proposed feature transforms, and also demonstrate the superiority of SAFT-GAN.


Adversarial Deep Learning based Dampster–Shafer data fusion model for intelligent transportation system

Abstract: Intelligent Transportation Systems (ITS) have revolutionized transportation by incorporating advanced technologies for efficient and safe mobility. However, these systems face challenges ensuring security and resilience against adversarial attacks. This research addresses these challenges and introduces a novel Dampster–Shafer data fusion-based Adversarial Deep Learning (DS-ADL) Model for ITS in fog cloud environments. Our proposed model focuses on three levels of adversarial attacks: original image level, feature level, and decision level. Adversarial examples are generated at each level to evaluate the system's vulnerability comprehensively. To enhance the system's capabilities, we leverage the power of several vital components. Firstly, we employ Dempster–Shafer-based Multimodal Sensor Fusion, enabling the fusion of information from multiple sensors for improved scene understanding. This fusion approach enhances the system's perception and decision-making abilities. For feature extraction and classification, we utilize ResNet 101, a deep learning architecture known for its effectiveness in computer vision tasks. We introduced a novel Monomodal Multidimensional Gaussian Model (MMGM-DD) based Adversarial Detection approach to detect adversarial examples. This detection mechanism enhances the system's ability to identify and mitigate adversarial attacks in real-time. Additionally, we incorporate the Defensive Distillation method for adversarial training, which trains the model to be robust against attacks by exposing it to adversarial examples during the training process. To evaluate the performance of our proposed model, we utilize two datasets: Google Speech Command version 0.01 and the German Traffic Sign Recognition Benchmark (GTSRB). Evaluation metrics include latency delay and computation time (fog–cloud), accuracy, MSE, loss, and F-score for attack detection and defense. The results and discussions demonstrate the effectiveness of our Dampster–Shafer data fusion-based Adversarial Deep Learning Model in enhancing the robustness and security of ITS in fog–cloud environments. The model's ability to detect and defend against adversarial attacks while maintaining low-latency fog–cloud operations highlights its potential for real-world deployment in ITS.


Coarse-to-fine online latent representations matching for one-stage domain adaptive semantic segmentation

Abstract: Domain adaptive semantic segmentation is meaningful since collecting numerous labeled samples in different domains is expensive and time-consuming. Recent domain adaptation methods yield not so efficient performance compared with supervised learning. With the hypothesis that semantic feature can be shared across domains, this paper proposes a coarse-to-fine online matching architecture (COM) for one-stage domain adaptation. We consider subsequent learning stages progressively refining the task in the latent feature space, i.e. the finer set at each component is hierarchically derived from the coarser set of the previous components, including cross-domain global prototypes, categories and instances matching and anchor-points contrastive learning, which further achieve self-supervised learning with region-level pseudo label generated only in a single training step. Beforehand, feature refinement are performed to realize edge perception and inter-feature augmentation. Then, coarse-to-fine network fuses global and local consistency matching via specific distribution alignment between the source and target domain. Finally, the adversarial structure controls the uncertainty of generator prediction through the maximization of classification results and minimization of two classifiers discrepancy. This proposed method is evaluated in two unsupervised domain adaptation tasks, i.e. GTA5 → Cityscapes and SYNTHIA → Cityscapes. Extensive experiments verify the effectiveness of our proposed COM model and demonstrate its superiority over several state-of-the-art approaches.


Parallel disentangling network for human–object interaction detection

Abstract: Human–object interaction (HOI) detection aims to localize and classify triplets of human, object and interaction from a given image. Earlier two-stage methods suffer both from mutually independent training processes and the interference of redundant negative human–object pairs. Prevailing one-stage transformer-based methods are free from the above problems by tackling HOI in an end-to-end manner. However, one-stage transformer-based methods carry the unnecessary entanglements of the query for different tasks, i.e., human–object detection and interaction classification, and thus bring in poor performance. In this paper, we propose a new transformer-based approach that parallelly disentangles human–object detection and interaction classification in a triplet-wise manner. To make each query focus on one specific task clearly, we exhaustively disentangle HOI by parallelly expanding the naive query in vanilla transformer as triple explicit queries. Then, we introduce a semantic communication layer to preserve the consistent semantic association of each HOI through mixing the feature representations of each query triplet of the correspondence constraint. Extensive experiments demonstrate that our proposed framework outperforms the existing methods and achieves the state-of-the-art performance, with significant reduction in parameters and FLOPs.


Multi-view graph imputation network

Abstract: Graph data in the real world is often accompanied by the problem of missing attributes. Recently, self-supervised graph representation learning, implementing data imputation according to observable nodes, has become a paradigm for studying attribute-missing graphs. However, existing methods directly encode attribute-missing graphs and then impute attribute-missing nodes in the latent space, which often increases the uncertainty of node embedding information, thereby limiting the performance of attribute imputation. To address this issue, we propose a novel method named Multi-view GrAph impuTation nEtwork (MATE), which performs attribute imputation in the input space for attribute-missing graphs. Specifically, we first employ parameter initialization and graph diffusion in the input space to generate relatively complete multi-view from both the attribute and structure levels. To provide reliable guidance in each epoch for parameter initialization, we propose a Dual Constraint Strategy (DCS) that maximizes the consistency of node embeddings between two views. In this way, the learning of parameter initialization and node embedding promotes each other, thus effectively improving the quality of attribute imputation. Extensive experiments on four benchmark datasets demonstrate that our proposed MATE achieves state-of-the-art performance. The corresponding code is available at https://github.com/XinPeng97/MATE.


Adaptive Trust Model for Multi-Agent Teaming Based on Reinforcement-Learning-Based Fusion

Abstract: The performance of agents is highly influenced by multiple factors, including ability, decision, and states. Trust modeling is widely used to boost the performance of multiagent teaming (MAT). However, most existing trust models rely on statistical methods or preset parameters to assess the trust value in the MAT scenario. In this article, an adaptive trust model is proposed to evaluate comprehensive trust values based on multiple pieces of evidence from variant sources. The proposed trust model leverages information fusion and RL to properly fuse multiple pieces of evidence to generate trust value for every agent in MAT. The trust value is then used in an interaction protocol with MAT to increase the efficiency of cooperation. To verify the performance of the proposed trust model, a ball-collection experiment is designed for MAT to work cooperatively in simulation environments. Two different scenario settings are used to demonstrate the adaptability and robustness of the proposed trust model. The results are further compared with human-designed fusion methods. The comparison shows that the proposed trust model has a better representation of agent performance, namely convergence speed, than human-designed methods in different scenario settings.


CS-GAC: Compressively sensed geodesic active contours

Abstract: This paper proposes an edge based compressively sensed (CS) geodesic active contour (GAC) model, termed CS-GAC, to ensure faithful edge detection and accurate object segmentation. The motivation behind this paper is that edge information driving the contour evolution can be iteratively obtained by incomplete CS measurements. In each iteration, the CS-GAC is a three-step process including edge detection, active contouring and sparse reconstruction. Instead of working on the final reconstructed images themselves, the evolution of the CS-GAC is driven by a few CS measurements and guided by updatable edge information. The edge information is generated by a complex shearlet transform (CST) based edge map. In the framework, reconstruction and edge detection work alternately. The iterative update property that takes advantages of both edge sparsity and edge detection can largely improve the evolution precision. Numerical experiments show that the CS-GAC can obtain challenging segmentation results in comparisons with the state of the art methods, and has competitive prospects.


Convolutional neural networks rarely learn shape for semantic segmentation

Abstract: Shape learning, or the ability to leverage shape information, could be a desirable property of convolutional neural networks (CNNs) when target objects have specific shapes. While some research on the topic is emerging, there is no systematic study to conclusively determine whether and under what circumstances CNNs learn shape. Here, we present such a study in the context of segmentation networks where shapes are particularly important. We define shape and propose a new behavioral metric to measure the extent to which a CNN utilizes shape information. We then execute a set of experiments with synthetic and real-world data to progressively uncover under which circumstances CNNs learn shape and what can be done to encourage such behavior. We conclude that (i) CNNs do not learn shape in typical settings but rather rely on other features available to identify the objects of interest, (ii) CNNs can learn shape, but only if the shape is the only feature available to identify the object, (iii) sufficiently large receptive field size relative to the size of target objects is necessary for shape learning; (iv) a limited set of augmentations can encourage shape learning; (v) learning shape is indeed useful in the presence of out-of-distribution data.


Weighted side-window based gradient guided image filtering

Abstract: Image filtering under guidance image, known as guided filtering (GF), has been successfully applied to a variety of applications. Existing GF methods utilize either conventional full window-based framework (FWF) or simple uniformly weighted aggregation strategy (UWA); thereby they suffer from edge-blurring. In this paper, based upon gradient guided filtering (GGF), a weighted side-window based gradient guided filtering (WSGGF) is proposed to address the aforementioned problem. First, both regression and adaptive regularization terms in GGF are improved upon eight side windows by introducing side window-based framework (SWF). L1 norm is adopted to choose the results calculated in side windows. Second, UWA strategy in GGF is replaced by a refined variance-based weighted average (VWA) aggregation. In VWA, the value of each weight is chosen inversely proportional to the corresponding estimator. We show that with these improvements our method can well retain the edge sharpness and is robust to visual artifacts. To cut down the time consumption, a fast version of WSGGF (FWSGGF) is further proposed by incorporating a simple but effective down-sampling strategy, which is about four times faster while maintaining the superior performance. By comparing with the state-of-the-art (SOTA) methods on edge-aware smoothing, detail enhancement, high dynamic range image (HDR) compression, image luminance adjustment, depth map upsampling and single image haze removal, the effectiveness and flexibility of our proposed methods are verified. The source code is available at: https://github.com/weimin581/WSGGF


Matrix randomized autoencoder

Abstract: Randomized autoencoder (RAE) has attracted much attention due to its strong capability of representation with fast learning speed. However, the mainstream RAEs are still designed for scalar/vector data, which inevitably destroys the structure information of tensor data. To alleviate this deficiency, a novel convolutions based matrix randomized autoencoder (MRAE) is developed for two-dimensional (2D) data in this paper, including a one-side MRAE (OMRAE) exploiting the row or column information and a double-side MRAE (DMRAE) that simultaneously extracts the row and column information by 2 parallel OMRAEs. To reduce meaningless encoded features, the within-class scatter matrix (WSI) and within-class interaction distance (WID) constraints are added into OMRAE resulting WSI-OMRAE and WID-OMRAE, respectively. To demonstrate the superiority, stacked MRAEs are embedded into hierarchical regularized least squares for one-class classification and comparisons with several state-of-the-art methods are provided. The source code would be available at https://github.com/ML-HDU/MRAE.


Semi-supervised clustering with two types of background knowledge: Fusing pairwise constraints and monotonicity constraints

Abstract: This study addresses the problem of performing clustering in the presence of two types of background knowledge: pairwise constraints and monotonicity constraints. Pairwise constraint relate pair of instances, indicating if they must or must not belong to the same cluster, while monotonicity constraints impose and order relationship between the cluster labels, and are applied to the dataset globally. To fuse these to types of background knowledge, the formal framework to perform clustering under monotonicity constraints is, firstly, defined, resulting in a specific distance measure. Pairwise constraints are integrated afterwards by designing an objective function which combines the proposed distance measure and a pairwise constraint-based penalty term, in order to fuse both types of information. This objective function can be optimized with an EM optimization scheme. The proposed method serves as the first approach to the problem it addresses, as it is the first method designed to work with the two types of background knowledge mentioned above. Our proposal is tested with previous approaches from different clustering paradigms in monotonic datasets, including a real-world case of study. Bayesian statistical tests are used to provide empirical evidence in favor of our proposal, and fund subsequent conclusions: the proposed method can be used obtain better results than previous approaches within the monotonic constrained clustering paradigm.


Domain generalization via Inter-domain Alignment and Intra-domain Expansion

Abstract: The performance of traditional deep learning models tends to drop dramatically during being deployed in real-world scenarios when the distribution shift between the seen training and unseen test data occurs. Domain Generalization methods are designed to achieve generalizability to deal with the above issue. Since the features extracted by softmax cross-entropy loss are not adequately domain-invariant, previous works in Domain Generalization have attempted to overcome this problem by employing contrastive-based losses which pull positive pairs (i.e., samples with the same class label) from different domains closer. Unfortunately, these approaches tend to produce an extremely small feature space, which is not robust facing unseen domain and easily overfits to source domains. To address the aforementioned issue, we propose a novel loss named IAIE Loss to simultaneously perform Inter-domain Alignment and Intra-domain Expansion for positive pairs, which facilitates the model to extract domain-invariant features and mitigates overfitting. Specifically, we design two sets of positive samples named "easy positive samples" and "hard positive samples". IAIE Loss pulls the hard positive pairs closer (alignment) while pushing the easy positive pairs apart (expansion). The state-of-the-art results on multiple DG benchmark datasets verify the effectiveness of our method.


A segmentation method based on the deep fuzzy segmentation model in combined with SCANDLE clustering

Abstract: To enhance the low clustering accuracy of the fuzzy clustering segmentation algorithm for analyzing high spatial resolution remote sensing images (HSRRSIs), a deep fuzzy segmentation model (DFSM)combined with Spectral Clustering with Adaptive Neighbors for Deep Learning (SCANDLE) clustering is proposed. The DFSM is used to over-segment the image, and the automatic coding structure is used to adaptively fuse the image features, minimizing the internal compactness and maximizing the external separability of the clustering, yielding better results. Meanwhile, the SCANDLE clustering model is used to cluster the over-segmentation results, and the matrix construction algorithm for adaptive neighborhood allocation is used to map the frame of the connected layer and optimally combine the over-segmentation images to realize the final segmentation results. The new method can accurately segment HSRRSIs with good segmentation performance.


Medoid Silhouette clustering with automatic cluster number selection

Abstract: The evaluation of clustering results is difficult, highly dependent on the evaluated data set and the perspective of the beholder. There are many different clustering quality measures, which try to provide a general measure to validate clustering results. A very popular measure is the Silhouette. We discuss the efficient medoid-based variant of the Silhouette, perform a theoretical analysis of its properties, provide two fast versions for the direct optimization, and discuss the use to choose the optimal number of clusters. We combine ideas from the original Silhouette with the well-known PAM algorithm and its latest improvements FasterPAM. One of the versions guarantees equal results to the original variant and provides a run speedup of O(k2). In experiments on real data with 30000 samples and k= 100, we observed a 10464× speedup compared to the original PAMMEDSIL algorithm. Additionally, we provide a variant to choose the optimal number of clusters directly.


Adaptive Network Configuration for Efficient and Accurate Neural Video Inference

Abstract: Cameras are widely used in many fields, e.g., intelligent transportation, autonomous driving, surveillance, etc. It is thus vital to conduct video analytics in an efficient and accurate manner. However, camera’s built-in capacity is insufficient to support neural network processing, while offloading video streams incurs prohibitive latency and communication cost. In this paper, we find that frame rate, resolution, and neural network inference model, have an intertwined impact on network resource demand. The optimal configuration of these factors also varies with video content feature. To address these challenges, we propose a dynamic configuration update scheme based on predictive video perception using a long short-term memory (LSTM) neural network, to adapt configuration to content changes. This scheme consists of a change detector and a configuration profiler. Through theoretical modeling and analysis, we derive the detection thresholds for both dynamic and stationary video contents, considering the LSTM prediction error. The configuration profiler then updates system by solving an optimization problem, which maximizes the overall utility considering analytics accuracy and resource consumption. Extensive real-world traces-based experiments show that the proposed scheme can save profiling resources by up to 95% while ensuring high accuracy compared with other benchmarks.


Coarse-to-Fine Low-Light Image Enhancement With Light Restoration and Color Refinement

Abstract: Low-light image enhancement aims to improve the illumination intensity while restoring color information. Despite recent advancements using deep learning methods, they still struggle with over or under-exposure in complex lighting scenes and poor color recovery in dark regions. To address these drawbacks, we propose a novel pipeline (called LRCR-Net) to perform Light Restoration and Color Refinement in a coarse-to-fine manner. In the coarse step, we focus on improving the illumination adaptively while avoiding inappropriate enhancement in the brighter or darker regions. This is achieved by introducing a region-calibrated residual block (RCRB) that balances local and global dependencies among different image regions. In the fine step, we aim to retouch the color of the images enhanced in the coarse step. To achieve this goal, we propose learnable image processing operators (LIPOs), including contrast and saturation operators, to refine the color according to the input images' color and contrast information. The final result is an image with proper illumination and rich color. Experiments on four benchmark datasets (NASA, LIME, MEF, and NPE) show that our model outperforms state-of-the-art methods.


Reinforcement learning with time intervals for temporal knowledge graph reasoning

Abstract: Temporal reasoning methods have been successful in temporal knowledge graphs (TKGs) and are widely employed in many downstream application areas. Most existing TKG reasoning models transform time intervals into continuous time snapshots, with each snapshot representing a subgraph of the TKG. Although such processing can produce satisfactory outcomes, it ignores the integrity of a time interval and increases the amount of data. Meanwhile, many previous works focuses on the logic of sequentially occurring facts, disregarding the complex temporal logics of various time intervals. Consequently, we propose a Reinforcement Learning-based Model for Temporal Knowledge Graph Reasoning with Time Intervals (RTTI), which focuses on time-aware multi-hop reasoning arising from complex time intervals. In RTTI, we construct the time learning part to obtain the time embedding of the current path. It simulates the temporal logic with relation historical encoding and compute the time interval between two facts through the temporal logic feature matrix. Furthermore, we propose a new method for representing time intervals that breaks the original time interval embedding method, and express the time interval using median and embedding changes of two timestamps. We evaluate RTTI on four public TKGs for the link prediction task, and experimental results indicate that our approach can still perform well on more complicated tasks. Meanwhile, our method can search for more interpretable paths in the broader space and improve the reasoning ability in sparse spaces.


Feature incremental learning with causality

Abstract: With the emerging of new data collection ways, the features are incremental and accumulated gradually. Due to the expansion of feature spaces, it is more common that there are unknown biases between the distribution of training and testing datasets. It is known as the unknown data selection bias, which belongs to the learning scenario with non-i.i.d samples. The performance of traditional approaches, which need the i.i.d. assumption, will be aggravated seriously. How to design an algorithm to address the problem of data selection bias in this feature incremental scenario is crucial but rarely studied. In this paper, we propose a feature incremental classification algorithm with causality. Firstly, we embed the confounding variable balance algorithm in causal learning into the prediction modeling and utilize the logical regression algorithm with balancing regular terms as a baseline. Then, to satisfy the special requirement of feature increment, we design a new regularizer, which maintains the consistency of the regression coefficients between the data in the current and previous stages. It retains the correlation between the old features and labels. Finally, we propose the Multiple Balancing Logistic Regression model (MBRLR) to jointly optimize the balancing regularizer and weighted logistic regression model with multiple feature sets. We also present theoretical results to show that our proposed algorithm can make precise and stable predictions. Besides, the numerical results also demonstrate that our MBRLR algorithm is superior to other methods.


Deep Contrastive Clustering for Signal Deinterleaving

Abstract: In the complex electromagnetic environment, radar signal deinterleaving (RSD) is a challenging task. In this paper, a deep contrastive clustering algorithm (DCCA) is advanced in a new self-supervised paradigm, for accurate RSD without any prior information of radar emitters. First, a contrastive self-supervised deep attention network (CSDAN) is constructed to learn signal representations, by using self-defined pseudo-labels of augmented signals as supervision. We use CSDAN to learn the differences between different radiation source data and generate deep features suitable for clustering. Three metrics are then used to automatically determine the number of clusters for the subsequent clustering. Extensive experiments are performed on several datasets containing different numbers of emitters. The results show that the proposed DCCA can accurately determine the number of emitters and deinterleave radar pulses. Furthermore, CSDAN can extract discriminative features of emitters with low intra-class similarity and high inter-class similarity


A Novel Deep Belief Network Architecture with Interval Type-2 Fuzzy Set Based Uncertain Parameters Towards Enhanced Learning

Abstract: This paper proposes a novel Deep Belief Network (DBN) architecture, the ‘Interval Type-2 Fuzzy DBN (IT2FDBN)’, which models the weights and biases with IT2 FSs. Thus, it introduces a novel algorithm for augmented deep leaning, which has the capability to address all the limitations of the classical DBN (CDBN) and T1 fuzzy DBN (T1FDBN). We comparatively evaluate the performance of the IT2FDBN by conducting experiments using the popular MNIST handwritten digit recognition datasets. Additionally, to demonstrate its robustness and generalization capabilities, we also conduct experiments taking two noisy variants of MNIST dataset, viz. the MNIST with AWGN (additive white Gaussian noise) and the MNIST with motion blur. We conduct extensive simulations by considering different combinations of nodes in the hidden layers of the DBN for better model selection. We thoroughly compare the results using well-known performance measures such as root mean square error (RMSE) and Error rate. We show that, in terms of RMSE values and error rates, the proposed IT2FDBN outperforms both T1FDBN and CDBN across all the three datasets. Further, we also provide the results of convergence, runtime-based comparison, and statistical analysis in support of our proposal.


Auto-weighted multi-view clustering with the use of an augmented view

Abstract: Multi-view clustering is a powerful technique that leverages both consensus and complementary information from multiple perspectives to achieve impressive results. However, existing methods have not fully explored the inherent structural information contained in multi-view data, such as inadequately harnessing the consistency among views or neglecting to explore their high-order correlations. To address these limitations, we propose a novel auto-weighted multi-view clustering (AWMVC) method with the use of an augmented view. The main motivation of AWMVC is that each view naturally embodies consensus information, and concatenating the views maximizes the consistency properties. AWMVC first connects multiple views via feature concatenation to form a newly augmented view. Then, we stack the self-representation matrices of the original views and the newly augmented view, i.e., the concatenated view into a 3rd-order tensor, subject to a low-rank tensor constraint. Additionally, this approach automatically assigns weights to each view and constructs a more reliable affinity matrix. Our method unifies the feature concatenation process, high-order information, and a weighted strategy into one model, and is efficiently optimized through an iterative algorithm. Experimental results on benchmark datasets demonstrate that AWMVC outperforms state-of-the-art methods, highlighting the effectiveness of our approach.


Generative adversarial networks via a composite annealing of noise and diffusion

Abstract: Generative adversarial network (GAN) is a framework for generating fake data using a set of real examples. However, GAN is unstable in the training stage. In order to stabilize GANs, the noise injection has been used to enlarge the overlap of the real and fake distributions at the cost of increasing variance. The diffusion process (or data smoothing in its spatial domain) removes fine details in order to capture the structure and important patterns in data but it suppresses the capability of GANs to learn high-frequency information in the training procedure. Based on these observations, we propose a data representation for the GAN training, called noisy scale-space (NSS), that recursively applies the smoothing with a balanced noise to data in order to replace the high-frequency information by random data, leading to a coarse-to-fine training of GANs. We experiment with NSS using DCGAN and StyleGAN2 based on benchmark datasets in which the NSS-based GANs outperforms the state-of-the-arts in most cases.


Traffic sign attack via pinpoint region probability estimation network

Abstract: Recent work show that Deep Neural Networks (DNNs) have created great performance in many tasks, but they are vulnerable to adversarial examples which trigger Artificial Intelligence (AI) security risks. Especially in the autonomous driving field, attacking a traffic sign classification network results in a serious consequence. Most existing researches prefer to digital level attacks focusing on smaller or more imperceptible adversarial noise. Given that attacks available to real-world implementation usually emerge in more security-critical scenarios, we propose an adaptively adversarial example generation algorithm for physical attacks in the real-world setting. Taking account of the traffic sign classification, our approach is divided into two steps. The first step is to generate a probability map which precisely predicts the probability of being attacked for each pixel in the input image through the proposed Pinpoint Region Probability Estimation Network (PRPEN) and meanwhile, try to reduce the size of the highlighted area in the map. It can also be regarded as a classification problem in which every pixel has two classes, suitable for attacking or not, including the restrictions on the number of items in the suitable sort. The second one is to remake a mask depending on the probability map and optimize adversarial patches only on what mask decides. Experimental results show that our method achieves almost 100% misclassification rate in several widely used networks with even smaller patches. We also find how to effectively disguise as a target class to mislead the DNN classifiers and improve AI security.


A topic modeling and image classification framework: The Generalized Dirichlet variational autoencoder

Abstract: Latent Dirichlet allocation model (LDA) has been widely used in topic modeling. Recent works have shown the effectiveness of integrating neural network mechanisms with this generative model for learning text representation. However, one of the significant setbacks of LDA is that it is based on a Dirichlet prior that has a restrictive covariance structure. All its variables are considered to be negatively correlated, which makes the model restrictive. In a practical sense, topics can be positively or negatively correlated. To address this problem, we proposed a generalized Dirichlet variational autoencoder (GD-VAE) for topic modeling. The Generalized Dirichlet (GD) distribution has a more general covariance structure than the Dirichlet distribution because it takes into account both positively and negatively correlated topics in the corpus. Our proposed model leverages rejection sampling variational inference using a reparameterization trick for effective training. GD-VAE compares favorably to recent works on topic models on several benchmark corpora. Experiments show that accounting for topics' positive and negative correlations results in better performance. We further validate the superiority of our proposed framework on two image data sets. GD-VAE demonstrates its significance as an integral part of a classification architecture. For reproducibility and further research purposes, code for this work can be found at https://github.com/hormone03/GD-VAE.


Contrastive clustering with a graph consistency constraint

Abstract: Compared with classical contrastive learning methods, the performance of contrastive clustering is more easily affected by the quality of positive and negative samples, due to the fact that the clustering assumption requires neighbors of points as their positives. In order to reduce the effect of the uncertainty of positives and negatives on contrastive clustering, we propose a new contrastive clustering algorithm with graph consistency constraint. In this algorithm, a loss of graph consistency is proposed to reduce the impact of false negative samples by comparing the neighbor distribution of positive samples. Furthermore, an incremental training method is designed to control the quality of the selected positives. Extensive experiments show that our algorithm outperforms other deep clustering methods on wide-used benchmark data sets.


MinJoT: Multimodal infusion Joint Training for noise learning in text and multimodal classification problems

Abstract: Amidst the critical role that high-quality labeled data plays in advancing machine learning, the persistence of noise within widely-used datasets remains a challenge. While noise learning has gained traction within machine learning, particularly in computer vision, its exploration in text and multimodal classification domains has lagged. Furthermore, a comprehensive comparison of noise learning techniques in text and multimodal classification has been lacking, partly due to variations in experimental noise settings across prior studies. Addressing these gaps, this research introduces a pioneering Multimodal Infusion Joint Training (MinJoT) framework featuring a novel co-regularized loss function that seamlessly integrates multimodal information during joint training. This framework notably excels in maintaining model robustness amidst noisy data environments. Adapting established noise learning methods from computer vision to text classification, the study conducts extensive experiments across five English and Chinese textual and multimodal datasets, involving four methods, five noise modes, and seven noise rates. Critically, this work challenges the implicit assumption that widely-used datasets are devoid of noise, revealing that these datasets indeed encompass noise levels ranging from 0.61% to 15.77% which is defined as intrinsic noise in this study. For the first time, the study investigates the impact of intrinsic noise on model performance, categorizing it into distinct levels of ambiguity. To facilitate accurate method comparison, a new dataset, Golden-Chnsenticorp (G-Chnsenticorp), is introduced, carefully crafted to be free of intrinsic noise. This research establishes the inaugural noise learning benchmark for text classification, while simultaneously pioneering the first noise learning framework tailored for multimodal sentiment classification. Through these contributions, the study advances the understanding of noise learning in text and multimodal contexts, providing a novel framework, benchmarks, and insights to propel the field forward.


A new fast and efficient dehazing and defogging algorithm for single remote sensing images

Abstract: Information about the earth's surface is difficult to capture in remote sensing images because bad weather greatly curtails visibility and diminishes visual contrast in the images. For the purpose of military survey and aerial surveillance, these images are crucial for providing information. It is quintessential to eradicate bad weather conditions like haze and fog from remote sensing images. Recently, numerous deweathering initiatives and endeavors have been undertaken to alleviate these limitations. These contemporary deweathering approaches, however, are inadequate to recover dense haze images. This work presented a novel visibility restoration approach based on segmentation and unsharp mask guided filtering method. It consists of the following steps to restore the scene's radiance: First, a segmentation method is employed to determine atmospheric light to quantify contrast and color. Estimation of the transmission map utilizing the dark channel prior is then performed to precisely determine the proximity between objects. The resultant output image contains halo artifacts and inconsistencies in the structure. A guided filter method based on unsharp masking is employed to optimize the transmission map in order to solve this issue. The experimental results demonstrate that the proposed indicators ensures high uniformity regarding qualitative and quantitative evaluation using six performance metrics: mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), universal image quality index (UQI), fog-aware density evaluator (FADE), dehazing algorithm index (DHQI), standard deviation (CC), and blind contrast enhancement assessment (e and r).


Segmentation and thickness calculation of glomerular basement membrane using RADS-Net in glomerular microscopic images

Abstract: Glomerular ultrastructure needs to be observed under transmission electron microscopy during pathological diagnosis of renal biopsies. Evaluating the glomerular basement membranes (GBM) is necessary for the renal pathological diagnosis. There is little work that performs well for full-flow automated analysis of GBM in whole-glomerular electron microscopy images, including GBM segmentation, location, and thickness calculation. To address this problem, we propose a network architecture, RADS-Net, whose segmentation module combines the advantages of ViT and CNN to achieve better performance in GBM and electron dense deposit (EDD) segmentation tasks. It also contains an uncertainty evaluation module and a location module to exclude unsuitable regions for measurement and extract accurate GBM contours. We propose a GBM thickness calculation method based on statistical learning method, skeletonization algorithm, and improved region growth method. Systematic experiments demonstrate that our proposed segmentation network achieves superior accuracy compared to other competing methods. Additionally, the computed GBM thickness, after an automatic screening of suitable fields of view, meets the requirements for pathology evaluation.


Prototype-oriented contrastive learning for semi-supervised medical image segmentation

Abstract: Semi-supervised learning has been recently explored to advance medical image segmentation due to the challenges of acquiring sufficient labeled data. However, the mainstream semi-supervised learning methods leverage unlabeled data by enforcing the perturbation-based prediction consistency, which neglects the semantic relations across different images and cannot handle inductive bias problem about the feature distribution. To address the above problems, in this paper, a novel prototype-oriented contrastive learning framework is proposed for semi-supervised medical image segmentation. Specifically, inspired from prototypical learning, a set of prototypes is extracted first to represent the diverse feature distributions of different images. Then we present semi-supervised contrastive learning with a prototype-oriented sampling strategy, encouraging the network to explore voxel-wise semantic relations across different images and learn more discriminative features to enhance the segmentation ability. The effectiveness of the proposed method is illustrated on three public benchmark datasets. Extensive experiments show that the proposed method outperforms several state-of-the-art semi-supervised segmentation methods, demonstrating its effectiveness for the challenging semi-supervised medical image segmentation task.


GAN inversion-based semi-supervised learning for medical image segmentation

Abstract: The crux of medical image segmentation stems from learning pixel-wise semantic consistency from a mount of labeled samples through exhaustive annotation. Most existing methods rely heavily on labeled image pairs and are optimized in a fully supervised manner, often resulting in unsatisfactory results due to limited paired training samples. In this paper, we propose a GAN inversion-based semi-supervised learning framework InvSSL for medical image segmentation, which generates corresponding variants from labeled images by GAN inversion. Specifically, it first trains a StyleGAN as an inversion generator which is able to generate samples with the same distribution as training samples, and then obtains variant samples from a training sample by performing GAN inversion to strengthen the segmentation performance in a semi-supervised learning manner. In particular, inversion generator inverts a training sample into the latent space and adds interference to the latent codes to evolve variant samples. To model high-level consistency between pixels and decouple semantic components in the latent space, our InvSSL designs a multi-level dense contrastive learning mechanism. Extensive experiments on lung segmentation and skin lesion segmentation demonstrate that our InvSSL outperforms state-of-the-art methods for medical image segmentation. Our code will be released at https://github.com/funkdub/InvSSL.


Reciprocal cross-modal guidance for liver lesion segmentation from multiple phases under incomplete overlap

Abstract: Liver lesion segmentation is beneficial for computer-aided disease diagnosis, which can be further enhanced by incorporating multi-phase images. Nevertheless, existing segmentation models still need improvement as they build on the unsolid assumption that lesions are well-aligned between different phases. Our goal was to develop a robust model that achieves high accuracy in lesion segmentation, even in cases where incomplete overlap occurs across multiple phases. To overcome the strong assumption of precise alignment between phases, we introduce a novel deep neural network featuring multi-scale feature fusion and cross-phase feature guidance. The former enables the retrieval of effective information from each phase that is helpful in lesion segmentation, while the latter ensures exchanging of guidance maps for interphase segmentation. Comprehensive experiments were carried out on an in-house multi-phase dataset having liver lesions, as well as a multi-modal dataset from the CHAOS challenge. Results show that our model surpasses existing state-of-the-art methods in segmenting liver lesions and abdominal organs from multi-phase images, even if the registration between phases is inaccurate. A competent solution was proposed for accurate liver lesion segmentation from multi-phase images, considering the imperfect registration between phases. The model was proved to be effective in extracting pertinent information from each phase and sharing useful information between phases. The accommodation of incomplete overlap in multi-phase images provides a new paradigm of accurate and robust lesion segmentation, which is particularly helpful in real-world scenarios where registration accuracy may be compromised.


Two-stage fine-grained image classification model based on multi-granularity feature fusion

Abstract: Fine-grained visual classification (FGVC) is a difficult task due to the challenges of discriminative feature learning. Most existing methods directly use the final output of the network which always contains the global feature with high-level semantic information. However, the differences between fine-grained images are reflected in subtle local regions which often appear in the front of the network. When the texture of the background and object are similar or the proportion of the background is too large, the prediction will be greatly affected. In order to solve the above problems, this paper proposes multi-granularity feature fusion module (MGFF) and two-stage classification based on Vision-Transformer (ViT). The former comprehensively represents images by fusing features of different granularities, thus avoiding the limitations of single-scale features. The latter leverages the ViT model to separate the object from the background at a very small cost, thereby improving the accuracy of the prediction. We conduct comprehensive experiments and achieves the best performance in two fine-grained tasks on CUB-200-2011 and NA-Birds.


Robust Multitarget Tracking in Interference Environments: A Message-Passing Approach

Abstract: Multitarget tracking in the interference environments suffers from the nonuniform, unknown and time-varying clutter, resulting in dramatic performance deterioration. We address this challenge by proposing a robust multitarget tracking algorithm, which estimates the states of clutter and targets simultaneously by the message-passing (MP) approach. We define the non-homogeneous clutter with a finite mixture model containing a uniform component and multiple nonuniform components. The measured signal strength is utilized to estimate the mean signal-to-noise ratio (SNR) of targets and the mean clutter-to-noise ratio (CNR) of clutter, which are then used as additional feature information of targets and clutter to improve the performance of discrimination of targets from clutter. We also present a hybrid data association which can reason over correspondence between targets, clutter, and measurements. Then, a unified MP algorithm is used to infer the marginal posterior probability distributions of targets, clutter, and data association by splitting the joint probability distribution into a mean-field approximate part and a belief propagation part. As a result, a closed-loop iterative optimization of the posterior probability distribution can be obtained, which can effectively deal with the coupling between target tracking, clutter estimation and data association. Simulation results demonstrate the performance superiority and robustness of the proposed multitarget tracking algorithm compared with the probability hypothesis density (PHD) filter and the cardinalized PHD (CPHD) filter.


Nonexact oracle inequalities, r-learnability, and fast rates

Abstract: As an extension of the standard paradigm in statistical learning theory, we introduce the concept of r-learnability, 0<r≤1, which is a notion very closely related to that of nonexact oracle inequalities (see Lecue and Mendelson (2012) [7]). The r-learnability concept can enable so-called fast learning rates (along with corresponding sample complexity-type bounds) to be established at the cost of multiplying the approximation error term by an extra (1+r)-factor in the learning error estimate. We establish a new, general r-learning bound (nonexact oracle inequality) yielding fast learning rates in probability (up to at most a logarithmic factor) for proper learning in the general setting of an agnostic model, essentially only assuming a uniformly bounded squared loss function and a hypothesis class of finite VC-dimension (that is, finite pseudo-dimension).


DSCA-Net: Double-stage Codec Attention Network for automatic nuclear segmentation

Abstract: The rapid and precise segmentation of cell nuclei from hematoxylin and eosin-stained tissue images is an essential clinical undertaking with significant implications for various clinical applications. The segmentation of cell nuclei poses specific challenges due to the inherent instability of nuclear morphology and the complexity of the segmentation environments. Furthermore, previous studies have primarily relied on small-scale and limited-diverse datasets, potentially hindering their applicability to clinical tasks. This study introduces a novel approach, the Double-stage Codec Attention Network, designed to automatically and accurately segment nuclei. Specifically, we present a hierarchical feature extraction module, which maximizes the utilization of cell nuclei's morphological characteristics in the tissue, thereby providing critical semantic information for nucleus segmentation. Furthermore, the feature selection units are employed to enhance relevant features and suppress interfering ones, thereby enhancing the overall expressive capacity of the information. The multi-scale deep feature fusion module utilizes interrelated encoder–decoder connections to jointly optimize and integrate this information, generating a robust hierarchical feature pyramid. Finally, the feature attention fusion mechanism captures spatial and directional information, aiding the model in the accurate localization and recognition of cell nuclei. We rigorously evaluated our proposed method using the PanNuke dataset, the largest comprehensive histology dataset of cancer tissues. In terms of the average F1-score across all segmentation classes in the PanNuke dataset, DSCA-Net outperforms other state-of-the-art models such as DeepLabV3+, TransUNet, Triple U-net, and TransNuSeg by 1.38, 1.44, 2.64, and 1.02, respectively. Additionally, DSCA-Net shows excellent efficiency in generating predictive images, outperforming all comparative models.


Phase Randomization: A data augmentation for domain adaptation in human action recognition

Abstract: Human action recognition models often suffer from achieving both accurate recognition and subject independence when the amount of training data is limited. In this paper, we propose a data-efficient domain adaptation approach to learning a subject-agnostic action recognition classifier. The core component of our approach is a novel data augmentation called Phase Randomization. On the basis of the observation that individual body size is highly correlated with the amplitude component of the motion sequence, we disentangle the individuality and action features by using contrastive self-supervised learning with data augmentation that randomizes only the phase component of the motion sequence. This enables us to estimate the subject label of each motion sequence and to train a subject-agnostic action recognition classifier by performing adversarial learning with the estimated subject labels. We empirically demonstrate the superiority of our method on two different action recognition tasks (skeleton-based action recognition and sensor-based activity recognition).


Efficient search of comprehensively robust neural architectures via multi-fidelity evaluation

Abstract: Neural architecture search (NAS) has emerged as one successful technique to find robust deep neural network (DNN) architectures. However, most existing robustness evaluations in NAS only consider l∞ norm-based adversarial noises. In order to improve the robustness of DNN models against multiple types of noises, it is necessary to consider a comprehensive evaluation in NAS for robust architectures. But with the increasing number of types of robustness evaluations, it also becomes more time-consuming to find comprehensively robust architectures. To alleviate this problem, we propose a novel efficient search of comprehensively robust neural architectures via multi-fidelity evaluation (ES-CRNA-ME). Specifically, we first search for comprehensively robust architectures under multiple types of evaluations using the weight-sharing-based NAS method, including different lp norm attacks, semantic adversarial attacks, and composite adversarial attacks. In addition, we reduce the number of robustness evaluations by the correlation analysis, which can incorporate similar evaluations and decrease the evaluation cost. Finally, we propose a multi-fidelity online surrogate during optimization to further decrease the search cost. On the basis of the surrogate constructed by low-fidelity data, the online high-fidelity data is utilized to finetune the surrogate. Experiments on CIFAR10 and CIFAR100 datasets show the effectiveness of our proposed method.


Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review

Abstract: The expanding scale of cloud data centers and the diversification of user services have led to an increase in energy consumption and greenhouse gas emissions, resulting in long-term detrimental effects on the environment. To address this issue, scheduling techniques that reduce energy usage have become a hot topic in cloud computing and cluster management. The Deep Reinforcement Learning (DRL) approach, which combines the advantages of Deep Learning and Reinforcement Learning, has shown promise in resolving scheduling problems in cloud computing. However, reviews of the literature on task scheduling that employ DRL techniques for reducing energy consumption are limited. In this paper, we survey and analyze energy consumption models used for scheduling goals, provide an overview of the DRL algorithms used in the literature, and quantitatively compare the model differences of Markov Decision Process elements. We also summarize the experimental platforms, datasets, and neural network structures used in the DRL algorithm. Finally, we analyze the research gap in DRL-based task scheduling and discuss existing challenges as well as future directions from various aspects. This paper contributes to the correlation perspective on the task scheduling problem with the DRL approach and provides a reference for in-depth research on the direction of DRL-based task scheduling research. Our findings suggest that DRL-based scheduling techniques can significantly reduce energy consumption in cloud data centers, making them a promising area for further investigation.


SRMADNet: Swin ResUnet3+-based mammogram image segmentation and heuristic adopted multi-scale attention based DenseNet for breast cancer detection

Abstract: Breast tumor is the second wide largest mortality rate of all the other cancers that are found in women and it is also thought to be the main factor in a high death rate. Mammographic pictures are mostly examined by the trained and expert radiologists to identify the abnormalities in the breast such as microcalcifications and masses. A correct diagnosis and early detection can aid in the fight against this form of dangerous disease. Breast cancer identification and categorization have been found to benefit greatly from the segmentation of mammography lesions. Segmentation can help with both shape-related feature extraction and precise lesion localization. Therefore, specialists implemented the latest image segmentation for mammogram images and the identification architecture for the identification of breast cancers, most importantly for the women. The wanted mammogram pictures are collected from traditional global databases. The gathered pictures are fed into the segmentation process, where the images are segmented employing Swin ResUnet3+. Thus, the mammogram segmented pictures are forwarded to the identification stage, where Adaptive Multi-scale Attention-based Densenet with Extreme Learning Machine (AMAD-ELM) model is implemented to identify the breast tumor effectively from the mammogram pictures. The parameters from AMAD-ELM are optimized through the suggested Position-based Improved invasive Weed and Crisscross Optimization (PIWCO). The effectiveness of the recommended breast tumor identification system with the help of deep learning is analyzed through various existing models to show its effectiveness.


A multispectral transmission image cluster analysis method based on “Terrace compression Method” and window function

Abstract: Multispectral transmission imaging has a great potential for the early screening of breast cancer due to the low cost, safety and ease in operation. Accurate detection of heterogeneity is important for the diagnosis of breast disease. The low contrast and unclear heterogeneity boundaries of transmission images can lead to difficulties in recognition and segmentation. Therefore, we propose a clustering segmentation method of multispectral transmission images based on "terrace compression method" and window transformation. The images are preprocessed by the frame accumulation to improve the signal-to-noise ratio. After that, the "terrace compression method" is used to compress the images nonlinearly, to reduce data redundancy and then to improve the edge information of heterogeneities. Afterwards, the window function is used to eliminate the redundant information of the background and to reduce the influence of background noise on clustering. Finally, the processed images at each wavelength are transformed into multidimensional data for cluster analysis. The multispectral transmission images of breast phantom are acquired for experimental validation. Then, compared the method with common clustering segmentation methods (including K-means, K-means++, Mean-shift, Gaussian Mixture). The results showed that this processing method can effectively segment and classify the three heterogeneities in the breast phantom. Among these methods, the method proposed in the paper has the best segmentation and classification results for the three types of heterogeneities in the breast phantom. The Dice coefficients of all the heterogeneities segmentation reached more than 0.84 and increased by a maximum of 1.08 times as compared to the common clustering methods. The applications of the terrace compression method and the grayscale window transformation improved the effect of image clustering segmentation.


Machine Translation for Historical Research: A Case Study of Aramaic-Ancient Hebrew Translations

Abstract: In this article, by the ability to translate Aramaic to another spoken languages, we investigated machine translation in a cultural heritage domain for two primary purposes: evaluating the quality of ancient translations and preserving Aramaic (an endangered language). First, we detailed the construction of a publicly available Biblical parallel Aramaic-Hebrew corpus based on two ancient (early 2 nd to late 4 th century) Hebrew-Aramaic translations: Targum Onkelus and Targum Jonathan. Then using the statistical machine translation approach, which in our use case significantly outperforms neural machine translation, we validated the excepted high quality of the translations. The trained model failed to translate Aramaic texts of other dialects. However, when we trained the same statistical machine translation model on another Aramaic-Hebrew corpus of a different dialect (Zohar, 13 th century), a very high translation score was achieved. We examined an additional important cultural heritage source of Aramaic texts, the Babylonian Talmud (early 3 rd to late 5 th century). Since we do not have a parallel Aramaic-Hebrew corpus of the Talmud, we used the model trained on the Bible corpus for translation. We performed an analysis of the results and suggest some potential promising future research.


Variable sample-size operator extrapolation algorithm for stochastic mixed variational inequalities

Abstract: In this paper, we present a variable sample-size operator extrapolation algorithm for solving a class of stochastic mixed variational inequalities. One distinctive feature of our algorithm is that it updates a single search sequence by solving a prox-mapping subproblem and computing an evaluation of the expected mapping at each iteration and hence it may significantly reduce computation load. In particular, the iteration sequence generated by our algorithm always belongs to the feasible region. We show that, under some moderate conditions, the proposed algorithm can achieve O(1/T) ergodic convergence rate in terms of the expected restricted gap function, where T denotes the number of iterations. We derive some results related to the convergence rate of the Bregman distance between iterates and solutions, the iteration complexity, and the oracle complexity for the proposed algorithm when the sample size increases at a geometric rate. We also investigate the sublinear convergence rate in terms of the residual function under the generalized monotonicity condition. Numerical experiments on stochastic network game, stochastic sparse traffic assignment problems and sparse classification problem indicate that the proposed algorithm is promising compared with some existing algorithms.


A non-regularization self-supervised Retinex approach to low-light image enhancement with parameterized illumination estimation

Abstract: In current Retinex-based low-light image enhancement (LLIE) methods, fine-tuning regularization parameters for Retinex decomposition and illumination estimation can be cumbersome. To address this, we present a novel non-regularization self-supervised Retinex approach for illumination estimation. Our contributions are twofold: First, we introduce a self-supervised method that incorporates edge-aware smoothness properties in bilateral learning, eliminating the need for regularization terms and simplifying parameter adjustments. Second, to enforce smoothness constraints on the estimated bilateral grid, we propose a bilateral grid parameterization network. This network employs a generative encoder to parameterize the bilateral grid of illumination and a trainable slicing layer guided by a map, reconstructing the grid into an illumination map. Despite the absence of regularization terms, our model excels in generating piece-wise smooth illumination, resulting in enhanced naturalness and improved contrast in images. Our model offers exceptional flexibility by eliminating the need for additional regularization terms and parameter fine-tuning. Moreover, it does not depend on external datasets for training, overcoming dataset collection challenges. Extensive experiments, comparing our model with eight state-of-the-art methods across five public available datasets, unequivocally demonstrate our model's state-of-the-art performance based on key metrics such as NIQE, NIQMC, and CPCQI. These results reaffirm the effectiveness of our approach in low-light image enhancement. Code will be available at: https://github.com/zhaozunjin/NeurBR.


Conditional feature generation for transductive open-set recognition via dual-space consistent sampling

Abstract: Open-set recognition (OSR) aims to simultaneously detect unknown-class samples and classify known-class samples. Most of the existing OSR methods are inductive methods, which generally suffer from the domain shift problem that the learned model from the known-class domain might be unsuitable for the unknown-class domain. Addressing this problem, inspired by the success of transductive learning for alleviating the domain shift problem in many other visual tasks, we propose an Iterative Transductive OSR framework, called IT-OSR, which implements three explored modules iteratively, including a reliability sampling module, a feature generation module, and a baseline update module. Specifically at the initialization stage, a baseline method, which could be an arbitrary inductive OSR method, is used for assigning pseudo labels to the test samples. At the iteration stage, based on the consistency of the assigned pseudo labels between the output/logit space and the latent feature space of the baseline method, a dual-space consistent sampling approach is presented in the reliability sampling module for sampling some reliable ones from the test samples. Then in the feature generation module, a conditional dual-adversarial generative network is designed to generate discriminative features of both known and unknown classes. This generative network employs two discriminators for implementing fake/real and known/unknown-class discriminations respectively. And it is trained by jointly utilizing the test samples with their pseudo labels selected in the reliability sampling module and the labeled training samples. Finally in the baseline update module, the above baseline method is updated/re-trained for sample re-prediction by jointly utilizing the generated features, the selected test samples with pseudo labels, and the training samples. Extensive experimental results on both the standard-dataset and the cross-dataset settings demonstrate that the derived transductive methods, by introducing two typical inductive OSR methods into the proposed IT-OSR framework, achieve better performances than 19 state-of-the-art methods in most cases.


Weighted subspace anomaly detection in high-dimensional space

Abstract: Anomaly detection aims at finding anomalies deviating from the normal data patterns. Virtually all anomaly detection methods create a model of the normal patterns before finding anomalies. In high-dimensional scenarios, due to the curse of dimensionality, it is difficult to construct the model of normal patterns in the full dimensional space. Subspace methods assuming that data can be characterized by low-dimensional manifolds have attracted a great deal of research. However, in unsupervised setting, unlabeled data is composed of both the normal and the abnormal data. The existence of anomalies might affect the establishment of the underlying normal subspaces. The undetermined number of the underlying subspaces also brings difficulties in subspace selection. To tackle the aforementioned problems, we come up with a weighted subspace anomaly detection (WSAD) method. We utilize correntropy to construct an objective function to mitigate the influence of the anomalies, which can be regarded as a weighting method for different data. Besides, we introduce an auxiliary variable with block sparsity regularization to achieve adaptive subspace selection, which can be regarded as a weighting method for different subspaces. After the normal underlying subspaces being established, we define the outlier scores by considering the deviation from the underlying subspaces, the local outlier score within subspaces, and the subspace scale. We use the half-quadratic theory to transform the optimization problem defined in WSAD, and apply alternating optimization to solve the transformed problem. Theoretically, we prove the convergence of the optimization algorithm. Experimentally, we demonstrate the effectiveness of the proposed method on both synthetic data and real datasets.


Deep image clustering with contrastive learning and multi-scale graph convolutional networks

Abstract: Deep clustering has shown its promising capability in joint representation learning and clustering via deep neural networks. Despite the significant progress, the existing deep clustering works mostly utilize some distribution-based clustering loss, lacking the ability to unify representation learning and multi-scale structure learning. To address this, this paper presents a new deep clustering approach termed Image clustering with contrastive learning and multi-scale Graph Convolutional Networks (IcicleGCN), which bridges the gap between convolutional neural network (CNN) and graph convolutional network (GCN) as well as the gap between contrastive learning and multi-scale structure learning for the deep clustering task. Our framework consists of four main modules, namely, the CNN-based backbone, the Instance Similarity Module (ISM), the Joint Cluster Structure Learning and Instance reconstruction Module (JC-SLIM), and the Multi-scale GCN module (M-GCN). Specifically, the backbone network with two weight-sharing views is utilized to learn the representations for the two augmented samples (from each image). The learned representations are then fed to ISM and JC-SLIM for joint instance-level and cluster-level contrastive learning, respectively, during which an auto-encoder in JC-SLIM is also pretrained to serve as a bridge to the M-GCN module. Further, to enforce multi-scale neighborhood structure learning, two streams of GCNs and the auto-encoder are simultaneously trained via (i) the layer-wise interaction with representation fusion and (ii) the joint self-adaptive learning. Experiments on multiple image datasets demonstrate the superior clustering performance of IcicleGCN over the state-of-the-art. The code is available at https://github.com/xuyuankun631/IcicleGCN.


Open set transfer learning through distribution driven active learning

Abstract: Domain adaptation enables effective transfer between source and target domains with different distributions. The latest research focuses on open set domain adaptation; that is, the target domain contains unknown categories that do not exist in the source domain. The existing open set domain adaptation cannot realize the fine-grained recognition of unknown categories. In this paper, we propose an uncertainty analysis evidence model and design a distribution driven active transfer learning (DATL) algorithm. DATL realizes fine-grained recognition of unknown categories with no requirements on the source domain to contain the unknown categories. To explore unknown distributions, the uncertainty analysis evidence model was adopted to divide the high uncertainty space. To select critical instances, a cluster-diversity query strategy was proposed to identify new categories. To enrich the label categories of the source domain, a global dynamic alignment strategy was designed to avoid negative transfers. Comparative experiments with state-of-the-art methods on the standard Office-31/Office-Home/Office-Caltech10 benchmarks showed that the DATL algorithm: (1) outperformed its competitors; (2) realized accurate identification of unknown subcategories from a fine-grained perspective; and (3) achieved outstanding performance even with a very high degree of openness.


A Novel Attention-Driven Framework for Unsupervised Pedestrian Re-identification with Clustering Optimization

Abstract: Unsupervised pedestrian re-identification (re-ID) is not merely a visual recognition task; it represents a significant sub-field within the domain of pattern recognition. Despite the remarkable success of Convolutional Neural Networks (CNN) in re-ID, they still face challenges in handling variations in pose, occlusion, and lighting conditions. To effectively tackle these challenges, it is imperative to prioritize implementing efficient sampling strategies. We propose a Novel Attention-Driven Framework for Unsupervised Pedestrian re-ID with Clustering Optimization (AFC) to address the above issues. First, we introduce a new attention mechanism that enhances multi-scale spatial attention and reduces the number of trainable parameters. Then, we employed a straightforward and effective method of group sampling. In addition, we apply a clustering consensus approach to estimate pseudo-label similarity in continuous training and use temporal propagation and ensembles to improve pseudo-labels. Extensive experiments on Market-1501, duketmc-reID and MSMT17 datasets show that our method achieves significant performance improvement in unsupervised pedestrian re-ID, which provides important theoretical and practical value for the research on deep fusion of pattern recognition field with pedestrian re-ID and promotes the further development of the related fields.


So Many Heads, So Many Wits: Multimodal Graph Reasoning for Text-Based Visual Question Answering

Abstract: While texts related to images convey fundamental messages for scene understanding and reasoning, text-based visual question answering tasks concentrate on visual questions that require reading texts from images. However, most current methods add multimodal features that are independently extracted from a given image into a reasoning model without considering their inter-and intra-relationships according to three modalities (i.e., scene texts, questions, and images). To this end, we propose a novel text-based visual question answering model, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multimodal graph reasoning</i> . Our model first extracts intramodality relationships by taking the representations from identical modalities as semantic graphs. Then, we present graph multihead self-attention, which boosts each graph representation through graph-by-graph aggregation to capture the intermodality relationship. It is a case of “so many heads, so many wits” in the sense that as more semantic graphs are involved in this process, each graph representation becomes more effective. Finally, these representations are reprojected, and we perform answer prediction with their outputs. The experimental results demonstrate that our approach realizes substantially better performance compared with other state-of-the-art models.


The (+)-extended twisted generalized Reed-Solomon code

Abstract: In this paper, we give a parity check matrix of the (+)-extended twisted generalized Reed Solomon code, and then not only prove that it is MDS or NMDS, but also determine the weight distribution. Especially, based on Schur method, we show that the (+)-extended twisted generalized Reed Solomon code is neither the generalized Reed Solomon code nor extended generalized Reed Solomon code. Furthermore, we present necessary and sufficient conditions for the (+)-twisted generalized Reed Solomon code and the (+)-extended twisted generalized Reed Solomon code to be self-orthogonal, respectively. Finally, several classes of self-dual (+)-twisted generalized Reed Solomon code codes and almost self-dual (+)-extended twisted generalized Reed Solomon codes are constructed.


Boundary fusion multi-scale enhanced network for gland segmentation in colon histology images

Abstract: Morphological features of glands serve as essential diagnostic criteria for colon cancer, and physicians must accurately segment glands in histopathological sections to grade the cancer. However, owing to the diverse gland morphologies and blurred edges of malignant glands, the contextual representations of edges have received limited attention in current segmentation methods, leading to less-than-ideal segmentation results, particularly in malignant cases. We present an edge-based multi-scale enhanced network (BFMSE-Net) for gland segmentation to address these issues. This network is based on U-net and fuses an edge fusion module (BFM) and a multi-scale enhancement module (MSEM). The BFM extracts detailed edge features, fuses edge information, and enhances edge pixel values, which effectively facilitates the segmentation of glands with blurred edges and adhesions. MSEM can handle complex scenarios where gland size and shape significantly change because it adaptively gathers information from various receptive fields in the region of interest. The method applied to evaluate this proposed network was to compare it to current methods through experiments, applying the gland segmentation (GlaS) and colorectal adenocarcinoma gland (CRAG) datasets. The experimental results demonstrate that our proposed method achieves clear gland segmentation, with shape similarity indices of 72.617 and 69.792 on the GlaS and CRAG datasets, respectively. These results show an improvement of 20.627 and 76.978, respectively, over current state-of-the-art methods. In conclusion, BFMSE-Net, which clearly focuses on edge representation, can accurately segment various types of glands.


Hierarchical visual-semantic interaction for scene text recognition

Abstract: Proper interaction between visual and semantic features is crucial to obtain a powerful feature representation for scene text recognition (STR). The existing interaction methods usually treat visual and semantic features as distinct tokens and use transformers to learn contextual information and prior language knowledge, and they achieve promising performance for STR task. However, there still remain several issues needed to be further addressed such as the imbalance number and mis-alignment between visual and semantic features, and the necessarily of stacking transformers to a progressive improvement in accuracy. To this aim, this paper proposes a novel interaction manner namely hierarchical visual-semantic interaction (HVSI) which contains three novel modules including a hierarchical visual-semantic interaction module, fusion module, and visual-semantic alignment module. The hierarchical visual-semantic interaction module employs multiple visual-semantic interaction blocks in various scales to enhance the representation power of visual features and semantic features. To better exploit multi-scale visual and semantic features, the fusion module is introduced to fuse multiple semantic features based on attention mechanisms. Furthermore, our HVSI presents a simple plug-in block named visual-semantic alignment module to alleviate mis-alignment of semantic features by mapping them into a unified semantic space, which helps improve the performance of HVSI. Extensive experiments on multiple benchmarks including English and Chinese text recognition datasets show that our method obtains state-of-the-art or competitive performances.


GNaN: A natural neighbor search algorithm based on universal gravitation

Abstract: The natural neighbor (NaN) method and its search algorithm (NaN-Searching) are widely used in many fields, including pattern recognition and image processing. NaN-Searching fundamentally overcomes the problem of the conventional nearest neighbor algorithm in selecting parameters for datasets with arbitrary shapes and achieves good results. However, this algorithm uses the conventional distance metric as the neighbor judgment criterion, which cannot accurately reflect the overall structure of the dataset in the process of neighbor search. Inspired by Newton's law of universal gravitation, we propose a NaN search algorithm based on universal gravitation (GNaN-Searching). Our algorithm calculates gravitation using the structural features of data points in the dataset, it utilizes the gravitation between data as the neighbor judgment criterion, and inherits the no-parameter and dynamic neighborhood characteristics of the NaN search algorithm. Experimental results show that the natural neighborhood graph obtained by our method has a high performance in the representation of manifold data. We also applied the new method to clustering and outlier detection and achieved satisfactory results.


Frequency-aware feature aggregation network with dual-task consistency for RGB-T salient object detection

Abstract: RGB-Thermal salient object detection (SOD) aims to merge two spectral images to segment visually appealing objects. Current methods primarily extract salient object information in the pixel perspective. However, biological and psychological research indicates notable frequency sensitivity of the human visual system (HVS). The high-frequency (HF) and low-frequency (LF) information in images are processed by different neural channels, which has been overlooked in SOD. In this study, we argue that the objective of RGB-T SOD is not only to enhance feature representation in the pixel-aware but also to emulate human visual mechanisms. To our best knowledge, we explore RGB-T SOD from the frequency perspective for the first time. Specifically, we first present a frequency-aware multi-spectral feature aggregation module (FMFA) to exploit the separability and complementarity of frequency-aware features, generating and making full use of LF and HF cues. FMFA improves the feature representation of RGB-T from the frequency perspective and provides stronger frequency cues for boundary auxiliary tasks. Then, we develop an HF-guided signed distance map prediction module (HF-SDM) with dual-task consistency to effectively alleviate the coarse mask caused by blur boundary. HF-SDM employs the geometric relationship of objects which boosts the interaction between salient regions and boundaries. As a result, the model can gain sharper boundaries for salient objects. Finally, we propose a frequency-aware feature aggregation network (FFANet) incorporated with dual-task learning. Extensive experiments on RGB-T SOD datasets demonstrate that our proposed method outperforms other state-of-the-art methods. Ablation studies and visualizations further verify the effectiveness and interpretability of our method.


Optimal Fuzzy Intensification System for Contrast Distorted Medical Images

Abstract: Contrast enhancement plays a significant role in medical imaging technology for clinical diagnosis and observing the patient's health conditions. However, medical images may suffer from different degradations, such as poor visibility and low contrast, which are significant challenges for the automatic diagnosis of diseases. This article develops an ameliorated fuzzy transformation model to significantly enhance the contrast by adjusting the imbalanced intensity levels of medical images. The proposed optimal fuzzy intensification system (OFIS) utilizes the entire dynamic range by expanding the segmented intensity level of the input image. The fuzzy expansion process can enrich the diagnostic details of the processed image. The moth flame optimization is incorporated to optimize the gamma parameter to improve the clear visibility by adjusting the local contrast of the dark regions. Experiments on benchmark datasets reveal that the proposed method has superior performance in enhancing contrast and visibility of the local details of abnormalities in medical images compared with existing methods.


An integrated approach for magnification independent breast cancer classification

Abstract: Breast cancer (BC) is the second most common cancer worldwide, and its diagnosis and treatment rely heavily on histopathology image analysis. Moreover, continuous evaluation of deep learning techniques in medical imaging modalities is essential for improving early diagnosis. The Break-His dataset is the de facto standard for such analysis. Researchers have actively explored various approaches to classifying these images into binary, four, and eight based on the magnification-independent (MI) and magnification-dependent (MD). However, existing processes necessitate separate models for each classification based on the magnification factor and entail training multiple models. It leads to increased training time and model complexity. Moreover, model performance in eight-class is relatively low compared to other classifications. This paper introduces a modified Swin-Transformer V2 architecture for multi-labeled and eight-class classification on BC histopathology images. Further, the eight-class output was reconfigured to operate in an integrated approach, allowing the model to predict binary, four, and eight classes simultaneously for both MI and MD. On the Break-His dataset, the proposed model achieved an accuracy of 98.27%, 97.95%, 98.97%, and 99.16% for eight-class, malignant, benign, and binary classification tasks, respectively. Additionally, the multi-labeled approach demonstrated a performance improvement in the eight-class category compared to current leading models, achieving an enhancement of up to 5%, and the model was trained in just 2 h. Overall, the proposed model can improve the prognosis of breast cancer patients by leading to more accurate diagnosis and treatment. The implemented code can be accessed at https://github.com/kbnarayanavit/integrated-approach.


Explainable deep learning for attack intelligence and combating cyber-physical attacks

Abstract: Cyber-physical control loops comprising sensors, actuators and controllers pose the most valued and critical part of the industrial Internet of Things (IIoT) as it regulates the state of the physical process, such as water treatment or gas flow. Thus, any malicious activities could lead to physical damage, affecting human safety. Cyber-physical attacks against the physical process are difficult to detect using existing threats and attack intelligence due to the (1) lack of such intelligence for the physical process and operational technology systems and (2) such attacks affect the process parameters and states. Artificial Intelligence (AI)-based attack intelligence is required. This study proposes an attack intelligence framework for identifying cyber–physical attacks and extracting attack intelligence. We propose an attribution module for attack identification using various machine and deep learning algorithms. We also utilize Explainable AI (XAI) to improve the explainability of the attack attribution module and extract attack intelligence. Our proposed framework is evaluated and tested using a gas pipeline dataset as a use case. We demonstrate that the proposed framework improves the understanding of attacks and provides attack rules, assisting security analysts in securing critical physical processes.


Light-modulated graphene-based φ 0 Josephson junction and −φ 0 to φ 0 transition

Abstract: We investigate the chiral edge states-induced Josephson current–phase relation in a graphene-based Josephson junction modulated by the off-resonant circularly polarized light and the staggered sublattice potential. By solving the Bogoliubov–de Gennes equation, a φ 0 Josephson junction is induced in the coaction of the off-resonant circularly polarized light and the staggered sublattice potential, which arises from the fact that the center of-mass wave vector of Cooper pair becomes finite and the opposite center of-mass wave vector to compensate is lacking in the nonsuperconducting region. Interestingly, when the direction of polarization of light is changed, − φ 0 to φ 0 transition generates, which generalizes the concept of traditional 0– π transition. Our findings provide a purely optical way to manipulate a phase-controllable Josephson device and guidelines for future experiments to confirm the presence of graphene-based φ 0 Josephson junction.


Receptive Field-Based All-Optical Spiking Neural Network for Image Processing

Abstract: We report on a novel structure of a receptive field (RF)-based multi-layer all-optical neural network using a micropillar laser with a saturable absorber (SA) for image processing. From the perspective of biological vision, the realization of image processing based on the RF provides the biological rationality for the machine vision implemented by the spiking neural network (SNN). By exploiting the fast physical mechanisms of gain and absorption in the SA laser, the photonic spike-timing-dependent plasticity (STDP) curves are achieved to train the weights. Here, the source image pixels are mapped into the temporal information of spike trains injected into the neural network through the temporal coding method called time-to-first-spike encoding. Different source images are processed and tested by the proposed photonic SNN. Simulation results show that our proposed system can process not only simple binary images but also complex color images under the adjustment of STDP rules. When considering the robustness, we demonstrate the tolerance of the image segmentation to the time jitter. These results indicate that our proposed photonic SNN can achieve high-resolution processing of complex source images. Additionally, the time-multiplexing technique can be further adopted to simplify the RF structure, which is expected to reduce the complexity of the whole system, thus facilitating physical applications. Our work offers the prospect for a high-speed photonic spiking platform for image processing.


Domain Adaptation for In-Air to Underwater Image Enhancement via Deep Learning

Abstract: Underwater image enhancement has drawn a lot of attention due to its significance in ocean exploration. Since it is extremely hard to obtain a degraded image and its clear version in the same scene, most deep learning-based methods utilize synthetic datasets for training. However, synthetic datasets often fail to describe the natural appearance and show poor capabilities of generalization to real scenes. In this study, we present an in-air to underwater image enhancement (IUIE) framework that overcomes the limitations of underwater synthetic datasets. Specifically, the presented framework consists of a color correction step and a domain adaptation step. In the color correction step, we propose a color correction mechanism by utilizing the white balance algorithm to eliminate color bias and produce a more natural appearance. In the domain adaptation step, the network can reduce the domain gap between the in-air domain and the underwater domain and obtain a better generalization in real scenes. To accommodate spatially varying textures and edges, we propose a residual adaptive module in the domain adaptation network. Experimental results on the test data indicate that our IUIE framework not only obtains more remarkable UIQM and BRISQUE scores than the previous methods do but also achieves superior results in visual perception.


Visualization of object-oriented software in a city metaphor: Comprehending the implemented variability and its technical debt

Abstract: While many large-scale software systems intensively implement variability to reuse software and speed up development, they often do not document it, hampering its comprehension. This is especially the case for variability-rich object-oriented (OO) systems that heavily rely on existing OO mechanisms (i.e., inheritance, overloading and some patterns) to implement it in a single codebase. With no traceability information, the variability is buried in the codebase, hampering its identification, analysis, and understanding. While variability management becomes increasingly difficult over the system evolution, the implementation mechanisms also bring additional complexity to the codebase, which eventually leads to technical debt, threatening even more the software quality. In this article, we report on the design and evaluation of an extensible visualization, VariCity, that exhibits zones of high density of OO variability implementations. It relies on the city metaphor to represent the classes of the system as buildings whose dimensions are used to show variability metrics inherent to the implementation classes. They are linked together through streets depicting usage relationships and grouping in neighborhoods classes using each other. The extensibility of VariCity is demonstrated with VariMetrics, which highlights OO quality metrics on the buildings, revealing quality-critical classes concentrating variability implementations. We evaluate the visualization capacity to reveal zones concentrating variability implementations and being quality-critical by applying it to multiple variability-intensive open source software systems. We also report on a controlled experiment comparing the gain brought by the visualization to the use of an IDE.


Role Engine Implementation for a Continuous and Collaborative Multirobot System

Abstract: In situations involving teams of diverse robots, assigning appropriate roles to each robot and evaluating their performance is crucial. These roles define the specific characteristics of a robot within a given context. The stream of actions exhibited by a robot based on its assigned role are referred to as the process role. Our research addresses the depiction of process roles using a multivariate probabilistic function. The main aim of this study is to develop a role engine for collaborative multirobot systems and optimize the behavior of the robots. The role engine is designed to assign suitable roles to each robot, generate approximately optimal process roles, update them on time, and identify instances of robot malfunction or trigger replanning when necessary. The environment considered is dynamic, involving obstacles and other agents. The role engine operates hybrid, with central initiation and decentralized action, and assigns unlabeled roles to agents. We employ the Gaussian process (GP) inference method to optimize process roles based on local constraints and constraints related to other agents. Furthermore, we propose an innovative approach that utilizes the environment’s skeleton to address initialization and feasibility evaluation challenges. We successfully demonstrated the proposed approach’s feasibility, and efficiency through simulation studies and real-world experiments involving diverse mobile robots.


Privacy-Preserving Realization of Fuzzy Clustering and Fuzzy Modeling Through Vertical Federated Learning

Abstract: In this study, we elaborate on a realization of fuzzy clustering and the construction of fuzzy rule-based models on the basis of vertically partitioned datasets in a privacy-preserving federated learning approach. The main focus of the overall design process is to construct a family of information granules (clusters) and the corresponding fuzzy rules in the presence of a collection of vertically partitioned datasets without compromising data privacy. These datasets are composed of the same data but are described by different features, and due to security considerations, data cannot be shared. The vertical federated fuzzy clustering can be realized as an iterative optimization process composed of successive cycles: 1) computation (update) of the prototypes and partition matrices performed on the basis of local datasets and 2) an integration of the local sources of knowledge carried out on a central coordinator-server. The update of the partition matrices can be completed using a distance-based or gradient-based approach. The communication of findings between local clients and the coordinator-server is realized through exchanging partition matrices, which are more general than numeric data and can avoid leakage of data privacy. Fuzzy models are optimized in a similar manner through exchanging the gradients of the performance index computed with respect to the parameters between the clients and the global coordinator. The proposed mechanism exhibits significant originality since the realization of fuzzy modeling in a vertical federated learning environment has not been studied. Experimental studies show that the proposed federated clustering and fuzzy model design could effectively reveal the structure of the entire dataset and achieve high performance compared with the results obtained in a centralized manner.