Optical voice security scheme for anticounterfeiting

Abstract: In this work, we propose an optical voice security scheme for anticounterfeiting that uses voice watermark recording by an optical setup of parallel phase shifting digital holography (PPSDH) and an approach of converting the recorded voice into a suitable 2D voice watermark. The converted voice watermark is further encrypted in the optical domain in order to provide enhanced security in terms of physical keys for watermarking and other applications. The proposed scheme makes it difficult for counterfeiting in order to claim the ownership of voice or image data in case of watermarking. We present the experimental results of voice watermark recorded by optical microphone based on PPSDH and simulation results to verify the proposed optical security scheme for watermarking of optically recorded data. Best of our knowledge, this is the first proposal towards voice watermark generation in optical domain and the first demonstration of it for watermarking.


Ensemble of ghost convolution block with nested transformer encoder for dense object recognition

Abstract: The Technological advancement and innovation is happening at rapid pace and within the ambit of computerized era, the recognition models have displayed exceptionally outstanding performance in object detection but still the technology relies on ‘Dense-packed object detection”. This object detection remains one of the greatest challenges of the present situation due to redundant feature map calculation complexity, diverse shapes, and alignment of the object in various directions. This paper suggests a GCB (Ghost Convolution Block) based on nested-transformer encoder block in the feature refinement network to overcome these intricacies. The GCB effectively alleviates the redundant feature maps calculation complexity by using DW (Depth-Wise separable) convolution operation. Whereas, the nested-transformer encoder block extracts in-depth information from diverse shaped objects and misaligned objects based on the MHSA (Multi-Head-Self-Attention) mechanism's query, key, and value parameter information. We propose to perform quantitative evaluations on the VOC, GWHD (Global Wheat Head Detection), and SKU-110K data sets and carried out an ablation study by using the YOLOv5 model with GCB and GCBTR (Ghost Convolution Block-based Transformer) modules. We compared this model with conventional other models, our model achieved 84.2% mAP, 80% precision, 78.1% recall, and 79% F1-score on VOC, 81.8% mAP, 91.6% precision, 73.1% recall, and 81.3% F1-score on SKU-110K and also achieved 95.7% mAP, 94.4% precision, 90.2% recall on GWHD datasets. Thus, obtained reliable results prove that the suggested model has shown superior performance when compared with other existing models as well as the YOLOv5 model in detecting dense objects.


An objective quality indicator for evaluating filtering of microscopic video in CASA system

Abstract: Computer-Aided Sperm Analysis (CASA) has received great attention in the diagnosis of male reproductive health for its objectivity and accuracy. But its accuracy may be greatly affected by algorithms used in microscopic video preprocessing, especially algorithms for microscopic videos filtering. Owing to the Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) indicators are not suitable for microscopic images, objective indicators for evaluating filtering of microscopic videos are scarce at present, leading to filtering algorithms are chosen arbitrarily. In addition, it is critical to establish quality control indicators for CASA from the perspective of image processing, but they are rarely mentioned for the reason that algorithms adopted in image processing aren’t public. To reduce influences of filtering on diagnostic results, it is imperative to establish an objective quality control indicator for evaluating filtering of microscopic video from the perspective of image preprocessing. The principle of formatting sperm tracking trajectory and the rules of grading trajectory continuity were introduced firstly. Then, impacts of filtering algorithms on the continuity of sperm trajectory were investigated and the relation between the continuity of sperm trajectory and the fluctuation of sperm segmentation thresholds was further explored. Finally, an objective indicator for appraising the quality of filtering for microscopic video was established based on SADTM (Sum of Absolute Difference between Threshold and the Mean). The proposed indicator can objectively evaluate the quality of microscopic video filtering and provide a guideline for algorithm selection, laying foundation for quality control for CASA.


HTC-Net: A hybrid CNN-transformer framework for medical image segmentation

Abstract: Automated medical image segmentation is a crucial step in clinical analysis and diagnosis, as it can improve diagnostic efficiency and accuracy. Deep convolutional neural networks (DCNNs) have been widely used in the medical field, achieving excellent results. The high complexity of medical images poses a significant challenge for many networks in balancing local and global information, resulting in unstable segmentation outcomes. To address the challenge, we designed a hybrid CNN-Transformer network to capture both the local and global information. More specifically, deep convolutional neural networks are introduced to exploit the local information. At the same time, we designed a trident multi-layer fusion (TMF) block for the Transformer to fuse contextual information from higher-level (global) features dynamically. Moreover, considering the inherent characteristic of medical image segmentation (e.g., irregular shapes and discontinuous boundaries), we developed united attention (UA) blocks to focus on important feature learning. To evaluate the effectiveness of our proposed approach, we performed experiments on two publicly available datasets, ISIC-2017, and Kvasir-SEG, and compared our results with state-of-the-art approaches. The experimental results demonstrate the superior performance of our approach. The codes are available at https://github.com/Tanghui2000/HTC-Net.


How anti-gender and gendered imagery translate the Great Replacement conspiracy theory in online far-right platforms

Abstract: This article is based on a case study of online media practices of the Czech far-right group Angry Mothers, the biggest far-right Facebook group in the Czech political context in 2018. We show how the group used visual storytelling to translate the narratives of the Great Replacement conspiracy theory, which has become one of the most dominant narratives in European and US far-right discourses over recent years. The conspiracy theory was introduced in the book Le Grande Replacement by Renaud Camus in 2011 and claims that powerful Jewish elites use their financial resources to promote ‘Lesbian, gay, bisexual, transgender, queer, intersex, asexual + (LGBTQIA+) and gender ideologies’ and multiculturalism to destroy the white race, which will eventually die out. Based on theories of translation and visual storytelling, we demonstrate how the main tropes of its anti-Semitic narrative were diffused through the use of images stigmatising LGBTQIA+ people and other minorities in online communications of the Czech far-right group Angry Mothers.


Linear codes of larger lengths with Galois hulls of arbitrary dimensions and related entanglement-assisted quantum error-correcting codes

Abstract: Galois hulls are generalizations of Euclidean and Hermitian hulls, which have attracted interest because of their important applications in determining the complexity of some algorithms about linear codes and in constructing entanglement-assisted quantum error-correcting codes (EAQECCs). In this paper, the whole contribution is two-folded. On one hand, we propose explicit methods to construct linear codes of larger lengths with Galois hulls of arbitrary dimensions from given Galois self-orthogonal codes. Conditions required in our approach are proven to be relatively weak. On the other hand, we apply these results to construct EAQECCs. Two bounds for EAQECCs constructed from linear codes with prescribed dimensional Galois hull are given and EAQECCs with rates greater than or equal to 12 and positive net rates can be obtained. We also present many interesting examples to explain visually how these two aspects work.


Adaptive Sampling of 3D Spatial Correlations for Focus+Context Visualization

Abstract: Visualizing spatial correlations in 3D ensembles is challenging due to the vast amounts of information that need to be conveyed. Memory and time constraints make it unfeasible to pre-compute and store the correlations between all pairs of domain points. We propose the embedding of adaptive correlation sampling into chord diagrams with hierarchical edge bundling to alleviate these constraints. Entities representing spatial regions are arranged along the circular chord layout via a space-filling curve, and Bayesian optimal sampling is used to efficiently estimate the maximum occurring correlation between any two points from different regions. Hierarchical edge bundling reduces visual clutter and emphasizes the major correlation structures. By selecting an edge, the user triggers a focus diagram in which only the two regions connected via this edge are refined and arranged in a specific way in a second chord layout. For visualizing correlations between two different variables, which are not symmetric anymore, we switch to showing a full correlation matrix. This avoids drawing the same edges twice with different correlation values. We introduce GPU implementations of both linear and non-linear correlation measures to further reduce the time that is required to generate the context and focus views, and to even enable the analysis of correlations in a 1000-member ensemble.


PoSE: Suppressing Perceptual Noise in Embodied Agents for Enhanced Semantic Navigation

Abstract: Embodied agents navigating unknown environments face the challenge of optimizing exploration based on semantic information. Conventional methods, reliant on collected data or pre-defined rules, have limitations in scalability and applicability, while pretrained language models based methods focusing on textual modality encounter perceptual noise, which affects decision-making. To mitigate these problems, this paper presents Prompt-based Vision Context Semantic Exploration (PoSE), an innovative method that leverages prior knowledge from vision language models (VLMs) to suppress perceptual noise. Through its unique design of prompts that use existential logic, PoSE reduces misidentifications of target objects within the observed environment. It also introduces a unique exploration map to translate target locations into exploration coordinates. The proposed methodology is evaluated on the ALFRED benchmark, demonstrating improved performance against previous rulebased and task-specific data-driven exploration policies. Furthermore, PoSE's semantic exploration policy offers enhancements over pretrained language-model based exploration methods that focus on text modality, showcasing its effectiveness and generality.


Heterogeneous Graph Contrastive Learning With Metapath-Based Augmentations

Abstract: Heterogeneous graph contrastive learning is an effective method to learn discriminative representations of nodes in heterogeneous graph when the labels are absent. To utilize metapath in contrastive learning process, previous methods always construct multiple metapath-based graphs from the original graph with metapaths, then perform data augmentation and contrastive learning on each graph respectively. However, this paradigm suffers from three defects: 1) It does not consider the augmentation scheme on the whole metapath-based graph set, which hinders them from fully leveraging the information of metapath-based graphs to achieve better performance. 2) The final node embeddings are not optimized from the contrastive objective directly, so they are not guaranteed to be distinctive enough. It leads to suboptimal performance on downstream tasks. 3) Its computational complexity for contrastive objective is high. To tackle these defects, we propose a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H eterogeneous <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G raph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C ontrastive learning model with <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M etapath-based <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A ugmentations ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HGCMA ), which is designed for downstream tasks with a small amount of labeled data. To address the first defect, both semantic-level and node-level augmentation schemes are proposed in our HGCMA for augmentation, where a metapath-based graph and a certain ratio of edges in each metapath-based graph are randomly masked, respectively. To address the second and third defects, we utilize a two-stage attention aggregation graph encoder to output final node embedding and optimize them with contrastive objective directly. Extensive experiments on three public datasets validate the effectiveness of HGCMA when compared with state-of-the-art methods.


Alternatives for the q-matroid axioms of independent spaces, bases, and spanning spaces

Abstract: It is well known that in q-matroids, axioms for independent spaces, bases, and spanning spaces differ from the classical case of matroids, since the straightforward q-analogue of the classical axioms does not give a q-matroid. For this reason, a fourth axiom has been proposed. In this paper we show how we can describe these spaces with only three axioms, providing two alternative ways to do that. As an application, we show direct cryptomorphisms between independent spaces and circuits and between independent spaces and bases.


EsArCost: Estimating repair costs of software architecture erosion using slice technology

Abstract: Software architecture erosion has a negative effect on software quality, software performance and evolution cost, so repairing architecture erosion is an important task. However, in the actual evolution process, due to the pressure of cost, it is not practical to repair all erosion problems. To repair more erosion problems at a certain total cost, developers would better know the appropriate repair cost of each erosion problem. In this paper, we propose an approach called EsArCost, which can locate the reasons for architecture erosion and estimate the repair cost of each erosion problem. To this end, EsArCost detects multi-level changes of software architecture and calculates the architecture erosion degree, then locates which changed codes cause architecture erosion. EsArCost further calculates the full slice of each erosion problem to estimate the difficulty and repair costs. We evaluate our approach on small and large open source programs, and the experiment results indicate that EsArCost can effectively and efficiently estimate repair costs.


Underwater image enhancement method based on golden jackal optimization

Abstract: The propagation of light underwater is affected by water absorption and scattering, resulting in poor visibility of obtained underwater images, and exhibiting degradation phenomena such as color deviation and low contrast. To improve the quality and visibility of underwater images, a method of underwater image enhancement based on Golden Jackal optimization is proposed. The algorithm consists of two processing stages. The first stage is mainly processed in the RGB color space. Based on the attenuation characteristics of underwater color channels, a constructed color compensation and correction strategy is used to eliminate color deviation caused by water absorption. In second stage, it is mainly processed in the Lab color space, using the Golden Jackal optimization algorithm and fitness function to search the optimal parameters of transformation function to improve the image contrast by optimizing the grey intensity of L component and its histogram distribution. The feasibility and effectiveness of the proposed method have been verified through testing and analysis of much underwater degraded image data. In subjective and objective analysis with multiple state-of-the-art underwater image clarity methods, the proposed method performs excellent in color restoration and detail texture enhancement. Further experiments have shown that the proposed method has strong robustness and universality. It is not only suitable for enhancing various underwater scene images, but also for enhancing environmental images with fog, low lighting, and sandstorms. Meanwhile, the proposed method can effectively improve the performance of underwater image segmentation, keypoint detection, and saliency detection.


Leveraging Machine Learning for Generating and Utilizing Motion Primitives in Adversarial Environments

Abstract: Motion primitives enable fast planning for complex and dynamic environments. Adversarial environments pose a particularly challenging and unpredictable scenario. Motion-primitive-based planners have the potential to provide benefit in these types of environments. The key challenge is to design a library of maneuvers that effectively capture the necessary capabilities of the vehicle. This work presents a primitive-based game tree search to solve adversarial games in continuous state and action spaces and applies a reinforcement learning framework to autonomously generate effective primitives for the given task. The results demonstrate the ability of the learning framework to produce maneuvers necessary for competing against adversaries. Furthermore, we propose a method for learning a model to estimate the state-dependent value of each motion primitives and demonstrate how to incorporate this model to increase planning performance under time constraints. Additionally, we compare our primitive-based algorithm against forward simulated methods from existing literature and highlight the benefits of motion primitives.


MFI-YOLO: Multi-fault insulator detection based on an improved YOLOv8

Abstract: Insulators are essential components in power transmission lines. Due to the harsh variations in bad environments, insulators may experience faults. Detecting these insulator faults promptly and effectively is an urgent issue. To rapidly and accurately locate insulators and their faulty regions in aerial images of insulators with complex backgrounds and varying fault sizes, this paper proposes an improved YOLOv8 algorithm for the detection of multiple insulator fault types (MFI-YOLO). This algorithm achieved target feature extraction in complex background images by replacing the C2F network constructed by fusing the GhostNet and multi-scale asymmetric convolution (MSA-GhostBlock). Furthermore, in the feature fusion stage, a multi-scale feature fusion structure called ResPANet, based on residual skip connections, was constructed to replace the PANet. This enhancement aims to improve the network detection accuracy in multi-target scenarios. Finally, to evaluate the algorithm's performance, this study constructed a target detection dataset containing four types of insulators: normal, self-explosive, damaged, and flashover. Experimental results indicate that, compared to the original model, the improved model has increased mean accuracy from 89.2% to 93.9%. The designed model exhibits high detection accuracy in the insulator and its three fault categories, especially for some hard-to-detect fitting.


Reading compound words in Finnish and Chinese: An eye-tracking study

Abstract: Two eye-tracking experiments in alphabetic Finnish and two in logographic Chinese examined the recognition of two-constituent compound words in reading. In Finnish, two-constituent compound words vary greatly in length, whereas in Chinese they are identical in length. According to the visual acuity principle (Bertram & Hyönä, 2003), short Finnish compound words and all two-character Chinese compound words that fit in foveal vision are recognized holistically, whereas long Finnish compound words are recognized via components. Experiment 1 in Finnish provided evidence consistent with the account, whereas the results for long compound words presented in condensed font in Experiment 2 were inconsistent with it. In Chinese, the first-character frequency effect was non-significant even when the compound words were presented in large font. The Finnish results suggest that componential processing is necessary when the compound word entails more than 10 letters. The Chinese results are compatible with the Chinese Reading Model (Li & Pollatsek, 2020) that assumes whole-word representations to overrule the activation of components during compound word recognition.


Deep learning-based software bug classification

Abstract: Accurate classification of bugs can help accelerate the bug triage process, code inspection, and repair activities. In this context, many machine learning techniques have been proposed to classify bugs. The expressive power of deep learning could be used to further improve classification. We propose a novel deep learning-based bug classification approach. We first build a bug taxonomy with eight bug classes, each characterized by a set of keywords. Subsequently, we heuristically annotate a moderately large set (∼1.36M) of software bug resolution reports using an earth-mover distance technique based on the keywords. Finally, we use four attention-based classification techniques to classify these curated bugs. Our experiments on a carefully collected dataset indicate that our proposed technique achieved a mean F1-Score of 84.78% and a mean macro-average ROC of 98.25%. Our proposed approach was observed to outperform the existing techniques by 16.88% on an average in terms of F1-Score for the considered dataset.


Complex amplitude extraction from a set of interferograms by simultaneous modulation of amplitude and phase steps

Abstract: The measurement of an optical field arises in various fields of science and engineering, including holography, crystallography, microscopy, and many others. Nowadays, there are several methods to estimate amplitude and phase including Shack-Hartman wavefront sensor, algorithms type Gerchberg-Saxton, algorithm-based on the transport of irradiance equation, holographic, interferometric, among others. This paper introduces an interferometric method for retrieving the complex amplitude from a set of interferograms modulated simultaneously in visibility and phase steps. Instead of phase only as the method of so-called phase-shifting interferometry. This approach is based on the modulation of amplitude and phase steps and is presented for the case of synchronous detection and coherence light. It is validated with a numerical noise study, turning out to be accurate and robust to detuning errors, with low distortion, and a very high signal-to-noise. The experiment is carried out on a Mach–Zehnder interferometer adapted to modulate the reference amplitude, and the phase steps by modulation of polarization, instead of phase steps only as in phase-shifting interferometry.


Unawareness detection: Discovering black-box malicious models and quantifying privacy leakage risks

Abstract: Because machine learning models, especially black-box malicious models vulnerable to attribute inference attacks, are capable of generating a great deal of privacy leakage, recent work has focused on assessing these models in an attempt to prevent unexpected attribute privacy leakage. While there has been some success at model privacy risk evaluations, these traditional solutions are almost brittle in practice because they not only require white-box access to obtain model feature layer outputs but also their evaluation results are heavily influenced by the training dataset and the model structure, leading to difficulty in generalization. In this paper, we propose a novel unawareness detection mechanism for discovering black-box malicious models and quantifying potential unawareness privacy leakage risk along with machine learning models to overcome the two limitations. A new method for quantifying the privacy risk caused by a specific loss function has been proposed to mitigate the impact of the training dataset and the model structure. A new evaluation model has also been proposed that uses Matthew's correlation coefficient score as a new metric and the final output of the target model as a new input. In addition, the theoretical upper bound of the model privacy risk has also been given a mathematical formula that is positively correlated to the mutual information between the sensitive attributes and the target model outputs. Compared with traditional detection methods, our evaluation model reduces the requirement for model access and minimizes evaluation errors caused by data imbalance, and our privacy risk assessment method and theoretical upper bounds on privacy risk can be applied to a broader range of datasets and target model structures. The experimental results show that the adversary's prediction capability is affected by the distribution of datasets and the level of malicious intent of the model, which is consistent with the theoretical prediction, and the detecting method can find potential model privacy leaks in public datasets UTKFace and FairFace.


Numerical model study on influences of photovoltaic plants on local microclimate

Abstract: The construction of large-scale photovoltaic plants will inevitably change the local environment. We consider that the influences of such increasing large-scale photovoltaic installations on radiation, temperature, wind profile and other factors within the region can be described in a layer of photovoltaic canopy. This paper improves the UCRC-Solar model for photovoltaic plants, with reference to the calculation of sensible heat flux of the underlying forest surface. Model verification was done with measured data from the Red Rock photovoltaic plant, Arizona, USA. In this study, it indicates that photovoltaic plant has locally generated heat island effects in daytime. The calculation of local microclimate changes before and after the construction of the photovoltaic plants and, the impact of photovoltaic coverage finds an increase in the temperature daily range and a reduction in the ground temperature by about 5 °C. Furthermore, a 15 % increase in photovoltaic coverage increased the daily 3 m air temperature difference by 0.55 °C and reduced the ground temperature by 3.6 °C in the day and 1.1 °C at night, respectively. In the future, rain and snow processes would be considered to further explore the microclimate impact of photovoltaic plants.


Real-time Multi-CNN based Emotion Recognition System for Evaluating Museum Visitors’ Satisfaction

Abstract: Conventional studies on the satisfaction of museum visitors focus on collecting information through surveys to provide a one-way service to visitors, and thus it is impossible to obtain feedback on the real-time satisfaction of visitors who are experiencing the museum exhibition program. In addition, museum practitioners lack research on automated ways to evaluate a produced content program's lifecycle and its appropriateness. To overcome these problems, we propose a novel multi-convolutional neural network, called VimoNet, which is able to recognize visitors emotions automatically in real-time based on their facial expressions and body gestures. Furthermore, we design a user preference model of content and a framework to obtain feedback on content improvement for providing personalized digital cultural heritage content to visitors. Specifically, we define seven emotions of visitors and build a dataset of visitor facial expressions and gestures with respect to the emotions. Using the dataset, we proceed with feature fusion of face and gesture images trained on the DenseNet-201 and VGG-16 models for generating a combined emotion recognition model. From the results of the experiment, VimoNet achieved a classification accuracy of 84.10%, providing 7.60% and 14.31% improvement, respectively, over a single face and body gesture-based method of emotion classification performance. It is thus possible to automatically capture the emotions of museum visitors via VimoNet, and we confirm its feasibility through a case study with respect to digital content of cultural heritage.


Stealthy Attack on Remote Control System With Local Controller and Its Countermeasures

Abstract: Cyber–physical systems (CPSs) driven by a local controller and a remote controller have been gaining significant research interest in recent years due to its application scenarios in practice, such as unmanned aerial vehicles (UAVs). In this article, we consider the security issue in the remote control system with a local controller under stealthy attacks. Under this framework, one controller is designed locally based on the limited measurements collected by a local sensor, and the other controller is designed remotely and is transmitted to the actuator through a wireless communication channel, which may suffer malicious attacks due to its openness. To defend attacks on remote control signal, the K–L divergence-based detector or <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\chi^{2}$</tex-math> </inline-formula> detector is often adopted. However, there may be attackers adopting stealthy attacks, which can bypass such detectors. Therefore, we analyze the existence of such attacks, and analytically characterize the worst-estimation performance degradation induced by the remote control signal attack. Further, we construct the optimal attack signal to achieve the upper bound on the estimation performance degradation. In addition, we also give countermeasures against such stealthy attacks. Simulations are provided to illustrate the proposed results.


Measurement Combination Estimator for Multisensor Extended Object Tracking Using Random Matrix

Abstract: For extended object tracking (EOT), the random-matrix approach has received extensive attention. It is able to jointly estimate the kinematic state and extension of an extended object. Multiple sensors with different characteristics and perspectives can provide new information especially for EOT. However, how to fuse the measurements from multiple sensors to guarantee the improvement of the estimation accuracy of the state and extension still needs further research. This paper proposes a random-matrix approach to elliptical EOT using multiple sensors based on a centralized fusion architecture. In order to handle the different distortions of multiple sensors, a measurement combination (MC) approach is firstly proposed. As a result, a likelihood function with a proper form is obtained, and it can also pre-process the multi-sensor measurements and take the scattering matrix of measurement means into consideration. Based on the likelihood function, the measurement combination estimator (MCE) is derived for multi-sensor EOT in a Bayesian framework. It inherits the analytical and concise scheme of the single-sensor EOT approach. Based on the proposed MCE approach, it is theoretically proved that the estimation accuracy of the state and extension can be improved when the sensor number increases. Effectiveness of the MCE approach is demonstrated by simulation and experimental results compared with the existing multi-sensor EOT approaches using random matrix.


Up- and down-correlations in normal variance mixture models

Abstract: We study conditional correlations in normal variance mixture models. For several families, we determine explicit formulas for up-(down-)correlations defined as the correlation between the sum of risks and an individual component, conditionally on the sum being large (small).


Adaptive phase measuring profilometry for robustly detecting saturated pixels

Abstract: In this paper, we propose an adaptive method for phase measuring profilometry to more accurately reconstruct 3D point clouds of high-reflective surfaces. First, we project high-frequency sinusoidal fringe patterns to robustly detect saturated pixels and then obtain their corresponding optimal projection intensity. Second, based on the coordinate mapping relationship between a camera-projector pair, we generate a set of fringe patterns with adaptive intensity. Finally, we project the adaptive fringes patterns to achieve more accurate 3D reconstruction. The experimental results show that our method can more accurately detect saturated pixels, and thus improve the quality of 3D reconstruction.


Improved ShuffleNetV2 for Action Recognition in BPPV Treatment

Abstract: In clinical examination and repositioning for BPPV(benign paroxysmal positional vertigo), analyzing the head position trajectory at the algorithm level to obtain information on each examination and repositioning posture helps to provide guidance for subsequent treatment, which has important practical value in clinical applications. This paper uses the improved ShuffleNetV2 on the basic the data obtained by the swivel chair for action recognition, and locates the interval where the action occurs. Specific network improvements include: the introduction of the SE channel attention block, which improves the performance of the network with a small amount of parameters and computation; the Hardswish activation function is used instead of ReLU in the original network; by reducing the layers in the network, deleting the second 1*1 convolution in the right branch of the downsampling block, and changing the kernel size of the DW convolution, the network can be lightweight while maximizing the capability of action recognition. The network’s performance has been enhanced, resulting in an accuracy of 98.53% with only 71,444 parameters and 15.76M FLOPs, while maintaining a recognition speed of 97.1 images per second. It achieves a good balance between speed and accuracy, enabling it to effectively accomplish the task of action recognition.


Reproducible experiments for generating pre-processing pipelines for AutoETL

Abstract: This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Dıaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied.


Compressive hyperspectral imaging based on Images Structure Similarity and deep image prior

Abstract: In this paper, the structural similarity between RGB image and spectral image is studied, and a non-iterative Images Structure Similarity(ISS) method for fast reconstruction of spectral image is proposed. At the same time, the input of the Deep Image Prior (DIP) method is optimized for the first time by using the initial spectral data reconstructed by ISS. It raises the starting value of the iteration. Specifically, we take the RGB image data as the base of the spectral data, and solve the base coefficient by the least square method to quickly estimate the initial hyperspectral image. The Gaussian noise data is replaced by the estimated initial spectral data to constrain the solution space of the network and reduce the number of iterations. Finally, the structural similarity between RGB image and spectral image is used. The RGB three-channel graph is used to filter the iterative results to improve the reconstruction quality. Experimental results show that compared with other hyperspectral imaging methods, the proposed method can improve the quality of reconstruction in both spectral resolution and spatial resolution. In addition, compared with other methods based on Deep Image Prior (DIP), our improvement greatly reduces the reconstruction time and is more suitable for actual snapshot spectral imaging.


The Hybrid Features and Supervised Learning for Batik Pattern Classification

Abstract: Several countries have traditional textiles as a piece of their cultural heritage. Indonesia has a traditional textile called batik. Central Java is one of the regions producing batik known for its variety of distinctive themes. It has unique designs and several motifs that emphasize the beauty of historic sites. Since the diversity of central java batik motifs and the lack of knowledge from the surrounding community, only a select group of people, especially the batik craftsmen themselves, can recognize these motifs. Consequently, the method to identify the batik according to the primary ornament pattern is required. Therefore, this study proposes a computer vision-based method for classifying batik patterns. The proposed method required discriminating appropriate features to produce optimal results. The discriminating features were constructed based on color, shape, and texture. Those features were derived using the method of Color Moments, Area Based Invariant Moments, Gray Level Co-occurrence Matrix (GLCM), and Local Binary Pattern (LBP). This study’s proposed hybrid features were formed based on the most discriminating and appropriate features. These were yielded by the Correlation-based feature selection (CFS) method. The hybrid features were then fed into several classifiers to determine the batik pattern. The pattern consists of ten classes: Asem Arang, Asem Sinom, Asem Warak, Blekok, Blekok Warak, Gambang Semarangan, Kembang Sepatu, Semarangan, Tugu Muda, and Warak Beras Utah. Based on the experimental results, the most optimal predicted class of the batik pattern was generated using the Artificial Neural Network (ANN) classifier. It was indicated by achieving an accuracy value of 99.76% based on the 3,000 images (each class consists of 300 images) with cross-validation using a k-fold value of 10. This study has proved that the hybrid features incorporated with ANN can be selected as a suitable model to classify the batik patterns.


Privacy Protection for Marginal-Sensitive Community Individuals Against Adversarial Community Detection Attacks

Abstract: Social networks especially the social communities facilitate the rapid and rich social activities of all individuals in the world. However, advanced community detection brings serious privacy disclosure (e.g., whether two important and sensitive individuals belong to the same community?) to us. For instance, online plainclothes policemen can be regarded as marginal community users who are often in the same community initially and in need of penetrating into as many as possible different communities to collect illegal evidence of network criminals, but adversarial community inference which can maliciously disclose the sensitive user relationships within a target community will expose their privacy and lead to task failure. Thus, privacy protection for the marginal community users becomes an urgent issue which is still open so far. In this work, we aim to study the community privacy protection for target marginal individuals of a community against multiple adversarial community detection (ACD) attacks. First, we define the marginal community user hiding problem and propose a marginal user pair selection strategy. Second, to enhance the privacy effectiveness of conventional methods, we propose a deep graph learning approach to maximally find the minimum link perturbation cost. Finally, we conduct various community detection attacks on many real social graphs, and the experimental results show that our method can more effectively hide the marginal-sensitive user pairs than baselines.


Pedestrian Re-Identification Algorithm Based on Attention Pooling Saliency Region Detection and Matching

Abstract: The recognition accuracy of the pedestrian re-identification algorithm is affected by factors such as the scale, posture, occlusion level, and appearance of a pedestrian in an image. These factors bring great challenges to the further development of deep learning theory in the field of person re-identification. On the one hand, existing pedestrian re-identification model cannot accurately extract attention information containing contextual information and discriminative pedestrian features. On the other hand, images of the same pedestrian taken with different cameras have the problem of different degrees of occlusion. Therefore, we introduce an attention pooling mechanism, which allows the model to automatically focus on discriminative regions of pedestrian images. The model can then learn attention maps by automatically focusing on visually salient pedestrian regions. It can effectively address the influence of factors on the accuracy of pedestrian re-identification. Furthermore, we propose a pedestrian re-identification method based on saliency region detection and matching. The method first extracts multiple local saliency regions from pedestrian images using the pedestrian joint point detection method, then marks and extracts the positions of the abovementioned saliency regions, and finally performs matching and sorting. It can correct the impact caused by occlusion on the accuracy of pedestrian re-identification. We conduct tests and experiments on three public person re-identification datasets, and the results show that our method not only achieves the best recognition accuracy on the aforementioned datasets but also has better robustness.


Flexible Dual-Branch Siamese Network: Learning Location Quality Estimation and Regression Distribution for Visual Tracking

Abstract: Anchor-free based trackers introduce an extra branch in addition to classification and regression branches in the network to achieve comparable performance with anchor-based trackers. This extra branch is usually trained independently in the training phase and is used in combination with other branches in the inference phase. However, this can increase the inconsistency between the inference phase and the training phase, potentially degrading the tracking performance. To address this problem, we propose a new Siamese network-based object tracking framework that eliminates this inconsistency by unifying classification and additional branch tasks to achieve learning location quality estimation. Furthermore, regression tasks for bounding boxes are widely formulated based on Dirac <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\delta$</tex-math> </inline-formula> distribution. Though this assumption works well for many scenarios, it restricts the prediction of regression branches. To overcome this restriction, we propose discretizing the continuous offset of the regression branch into multiple offset predictions, which enables the network to learn more flexible distributions automatically. Meanwhile, the discrete distribution prediction of regression branches is utilized to further guide the classification of the trackers. Extensive experiments on the widely accepted benchmarks demonstrate the effectiveness and efficiency of the proposed model.


Complaint and Severity Identification From Online Financial Content

Abstract: The automatic detection of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">financial complaints ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FINCORP ) can benefit businesses and online merchants. Compared with manually tagged complaints, they can use this information to monitor and address issues and effectively route them to appropriate teams. This can also promote greater transparency and accountability when dealing with consumer financial products and services, strengthening the firm’s brand value. In linguistic studies, complaints have been classified into severity categories based on the level of risk the complainant is prepared to accept. Furthermore, since emotions influence every speech act, an individual’s emotional state considerably impacts the complaint expression. In this article, we introduce a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FINCORP resource, a collection of annotated complaints arising between financial institutions and consumers expressed in English on Twitter. The dataset has been enriched with the associated emotion, sentiment, and complaint severity classes. The dataset comprises 3149 complaint and 3133 noncomplaint instances spanning over ten domains (e.g., credit cards, mortgages). For a comprehensive evaluation of our dataset, we develop a multitask framework for complaint detection and severity classification with emotion recognition (ER) and sentiment classification as the additional tasks and compare it with several existing baselines. The corpus and code are available here: https://github.com/RohanBh23/FINCORP.


Supervised Local Training With Backward Links for Deep Neural Networks

Abstract: The restricted training pattern in the standard BP requires end-to-end error propagation, causing large memory costs and prohibiting model parallelization.Existing local training methods aim to resolve the training obstacles by completely cutting off the backward path between modules and isolating their gradients.These methods prevent information exchange between modules and result in inferior performance.This work proposes a novel local training algorithm, BackLink, which introduces intermodule backward dependency and facilitates information to flow backward along with the network.To preserve the computational advantage of local training, BackLink restricts the error propagation length within the module.Extensive experiments performed in various deep convolutional neural networks demonstrate that our method consistently improves the classification performance of local training algorithms over other methods.For example, our method can surpass the conventional greedy local training method by 6.45% in accuracy in ResNet32 classifying CIFAR100 and recent work by 2.58% in ResNet110 classifying STL-10 with much lower complexity, respectively.Analysis of computational costs reveals that small overheads are incurred in GPU memory costs and runtime on multiple GPUs.Our method can lead up to a 79% reduction in memory cost and 52% in simulation runtime in ResNet110 compared to the standard BP.Therefore, our method could create new opportunities for improving training algorithms towards better efficiency for real-time learning applications.Impact Statement-Deep learning models are generally bulky with abundant parameters, demanding long training time and excessive computing power.It poses a great challenge for deploying these models in real-world applications.This study looks into the intrinsic limitation of the standard training algorithm.The proposed method increases information flow in the algorithm and improves image classification accuracy over conventional methods.It can substantially reduce memory cost and hardware runtime by 50%.With the proposed method, an intelligent system can be readily deployed on mobile devices and function to meet practical demands.It also enables computing systems to reduce excessive energy waste and cut down carbon footprints for deep learning applications, such as traffic control and autonomous driving.


Knowledge Graph-Based Reinforcement Federated Learning for Chinese Question and Answering

Abstract: Knowledge question and answering (Q&A) is widely used. However, most existing semantic parsing methods in Q&A usually use cascading, which can incur error accumulation. In addition, using only one institution’s Q&A data definitely will limit the Q&A performance, while data privacy prevents sharing between institutions. This article proposes a knowledge graph-based reinforcement federated learning (KGRFL)-based Q&A approach to address these challenges. We design an end-to-end multitask semantic parsing model [MSP-bidirectional and auto-regressive transformers (BART)] that identifies question categories while converting questions into SPARQL statements to improve semantic parsing. Meanwhile, a reinforcement learning (RL)-based model fusion strategy is proposed to improve the effectiveness of federated learning, which enables multi-institution joint modeling and data privacy protection using cross-domain knowledge. In particular, it also reduces the negative impact of low-quality clients on the global model. Furthermore, a prompt learning-based entity disambiguation method is proposed to address the semantic ambiguity problem because of joint modeling. The experiments show that the proposed method performs well on different datasets. The Q&A results of the proposed approach outperform the approach of using only a single institution. Experiments also demonstrate that the proposed approach is resilient to security attacks, which is required for real applications.


Synergetic Focal Loss for Imbalanced Classification in Federated XGBoost

Abstract: Applying sparsity- and overfitting-aware eXtreme Gradient Boosting (XGBoost) for classification in federated learning allows many participants to train a series of trees collaboratively. Since various local multiclass distributions and global aggregation diversity, model performance plummets as convergence slowly and accuracy decreases. Worse still, neither the participants nor the server can detect this problem and make timely adjustments. In this paper, we provide a new local-global class imbalance inconsistency quantification and utilize softmax as the activation and focal loss, a dynamically scaled cross-entropy loss, in federated XGBoost to mitigate local class imbalance. Moreover, we propose a simple but effective hyperparameter determination strategy based on local data distribution to adjust the sample weights among non-communicating participants, synergetic focal loss, to solve the inconsistency of local and global class imbalance, a unique characteristic of federated learning. This strategy is perfectly integrated into the original classification algorithm. It requires no additional detectors or information transmission. Furthermore, a dynamical for loop is designed to capture an optimum hyperparameter combination. Finally, we conduct comprehensive tabular- and image-based experiments to show that synergetic focal loss used in federated XGBoost achieves faster convergency and significant accuracy improvement. Simulation results prove the effectiveness of the proposed principle of configuring sample weights.


Multidocument Aspect Classification for Aspect-Based Abstractive Summarization

Abstract: Multidocument aspect-based summarization (AspSumm) aims to generate focused summaries based on the target aspects from a cluster of relevant documents. Generating such summaries can better satisfy readers’ specific points of interest, as readers may have different concerns about the same articles. However, previous methods usually generate aspect-based summaries based on the given aspects without using the relationship among aspects to assist in the summarization. In this work, we propose a two-stage general framework for multidocument AspSumm. The model first discovers the latent relationship among aspects and then uses relevant sentences selected by aspect discovery to generate abstractive summaries. We exploit latent dependencies among aspects using a tag mask training (TMT) strategy, which increases the interpretability of the model. In addition to improvements in summarization over aspect-based strong baselines, experimental results show that our proposed model can accurately discover multidomain aspects on the WikiAsp dataset.


A multiview clustering framework for detecting deceptive reviews

Abstract: Online reviews, which play a key role in the ecosystem of nowadays business, have been the primary source of consumer opinions. Due to their importance, professional review writing services are employed for paid reviews and even being exploited to conduct opinion spam. Posting deceptive reviews could mislead customers, yield significant benefits or losses to service vendors, and erode confidence in the entire online purchasing ecosystem. In this paper, we ferret out deceptive reviews originated from professional review writing services. We do so even when reviewers leverage a number of pseudonymous identities to avoid the detection. To unveil the pseudonymous identities associated with deceptive reviewers, we leverage the multiview clustering method. This enables us to characterize the writing style of reviewers (deceptive vs normal) and cluster the reviewers based on their writing style. Furthermore, we explore different neural network models to model the writing style of deceptive reviews. We select the best performing neural network to generate the representation of reviews. We validate the effectiveness of the multiview clustering framework using real-world Amazon review data under different experimental scenarios. Our results show that our approach outperforms previous research. We further demonstrate its superiority through a large-scale case study based on publicly available Amazon datasets.


Detecting Fraudulent Student Communication in a Multiple Choice Online Test Environment

Abstract: Online evaluation systems, pervasive nowadays, are known to be susceptible to higher fraud risks. This work proposes a novel and robust method to detect potential fraud acts in online multiple-choice question (MCQ) exams. For the first time, the communication probability between the examinees is statistically assessed based on the concordance of responses and answer time against null expectations and is subsequently used to identify potential fraud behavior. The model is sensitive to the direction of communication acts, distinguishing content consumption from production, as well as multiwise communication channels. Online remote tests from engineering courses at Técnico Lisboa are used as a case study. We show that the cumulative contribution of concordant responses between students, when recurrent, offers a way of signaling fraud behavior. Separating content production from consumption reveals the underlying student role played in potential fraud acts. Collusion behavior is assessed against null models of fraud and conformity, and therefore being statistically framed and offering a solid criterion to guide tutors in ascertaining fraud and discouraging communication.


Mixed-Up Experience Replay for Adaptive Online Condition Monitoring

Abstract: Data-driven predictive maintenance reduces manufacturing downtime, and complex process-sensing relationships encourage the use of Deep Learning to automatically extract features. However, labeled training data is often lacking, and novel fault conditions may occur. Practical deployments must learn from unlabeled data, adapt to emerging conditions, and do so without prior knowledge of when the condition changes. Combining state-of-the-art Self-Supervised Learning (SSL) with Continual Learning (CL) facilitates adaptation as new conditions are observed. This study proposes a framework for adaptive online condition monitoring based on Barlow Twins SSL and novel Mixed-Up Experience Replay (MixER) for unsupervised CL. Tailored for 1D sensing data, Barlow Twins effectively clusters unlabeled data. When combined with MixER, the system outperforms state-of-the-art unsupervised CL on a motor health condition data set, reaching 92.4% classification accuracy. Future work will demonstrate human-in-the-loop integration for real manufacturing environments.


An Adaptive Semantic Mining Framework for Heterogeneous Information Network Embedding

Abstract: Heterogeneous information network (HIN) embedding aims to map heterogeneous nodes to the low-dimensional vector space. The existing embedding models cannot determine the optimal length of semantics automatically and reveal full semantic information adaptively for different heterogeneous networks. To address this challenge, an HIN embedding model with adaptive semantic mining is proposed. First, we project heterogeneous nodes into the same space and aggregate the features of target types in the first-degree range. Then, the semantics of different node types is combined through the attention mechanism, and latent meta-paths are mined using the attention coefficients. Finally, multiple feature aggregation layers are stacked with residual blocks. The residual weights control the proportion of semantics transferred between layers to aggregate more distal features selectively. In addition, we designed the Selected DropLink unit to remove links which transfer negative information, which can further improve the resistance of model to over-smoothing. Experiments show that our model can obtain more accurate embedding results and can automatically mine complex semantic connections between heterogeneous nodes without prior definition of meta-path and semantic depth.


FBSE-Based JPEG Image Compression

Abstract: The purpose of image compression is to minimize the storage space required to store images and maintain the relevant information in them. In this article, we presented a Fourier-Bessel series expansion (FBSE) based image compression technique, which is motivated by the Joint Photographic Experts Group (JPEG), which uses discrete cosine transform (DCT) in the process. FBSE of order-0 has structural similarity with DCT and provides real representation for real signals. The proposed methodology has used FBSE in place of DCT for image compression. The article compares the compression ratio of both the methodology and results have shown better compression performance of FBSE-based image compression.


Code Review Automation: Strengths and Weaknesses of the State of the Art

Abstract: The automation of code review has been tackled by several researchers with the goal of reducing its cost. The adoption of deep learning in software engineering pushed the automation to new boundaries, with techniques <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">imitating developers in generative tasks, such as commenting on a code change as a reviewer would do or addressing a reviewer’s comment by modifying code. The performance of these techniques is usually assessed through quantitative metrics, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g. , the percentage of instances in the test set for which correct predictions are generated, leaving many open questions on the techniques’ capabilities. For example, knowing that an approach is able to correctly address a reviewer’s comment in 10% of cases is of little value without knowing what was asked by the reviewer: What if in all successful cases the code change required to address the comment was just the removal of an empty line? In this paper we aim at characterizing the cases in which three code review automation techniques tend to succeed or fail in the two above-described tasks. The study has a strong qualitative focus, with ~105 man-hours of manual inspection invested in manually analyzing correct and wrong predictions generated by the three techniques, for a total of 2,291 inspected predictions. The output of this analysis are two taxonomies reporting, for each of the two tasks, the types of code changes on which the experimented techniques tend to succeed or to fail, pointing to areas for future work. A result of our manual analysis was also the identification of several issues in the datasets used to train and test the experimented techniques. Finally, we assess the importance of researching in techniques specialized for code review automation by comparing their performance with ChatGPT, a general purpose large language model, finding that ChatGPT struggles in commenting code as a human reviewer would do.


FAPP: Extremely Fast Approach to Boosting Image Matching Precision

Abstract: Feature matching is a critical problem in the field of computer vision, which serves as the foundation for many high-level computer vision applications. This paper aims to improve the accuracy of feature matching by eliminating mismatches in putative matches. To achieve this goal, we propose an extremely fast approach to boosting image matching precision (FAPP). The key idea behind FAPP is that correct feature matches have similar Euclidean distances, and the sine values of the angles between correct feature matches and the horizontal axis are also similar. Consequently, putative matches can be represented as two-dimensional coordinate points (sine value, Euclidean distance), which makes correct and incorrect feature matches have different degrees of clustering. Furthermore, the coordinate points are divided into grid spaces so that the coordinate points are distributed in different grid areas. Through adaptive parameter estimation, we determine a threshold for the number of correct feature matches within each grid, thereby eliminating false feature matches. In addition, to validate the effectiveness of FAPP, we conducted experiments on two public datasets and compared the results with several existing classical methods. The experimental outcomes demonstrate the superior performance of FAPP over the existing classical methods. Furthermore, the method has been applied to 3D reconstruction with good results. Source code: https://github.com/caomw/fapp.


A Semantics-enhanced Topic Modelling Technique: Semantic-LDA

Abstract: Topic modelling is a beneficial technique used to discover latent topics in text collections. But to correctly understand the text content and generate a meaningful topic list, semantics are important. By ignoring semantics, that is, not attempting to grasp the meaning of the words, most of the existing topic modelling approaches can generate some meaningless topic words. Even existing semantic-based approaches usually interpret the meanings of words without considering the context and related words. In this article, we introduce a semantic-based topic model called semantic-LDA that captures the semantics of words in a text collection using concepts from an external ontology. A new method is introduced to identify and quantify the concept–word relationships based on matching words from the input text collection with concepts from an ontology without using pre-calculated values from the ontology that quantify the relationships between the words and concepts. These pre-calculated values may not reflect the actual relationships between words and concepts for the input collection, because they are derived from datasets used to build the ontology rather than from the input collection itself. Instead, quantifying the relationship based on the word distribution in the input collection is more realistic and beneficial in the semantic capture process. Furthermore, an ambiguity handling mechanism is introduced to interpret the unmatched words, that is, words for which there are no matching concepts in the ontology. Thus, this article makes a significant contribution by introducing a semantic-based topic model that calculates the word–concept relationships directly from the input text collection. The proposed semantic-based topic model and an enhanced version with the disambiguation mechanism were evaluated against a set of state-of-the-art systems, and our approaches outperformed the baseline systems in both topic quality and information filtering evaluations.


Concept Drift Adaptation by Exploiting Drift Type

Abstract: Concept drift is a phenomenon where the distribution of data streams changes over time. When this happens, model predictions become less accurate. Hence, models built in the past need to be re-learned for the current data. Two design questions need to be addressed in designing a strategy to re-learn models: which type of concept drift has occurred, and how to utilize the drift type to improve re-learning performance. Existing drift detection methods are often good at determining when drift has occurred. However, few retrieve information about how the drift came to be present in the stream. Hence, determining the impact of the type of drift on adaptation is difficult. Filling this gap, we designed a framework based on a lazy strategy called Type-Driven Lazy Drift Adaptor (Type-LDA). Type-LDA first retrieves information about both how and when a drift has occurred, then it uses this information to re-learn the new model. To identify the type of drift, a drift type identifier is pre-trained on synthetic data of known drift types. Furthermore, a drift point locator locates the optimal point of drift via a sharing loss. Hence, Type-LDA can select the optimal point, according to the drift type, to re-learn the new model. Experiments validate Type-LDA on both synthetic data and real-world data, and the results show that accurately identifying drift type can improve adaptation accuracy.


Utility-aware Privacy Perturbation for Training Data

Abstract: Data perturbation under differential privacy constraint is an important approach of protecting data privacy. However, as the data dimensions increase, the privacy budget allocated to each dimension decreases and thus the amount of noise added increases, which eventually leads to lower data utility in training tasks. To protect the privacy of training data while enhancing data utility, we propose a Utility-aware training data Privacy Perturbation scheme based on attribute Partition and budget Allocation (UPPPA). UPPPA includes three procedures: the quantification of attribute privacy and attribute importance, attribute partition, and budget allocation. The quantification of attribute privacy and attribute importance based on information entropy and attribute correlation provide an arithmetic basis for attribute partition and budget allocation. During the attribute partition, all attributes of training data are classified into high and low classes to achieve privacy amplification and utility enhancement. During the budget allocation, a γ-privacy model is proposed to balance data privacy and data utility so as to provide privacy constraint and guide budget allocation. Three comprehensive sets of real-world data are applied to evaluate the performance of UPPPA. Experiments and privacy analysis show that our scheme can achieve the tradeoff between privacy and utility.


Quantum speed limit time, non-Markovianity and quantum phase transition in extended <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si2.svg" display="inline" id="d1e362"><mml:mrow><mml:mi>X</mml:mi><mml:mi>Y</mml:mi></mml:mrow></mml:math> spin chain with Dzyaloshinsky–Moriya interaction

Abstract: The effects of Dzyaloshinsky–Moriya (DM) interaction on quantum speed limit time (QSLT) of a single qubit and non-Markovianity of a coupled driven extended XY spin chain are investigated. The known quantum critical points of the model are left unaffected when DM interaction points in the direction of applied field. Anti-parallel DM interaction to the magnetic field leads to a new critical point of the model whose position is a function of the strength of the three-site interaction. Overall, the dynamics of the two quantities varies differently in a given phase of the model from point to point in the parameter space. Our findings also show that the anti-parallel DM interaction results in fast quantum evolution which might be beneficial for certain quantum computation purposes.


Automata for knowledge assessment based on the structure of observed learning outcome taxonomy

Abstract: The process of adaptive assessment of individuals' knowledge by knowledge space theory has been implemented in deterministic finite automata. However, such automata are not suitable for polytomous situations, which limits the application of automata for knowledge assessment. To overcome this limitation, this paper combines the structure of observed learning outcome (SOLO) taxonomy with the response scale and proposes an automata knowledge assessment process that can assess individuals' knowledge in polytomous situations. Specifically, an automaton suitable for assessing knowledge in the SOLO taxonomy situation is proposed, and a questioning rule is provided based on the quasi-order relation between items. Then, relying on the automaton state update and the questioning rule, an automata knowledge assessment process for knowledge assessment in the SOLO taxonomy situation is given. Finally, empirical research and simulation experiments show that it is necessary to provide a questioning rule for the automaton and that the automata knowledge assessment process proposed in this paper is feasible.


Contrastive fine-tuning for low-resource graph-level transfer learning

Abstract: Due to insufficient supervision and the gap between pre-training pretext tasks and downstream tasks, transferring pre-trained graph neural networks (GNNs) to downstream tasks in low-resource scenarios remains challenging. In this paper, a Contrastive Fine-tuning (Con-tuning) framework is proposed for low-resource graph-level transfer learning, and a graph-level supervised contrastive learning (SCL) task is designed within the framework as the first attempt to introduce SCL for fine-tuning processes of pre-trained GNNs. The SCL task compensates for the insufficient supervision problem in low-resource scenarios and narrows the gap between pretext tasks and downstream tasks. To further reinforce the supervision signal in the SCL task, we devise a graphon theory based labeled graph generator to extract the generalized knowledge of a specific class of graphs. Based on this knowledge, graph-level templates are generated for each class and used as contrastive samples in the SCL task. Then, the proposed Con-tuning framework jointly learns the SCL task and downstream tasks to effectively fine-tune the pre-trained GNNs for downstream tasks. Extensive experiments with eight real-world datasets show that Con-tuning framework enables pre-trained GNNs to achieve better performance on graph-level downstream tasks in low-resource settings.


Using outlier elimination to assess learning-based correspondence matching methods

Abstract: Recently, deep learning (DL) technology has been widely used in correspondence matching. The learning-based models are usually trained on benign image pairs with partial overlaps. Since DL model is usually data-dependent, non-overlapping images may be used as poison samples to fool the model and produce false registrations. In this study, we propose an outlier elimination-based assessment method (OEAM) to assess the registrations of learning-based correspondence matching method on partially overlapping and non-overlapping images. OEAM first eliminates outliers based on spatial paradox. Then OEAM implements registration assessment in two streams using the obtained core correspondence set. If the cardinality of the core set is sufficiently small, the input registration is assessed as a low-quality registration. Otherwise, it is assessed to be of high quality, and OEAM improves its registration performance using the core set. OEAM is a post-processing technique imposed on learning-based method. The comparison experiments are implemented on outdoor (YFCC100M) and indoor (SUN3D) datasets using four deep learning-based methods. The experimental results on registrations of partially overlapping images show that OEAM can reliably infer low-quality registrations and improve performance on high-quality registrations. The experiments on registrations of non-overlapping images demonstrate that learning-based methods are vulnerable to poisoning attacks launched by non-overlapping images, and OEAM is robust against poisoning attacks crafted by non-overlapping images.


Combined effects of temperature and confinement on the Shannon entropy of two-dimensional hydrogenic impurity states in the GaAs semiconductor quantum well

Abstract: Understanding the quantum properties of two-dimensional (2-D) hydrogenic impurity states in the GaAs quantum well is essential for their applications in the semiconductor devices. Here, we report the combined effects of temperature and quantum confienment on the Shannon entropy of 2-D hydrogenic impurity states in the GaAs quantum well. First, by calculating the position and momentum space Shannon entropy (Sr and Sp), as well as the Shannon entropy sum (St), for the hydrogenic impurity states in the GaAs quantum well at room temperature, we show fascinating quantum confinement properties in the two conjugated spaces. Then, the effects of temperature on the Shannon entropy for different hydrogenic impurity states are examined and compared. Intriguingly, we discover that the Shannon entropy sum exhibits the property of translation invariance under the influence of temperature. Finally, with the radius of the quantum well fixed at a constant, we observe that the value of the Shannon entropy for a given quantum state is highly sensitive to the variations in temperature. While the size of circular quantum well is constrained by design parameters, such as the fabrication process and intended application, temperature adjustment emerges as a viable means to control the quantum properties of 2-D hydrogenic impurities in the quantum well. This research presents a pioneering method that employs temperature as a versatile tool for modulating the Shannon entropy of hydrogenic impurity states in the quantum well, and has potential applications in the field of semiconductor physics, material physics, chemical physics, etc.


Siamese capsule network with position correlation and integrating articles of law for Chinese similar case matching

Abstract: The purpose of the Chinese similar case matching task is to compare the similarity of two case texts with a given anchor text and find out which text is more similar to the anchor text. In the area of law, it plays an important role and has been of interest to many researchers. Previous approaches have compared legal texts only at the text semantic level, without incorporating article information of law. In addition, the position correlation of words in case texts is often important, but it has not been considered in previous approaches. This paper proposes a method which extracts features from the semantic similarity level and from the level of related articles of law, respectively, to enable similarity comparisons of legal case texts. When similarity comparisons are made at the semantic similarity level, a novel capsule network method is proposed based on siamese structure that introduces the position correlation and the routing mechanism within the capsule network is improved so that deep text features between case pairs can be learned. When similarity comparisons are made at the level of related articles of law, related articles of law are selected and coded and interacted with the case text features to generate legal features. Experiment is conducted with a real-world legal text dataset, and the proposed model outperformed all baseline models, demonstrating effectiveness of the proposed model. Further, to confirm the generality of the improved capsule network proposed in the paper on long text datasets, this paper also carried out experiments on two long text datasets, demonstrating effectiveness of the improved capsule network proposed in the model.


LCEMH: Label Correlation Enhanced Multi-modal Hashing for efficient multi-modal retrieval

Abstract: Supervised multi-modal hashing can effectively improve the discriminative power of hash codes by leveraging semantic label information. However, most existing supervised multi-modal hashing models generally rely on discrete labels for supervision. They only provide limited guidance and may not fully capture the potential semantic similarity between multi-modal data. Moreover, the binary constraints of hash codes further restrict their ability to fully capture the rich semantic information embedded in the labels. To address these limitations, we propose a Label Correlation Enhanced Multi-modal Hashing (LCEMH) approach for efficient multi-modal retrieval. Specifically, our proposed model can effectively expand the inter-class boundaries of discrete labels to further enrich the supervision of similarity. Additionally, we directly transform the acquired richer label information into hash codes simply and effectively, thus retaining more comprehensive information in the labels with minimal quantization loss, and enhancing the discriminant ability of the generated hash codes. Experimental results on three public multimedia retrieval datasets demonstrate the superiority of LCEMH in performance. The source code of LCEMH is available online at: https://github.com/ChaoqunZheng/LCEMH.


Lackadaisical discrete-time quantum walk on Johnson graph

Abstract: Quantum walk is widely used in search problems, and it has an ideal acceleration effect compared with classical algorithm. Suppose there is a marked vertex w on Johnson graph J(n,k), we hope to find w using quantum walk. We adopt the coined quantum walk model, and we have proved that, for fixed order k, lackadaisical discrete-time quantum walk (DTQW) can find w with success probability 1−o(1), keeping a quadratic speedup. Before this paper, the best result of DTQW is that discrete-time quantum walk finds w with success probability 12−o(1). We improve the success probability to 1−o(1) by adding self-loops with proper weights and choose the appropriate oracle operator. Our result also matches the result of continuous-time quantum walk.


PTCAS: Prompt tuning with continuous answer search for relation extraction

Abstract: Tremendous progress has been made in the development of fine-tuned pretrained language models (PLMs) that achieve outstanding results on almost all natural language processing (NLP) tasks. Further stimulation of rich knowledge distribution within PLMs can be achieved through additional prompts for fine-tuning, namely, prompt tuning. Generally, prompt engineering involves prompt template engineering, which is the process of searching for an appropriate template for a specific task, and answer engineering, whose objective is to seek an answer space and map it to the original task label set. Existing prompt-based methods are primarily designed manually and search for appropriate verbalization in a discrete answer space, which is insufficient and always results in suboptimal performance for complex NLP tasks such as relation extraction (RE). Therefore, we propose a novel prompt-tuning method with a continuous answer search for RE, which enables the model to find optimal answer word representations in a continuous space through gradient descent and thus fully exploit the relation semantics. To further exploit entity-type information and integrate structured knowledge into our approach, we designed and added an additional TransH-based structured knowledge constraint to the optimization procedure. We conducted comprehensive experiments on four RE benchmarks to evaluate the effectiveness of the proposed approach. The experimental results show that our approach achieves competitive or superior performance without manual answer engineering compared to existing baselines under both fully supervised and low-resource scenarios.


Overlapping cytoplasms segmentation via constrained multi-shape evolution for cervical cancer screening

Abstract: Segmenting overlapping cytoplasms in cervical smear images is a clinically essential task for quantitatively measuring cell-level features to screen cervical cancer This task, however, remains rather challenging, mainly due to the deficiency of intensity (or color) information in the overlapping region Although shape prior-based models that compensate intensity deficiency by introducing prior shape information about cytoplasm are firmly established, they often yield visually implausible results, as they model shape priors only by limited shape hypotheses about cytoplasm, exploit cytoplasm-level shape priors alone, and impose no shape constraint on the resulting shape of the cytoplasm In this paper, we present an effective shape prior-based approach, called constrained multi-shape evolution, that segments all overlapping cytoplasms in the clump simultaneously by jointly evolving each cytoplasm's shape guided by the modeled shape priors We model local shape priors (cytoplasm-level) by an infinitely large shape hypothesis set which contains all possible shapes of the cytoplasm In the shape evolution, we compensate intensity deficiency for the segmentation by introducing not only the modeled local shape priors but also global shape priors (clump-level) modeled by considering mutual shape constraints of cytoplasms in the clump We also constrain the resulting shape in each evolution to be in the built shape hypothesis set for further reducing implausible segmentation results We evaluated the proposed method in two typical cervical smear datasets, and the extensive experimental results confirm its effectiveness.


Learning dynamic embeddings for temporal attributed networks

Abstract: Learning low-dimensional representations for networks has attracted considerable research interest due to the ever-increasing volume and variety of network-based information. It becomes especially important and challenging for temporal networks whose nodes and attributes evolve as time changes. To address the problem of temporal network embedding, we propose a novel Dynamic Bayesian Temporal Network Embedding (DBTNE) method that adopts a probabilistic framework and represents network embeddings as Gaussian distributions for learning effective embeddings for temporal networks. Unlike the existing temporal network embedding methods that learn dynamic embeddings only for nodes, our DBTNE jointly learns the dynamic embeddings of both nodes and attributes in temporal attributed networks for effectively capturing the similarities among them, which greatly facilitates attribute-related learning tasks such as link prediction, object classification and user profiling. In addition, we model the temporal dynamics of both nodes and attributes with neural ordinary differential equations, which supports effective and efficient capture of long-range dependencies among embeddings at different timestamps. Extensive experimental results on real-world temporal networks demonstrate that our DBTNE achieves better performance than popular state-of-the-art methods, including DeepWalk, Node2Vec, CAN, HTNE, M2DNE, MTSN, Dane-ATT, for typical graph learning tasks, such as dynamic link prediction and node classification.


A novel compression-based 2D-chaotic sine map for enhancing privacy and security of biometric identification systems

Abstract: Human biometric images are utilized for cell phone authentication, airport security, and biometric passports. To improve the biometric identification process, this should be protected from cyber attackers because it is sensitive to any minor changes. Thus, security is a major concern in biometric images. Traditional encryption and compression methods are ineffective for encrypting biometric images due to their high execution time and algorithm complexity. In this paper, a novel 2D chaotic sine map is proposed for generating encryption keys and improving security for biometric identification systems. The proposed scheme uses the 2D chaotic map to generate the private key and diffusion process. Here, the Hénon-Sine Map (HSM-512), Secure Hash Algorithm (SHA-256), and DNA computing are also used in the key generation process. The pixel values of the original images in the proposed schemes are shuffled using Mersenne Twister (MT) to improve the security of biometric images. Moreover, the Differential Huffman Compression (DHC) method is used for lossless compression while performing the XOR-based encryption process. The proposed model has been tested on different RGB biometric images, namely the SOCOFing dataset and the COVID-19 chest X-ray dataset. The experiment results have been evaluated using many performance metrics, such as entropy, key space, histogram analysis, key sensitivity, robustness analysis, correlation, and similarity analysis. The outcomes demonstrate that the proposed scheme is more effective than the state-of-the-art schemes.


A multi-scale contextual attention network for remote sensing visual question answering

Abstract: Remote sensing visual question answering (RSVQA) is a user-friendly method used for analyzing remote sensing images (RSIs) in various tasks. However, current methods often overlook geospatial objects, which possess a multi-scale representation and require contextual information. Furthermore, limited research has been conducted on modeling and reasoning the long-distance dependencies between entities, resulting in one-sided and inaccurate answer predictions. To overcome these limitations, we propose the Scale-Aware Multi-level Feature Pyramid Network (SAMFPN), which integrates contextual and multi-scale information using a Feature Pyramid Network (FPN) and Co-Attention mechanisms. The SAMFPN module incorporates a multi-level FPN to capture both global and local contextual information. Additionally, it introduces a Visual-Question Collaboration Fusion (VQCF) module that simultaneously embeds and learns visual and textual information. Our experimental results demonstrate the superior accuracy and robustness of our proposed model compared to existing models. These outcomes indicate that SAMFPN effectively captures multi-scale contextual information, making it a reliable solution for RSVQA tasks.


Cryptanalysis of cross-coupled chaotic maps multi-image encryption scheme

Abstract: Recently, Patro et al. proposed a multiple gray-scale image encryption scheme based on cross-coupled chaotic maps. The authors claimed the scheme could withstand known plain-text attacks and chosen-plaintext attacks. In this paper, we have thoroughly analysed the Patro et al. algorithm and reported the crucial loopholes. Patro et al. algorithm uses a row-wise and column-wise scrambling operation based on a secret sequence followed by a feed-forward XOR operation generating the cipher image. Exploiting the loopholes of feed-forward XOR operation and high correlation of pixels in standard images, a cryptanalysis algorithm is developed that deciphers the ciphered images generated by Patro et al. algorithm based on cipher-text only attack. Without using the secret keys, the proposed cryptanalysis algorithm successfully deciphers the cipher image and reveals the secret images.


Semantic information extraction and search of mineral exploration data using text mining and deep learning methods

Abstract: Large-scale mineral resource reports offer a wealth of information for geological knowledge mining and mineral explorers’ knowledge discovery. The geological conditions in which mineral deposits develop may be learned a great deal from these mineral exploration reports. Mineral exploration data may be queried and aggregated to effectively mitigate future exploration risks and reduce costs. However, due to the reports being presented in unstructured textual format, it becomes challenging to extract valuable geological data without manually scanning through a vast number of reports. This laborious process poses difficulties for geologists. To address this issue, this paper proposes a system that extracts a set of geologically relevant keywords/keyphrases of each chapter from each mineral exploration reposts using latent Dirichlet allocation (LDA), develops a topic graph, recognizes the geological entity and related relations for constructing a knowledge graph, and uses visualization of those graphs (e.g., topic graphs and knowledge graphs) to explore the contents of the report. The text mining and machine learning technique described here serves as the foundation for future research into incorporating semantic analysis into geological information extraction. The findings of this study show how automated text analysis may help with the quick processing of huge quantities of reports in order to identify target mineral systems and their related geological location and rock mineral composition. The suggested approaches can quickly and reliably convert mineral exploration data (e.g., text, figure, and table) into a structured form, which is a hitherto untouched field in geological knowledge mining.


Design of Motor Skill Recognition and Hierarchical Evaluation System for Table Tennis Players

Abstract: With the rapid development of electronic science and technology, the research on wearable devices is constantly updated. Still, the current wearable devices are not yet comprehensive in recognizing and analyzing the movement patterns of special sports. Based on this, this paper improves wearable devices for table tennis, a single sport, and realizes the recognition and evaluation of table tennis players’ motor skills based on the pattern recognition method and hierarchical evaluation system. First, a table tennis player’s motion information acquisition device was designed and the collected motion data were processed. Secondly, a sliding window was designed to segment the collected motion data and establish a feature database of six table tennis benchmark movements. Secondly, the motion features were constructed based on feature engineering, and after dimensionality reduction, the different models were used for action skill recognition. Finally, a hierarchical evaluation system of movement skills was established and the loss functions of different evaluation indexes were given. The experimental results show that in table tennis players’ action skill recognition, the neural network model based on motor feature extraction proposed in this paper improves the recognition accuracy by 6.5% than the traditional convolutional neural network, and in table tennis players’ action skill evaluation, the ANOVA shows that the hierarchical evaluation system proposed in this paper has a significance level of p<0.05 for the athletes’ professionalism level. The conclusion of the study indicates that the proposed paper table tennis players’ motor skills recognition and hierarchical evaluation assessment system can realize the accurate recognition of athletes’ motor skills and the precise determination of their professional level.


Federated Learning for Software Engineering: A Case Study of Code Clone Detection and Defect Prediction

Abstract:
In various research domains, artificial intelligence (AI) has gained significant prominence, leading to the development of numerous learning-based models in research laboratories, which are evaluated using benchmark datasets. While the models proposed in previous studies may demonstrate satisfactory performance on benchmark datasets, translating academic findings into practical applications for industry practitioners presents challenges. This can entail either the direct adoption of trained academic models into industrial applications, leading to a performance decrease, or retraining models with industrial data, a task often hindered by insufficient data instances or skewed data distributions. Real-world industrial data is typically significantly more intricate than benchmark datasets, frequently exhibiting data-skewing issues, such as label distribution skews and quantity skews. Furthermore, accessing industrial data, particularly source code, can prove challenging for Software Engineering (SE) researchers due to privacy policies. This limitation hinders SE researchers’ ability to gain insights into industry developers’ concerns and subsequently enhance their proposed models. To bridge the divide between academic models and industrial applications, we introduce a federated learning (FL)-based framework called A <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lmity . Our aim is to simplify the process of implementing research findings into practical use for both SE researchers and industry developers. A <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lmity enhances model performance on sensitive skewed data distributions while ensuring data privacy and security. It introduces an innovative aggregation strategy that takes into account three key attributes: data scale, data balance, and minority class learnability. This strategy is employed to refine model parameters, thereby enhancing model performance on sensitive skewed datasets.
In our evaluation, we employ two well-established SE tasks, i.e., code clone detection and defect prediction, as evaluation tasks. We compare the performance of Almity on both machine learning (ML) and deep learning (DL) models against two mainstream training methods, specifically the Centralized Training Method (CTM) and Vanilla Federated Learning (VFL), to validate the effectiveness and generalizability of Almity. Our experimental results demonstrate that our framework is not only feasible but also practical in real-world scenarios. Almity consistently enhances the performance of learning-based models, outperforming baseline training methods across all types of data distributions.


Self-Supervised Monocular Depth Estimation Based on High-Order Spatial Interactions

Abstract: Depth estimation plays a pivotal role in various applications, including autonomous driving and robot navigation. In contrast to depth estimation using multiple images, such as stereo depth perception, inferring depth relations from a monocular camera is notably more challenging yet highly valuable. Traditionally, convolutional neural networks with residual structures have been extensively employed for this task, but they inherently constrain the model’s feature extraction capabilities. Inspired by HorNet, in this paper, we propose a novel self-supervised monocular depth estimation framework based on high-order spatial interactions, referred to as the Hor-Depth. Furthermore, the Hor-Depth improves feature fusion efficiency in the depth network decoder by incorporating the attentional feature fusion module based on first-order spatial interaction, leading to more refined predicted disparity maps. To address issues of loss fluctuations and training instability, we introduce a progressive scale-weight adjustment strategy-based loss function. This strategy applies varying constraints to the model at different training stages, effectively reducing training fluctuations and mitigating outliers that significantly deviate from the predicted values. The proposed approach demonstrates exceptional performance in self-supervised monocular depth estimation, surpassing certain stereo supervised or monocular supervised methods, as evidenced by its impressive results on the KITTI Eigen split benchmark.


An Adaptive Track Initiation Method Adopting Multiscan Spatiotemporal Information

Abstract: Trajectory initiation of dense newborn targets is an important research topic in multi-target tracking, however, few reports concerning the related studies can be found. In response to the problem of accurately obtaining spatiotemporal information of densely populated newborn targets, we propose a novel adaptive track initiation method adopting multi-scan spatiotemporal information. Firstly, by combining the spatial information measured from multiple scans, an adaptive track initiation method is proposed to obtain the measurement sequence of newborn targets. Secondly, by establishing an adaptive newborn density model, the newborn state estimation of dense targets is obtained from the measurement sequence of newborn targets. Finally, the state estimation of the targets is input to the labelled random finite set filter to obtain the trajectory information. By quantifying the posterior density error of the labelled random finite set, the truncation error of the newborn state estimation is presented. The error analysis suggests that our proposed method can achieve track initiation with small boundary L1 error. The simulation results verify that the proposed method can effectively cope with the situation of dense newborn targets. Compared with the existing adaptive birth algorithms, our proposal can realize more accurate track initiation when newborn targets occur simultaneously.