AdaDet: An Adaptive Object Detection System Based on Early-Exit Neural Networks

Abstract: Recent researchers have proposed adaptive inference methods with an early-exiting mechanism, which stops the inference procedure of input if the prediction is with high confidence, leading to a fine-grained resource allocation based on the complexity of inputs. However, the adaptive inference strategy can only be applied for image classification tasks, such a system for object detection is still under-investigated since it lacks an appropriate way to measure the uncertainty of detection results for multiple objects in an image. To address this issue, in this paper, we propose a novel adaptive object detection system (AdaDet) based on early-exit neural networks. By using stochastic variables to represent localization predictions, our method can measure the uncertainty of the detection result for an object, based on which we further develop an entropy-based criterion to estimate the reliability of the whole detection results for an input image. With the proposed method, samples that achieve high-confidence detection results will exit early from the detection model, leading to low computational costs for inferring these samples. While only complex images need to finish the whole computational graph to achieve better detection results during inference. We experimentally demonstrate that the proposed AdaDet enhances detection performance under anytime prediction and adaptive inference settings on the most common dataset (MS COCO) used in this field.


Collaborative Parameter Estimation of Multiple Unmanned Surface Vessels: A Robust Distributed Estimator-Based Approach

Abstract: In this paper, the collaborative parameter estimation of multiple unmanned surface vessels (USVs) with model structure uncertainties is studied. The considered parameter estimation problem is first converted into a distributed state and parameter joint estimation problem. Then, the robust distributed estimator is constructed to handle the inevitable model structure uncertainties, and the upper bounds of prediction and estimation error covariance matrices are derived respectively. Additionally, the upper bound of the estimation error covariance matrix is minimized by designing appropriate estimator gains. Finally, the advantages of the proposed distributed parameter estimation approach from the perspectives of information interaction and consideration of model structure uncertainties are verified via simulations and practical experiments.


On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

Abstract: The convergence of policy gradient algorithms in reinforcement learning hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into these algorithms can often be acquired from analyzing those of linear quadratic control. However, most of the existing literature only considers the optimization landscape for static full-state or output feedback policies (controllers). We investigate the more challenging case of dynamic output-feedback policies for linear quadratic regulation (abbreviated as dLQR), which is prevalent in practice but has a rather complicated optimization landscape. We first show how the dLQR cost varies with the coordinate transformation of the dynamic controller and then derive the optimal transformation for a given observable stabilizing controller. At the core of our results is the uniqueness of the stationary point of dLQR when it is observable, which is in a concise form of an observer-based controller with the optimal similarity transformation. These results shed light on designing efficient algorithms for general decision-making problems with partially observed information.


Self-Calibrating Gaze Estimation With Optical Axes Projection for Head-Mounted Eye Tracking

Abstract: Gaze estimation suffers from burdensome personal calibration or complex all-device calibration. Self-calibrating methods can meet this challenge but depend on scenes and sacrifice accuracy. We propose a flexible and accurate gaze estimation approach calibrated implicitly with potential gaze patterns. By constructing an optical axis projection (OAP) plane and a visual axis projection (VAP) plane simultaneously, the optical axis and the visual axis can be represented as 2-D points, i.e., the OAP and VAP, which have a similarity transformation, indicating the linear consistency of OAP patterns with gaze patterns. Hence, a 3-D gaze estimation model using the OAP as an eye feature to predict the VAP is built. The unknown parameters are calculated separately by linearly aligning OAP patterns to natural and easily detectable gaze patterns. Experimental results show that the proposed gaze estimation approach is more accurate than state-of-the-art head-mounted gaze estimation methods, which require explicit calibration or multi-scene saliency.


Energy-Motion Features Aggregation Network for Players’ Fine-Grained Action Analysis in Soccer Videos

Abstract: Rich and complex events in sports have led to the development of a wide-variety of techniques for interpreting content of sports videos in terms of players’ actions, poses, gait, performance, etc. This is due to the requirements from coaches, trainers and players who expect to analyze actions in top sports events, as well as sports fans who practice to imitate professional playing skills, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g ., dribbling, shooting, etc. However, this poses two key challenges for automated sports analysis community. Firstly, there are extremely limited public sports datasets. Secondly, recent advances in interpretations of sports activities, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g ., soccer, are predominantly made through analyzing coarse-grained contents. Players’ fine-grained skills analysis still remains under-explored. To alleviate these problems, this paper (a) collects the dataset of highlight videos of soccer players, including two coarse-grained action types of soccer players and six fine-grained actions of players. Detailed annotations are provided for the collected dataset, in terms of action classes, bounding boxes, segmentation maps, and body keypoints of soccer players, and positions of a soccer ball in a game. (b) leverages the understanding of complex highlight videos by proposing an energy-motion features aggregation network- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EMA-Net to fully exploit energy-based representation of soccer players movements in video sequences and explicit motion dynamics of soccer players in videos for soccer players’ fine-grained action analysis. Experimental results and ablation studies validate the proposed approach in recognizing soccer players actions using the collected soccer highlight video datasets.


A Label-Efficient Semi Self-Supervised Learning Framework for IoT Devices in Industrial Process

Abstract: The industrial sector has experienced a tremendous advancement in deep supervised learning due to its representation ability, but it comes with high computing and labeled data demands. Recently, the demand for intelligent IoT devices on assembly and disassembly lines has surged. This necessitates algorithms that can use data to make intelligent decisions and a framework that can enable multiple IoT devices to learn collaboratively. Further, huge image generation using IoTs also needs an efficient data annotation scheme for classification problems. WeCollab is one such framework in federated learning that significantly reduces human efforts in data annotation with breakthroughs in self-supervised learning. The proposed framework is generic and can be adapted to any specific image data generated by industrial robots involved in assembly and disassembly lines. Our method outperforms supervised learning by 25% and 20% on the CIFAR-10 and CINIC-10 datasets, respectively, for the labeling task. We generate pseudo-labels for the unlabeled part of the data and train a model to achieve 30% better test accuracy on CIFAR-10 and 20% better test accuracy on the CINIC-10 dataset as compared to supervised learning. Extensive experiments unveil the effectiveness of the method and proposed combination of loss functions used by WeCollab.


Vertical Federated Learning Over Cloud-RAN: Convergence Analysis and System Optimization

Abstract: Vertical federated learning (FL) is a collaborative machine learning framework that enables devices to learn a global model from the feature-partition datasets without sharing local raw data. However, as the number of the local intermediate outputs is proportional to the training samples, it is critical to develop communication-efficient techniques for wireless vertical FL to support high-dimensional model aggregation with full device participation. In this paper, we propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation by leveraging over-the-air computation (AirComp) and alleviating communication straggler issue with cooperative model aggregation among geographically distributed edge servers. However, the model aggregation error caused by AirComp and quantization errors caused by the limited fronthaul capacity degrade the learning performance for vertical FL. To address these issues, we characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. To improve the learning performance, we establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed. We conduct extensive simulations to demonstrate the effectiveness of the proposed system architecture and optimization framework for vertical FL.


Green, Quantized Federated Learning Over Wireless Networks: An Energy-Efficient Design

Abstract: The practical deployment of federated learning (FL) over wireless networks requires balancing energy efficiency, convergence rate, and a target accuracy due to the limited available resources of devices. Prior art on FL often trains deep neural networks (DNNs) to achieve high accuracy and fast convergence using 32 bits of precision level. However, such scenarios will be impractical for resource-constrained devices since DNNs typically have high computational complexity and memory requirements. Thus, there is a need to reduce the precision level in DNNs to reduce the energy expenditure. In this paper, a green-quantized FL framework, which represents data with a finite precision level in both local training and uplink transmission, is proposed. Here, the finite precision level is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. In the considered FL model, each device trains its QNN and transmits a quantized training result to the base station. Energy models for the local training and the transmission with quantization are rigorously derived. To minimize the energy consumption and the number of communication rounds simultaneously, a multi-objective optimization problem is formulated with respect to the number of local iterations, the number of selected devices, and the precision levels for both local training and transmission while ensuring convergence under a target accuracy constraint. To solve this problem, the convergence rate of the proposed FL system is analytically derived with respect to the system control variables. Then, the Pareto boundary of the problem is characterized to provide efficient solutions using the normal boundary inspection method. Design insights on balancing the tradeoff between the two objectives while achieving a target accuracy are drawn from using the Nash bargaining solution and analyzing the derived convergence rate. Simulation results show that the proposed FL framework can reduce energy consumption until convergence by up to 70% compared to a baseline FL algorithm that represents data with full precision without damaging the convergence rate.


A Unified Object Counting Network With Object Occupation Prior

Abstract: The counting task, which plays a fundamental role in numerous applications (e.g., crowd counting, traffic statistics), aims to predict the number of objects with various densities. Existing object counting tasks are designed for a single object class. However, it is inevitable to encounter newly coming data with new classes in our real world. We name this scenario as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">evolving object counting</i> . In this paper, we build the first evolving object counting dataset and propose a unified object counting network as the first attempt to address this task. The proposed network consists of two key components: a class-agnostic mask module and a class-incremental module. The class-agnostic mask module learns generic object occupation prior by predicting a class-agnostic binary mask (e.g., 1 denotes there exists an object at the considering position in an image and 0 otherwise). The class-incremental module is used to handle new classes and provides discriminative class guidance for density map prediction. The combined outputs of the class-agnostic mask module and image feature extractor are used to predict the final density map. When new classes arrive, we first add new neural nodes to the last regression and classification layers of the class-incremental module. Then, instead of retraining the model from scratch, we utilize knowledge distillation to help the model retain and consolidate what it has previously learned. We also employ a support sample bank to store a small number of typical training samples for each class, which are used to prevent the model from forgetting key information from old data. With this design, our model can efficiently and effectively adapt to new classes while maintaining good performance on already-seen data without large-scale retraining. Extensive experiments on the collected dataset demonstrate favorable performance. The dataset and code will be available at: https://github.com/Tanyjiang/EOCO.


Complete coverage path planning scheme for autonomous navigation ROS-based robots

Abstract: In this paper, a new Complete Coverage Path Planning (CCPP) scheme is proposed which combines path planning and dynamic tracking for robot operating system-based robots. For the path planning, firstly a sub-area division algorithm is considered to decompose the occupancy map according to the wall or obstacle position after simultaneous localization and mapping process. For each sub-area, an “S” shape path planning is employed, and then a Bidirectional A-star connects them. Additionally, the dynamic tracking ensures that the robot moves continuously. Simulation results show that the coverage ratio of the planning path is improved as 98% by the proposes scheme.


Parallel Successive Learning for Dynamic Distributed Model Training Over Heterogeneous Wireless Networks

Abstract: Federated learning (FedL) has emerged as a popular technique for distributing model training over a set of wireless devices, via iterative local updates (at devices) and global aggregations (at the server). In this paper, we develop <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">parallel successive learning (PSL), which expands the FedL architecture along three dimensions: (i) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Network , allowing decentralized cooperation among the devices via device-to-device (D2D) communications. (ii) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Heterogeneity , interpreted at three levels: (ii-a) Learning: PSL considers heterogeneous number of stochastic gradient descent iterations with different mini-batch sizes at the devices; (ii-b) Data: PSL presumes a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dynamic environment with data arrival and departure, where the distributions of local datasets evolve over time, captured via a new metric for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model/concept drift . (ii-c) Device: PSL considers devices with different computation and communication capabilities. (iii) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Proximity , where devices have different distances to each other and the access point. PSL considers the realistic scenario where global aggregations are conducted with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">idle times in-between them for resource efficiency improvements, and incorporates <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">data dispersion and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model dispersion with local model condensation into FedL. Our analysis sheds light on the notion of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cold vs. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">warmed up models, and model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inertia in distributed machine learning. We then propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">network-aware dynamic model tracking to optimize the model learning vs. resource efficiency tradeoff, which we show is an NP-hard signomial programming problem. We finally solve this problem through proposing a general optimization solver. Our numerical results reveal new findings on the interdependencies between the idle times in-between the global aggregations, model/concept drift, and D2D cooperation configuration.


The digital economy: Challenges and opportunities in the new era of technology and electronic communications

Abstract: The digital economy has emerged as an innovative economy that utilizes digital technologies and electronic communication to conduct economic and business activities across a wide range of sectors, including e-commerce, digital marketing, digital financial services, software development, computer games, and cloud services. The use of digital technologies and electronic communication has resulted in a significant shift towards online business interactions, leading to improved user experiences, faster processing, and easier access to services and products. The digital economy is recognized as a primary driver of economic growth and development in many advanced countries and is continually expanding due to its significant impact on economic and business activities. In this research, we aim to examine the digital economy and its impact on businesses and users, evaluating the opportunity for easy and quick access to products and services that lead to increased efficiency and productivity. The digital economy has significant impacts on various economic, social, and cultural fields, including changes in the way people work and interact with each other, promoting more flexible and remote work arrangements, and increasing global connectivity. Additionally, the digital economy has impacted education, healthcare, entertainment, and other sectors. Advances in technology and electronic communication have driven the global economy towards digitization, and the role of the digital economy in advancing global digitalization is critical. Its impact is expected to continue to increase in the coming years.


Time-Aware LSTM Neural Networks for Dynamic Personalized Recommendation on Business Intelligence

Abstract: Personalized recommendation plays a critical role in providing decision-making support for product and service analysis in the field of business intelligence.Recently, deep neural network-based sequential recommendation models gained considerable attention.However, existing approaches pay little attention to users' dynamically evolving interests, which are influenced by product attributes, especially product category.To overcome these challenges, we propose a dynamic personalized recommendation model: DynaPR.Specifically, we first embed product information and attribute information into a unified data space.Then, we exploit long short-term memory (LSTM) networks to characterize sequential behavior over multiple time periods and seize evolving interests by hierarchical LSTM networks.Finally, similarity values between users are measured through pairwise interest features, and personalized recommendation lists are generated.A series of experiments reveal the superiority of the proposed method compared with other advanced methods.


A fast density peak clustering based particle swarm optimizer for dynamic optimization

Abstract: Dynamic optimization problems (DOPs) are optimization problems with time evolution characteristics. In this type of problem, the decision variables and the state variables change over time and produce results that will impact the future. To effectively address DOPs, this paper proposes a fast density peak clustering based particle swarm optimizer for dynamic optimization (DPCPSO). The main innovations of DPCPSO contain three critical components. First, a fast density peak clustering is applied to create multiple sub-populations, which can help the algorithm locate peaks. Second, stagnation detection is used to tackle the loss of diversity. Third, an optimal particle calibration strategy which can find the optimal solution quickly in a changing environment is proposed in response to environmental changes. Moreover, the hill climbing method is applied to help the memory quickly locate new peaks if the environment changes. The performance of our proposed algorithm has been tested on Mobile Peak Benchmark (MPB), Generalized Dynamic Benchmark Generator (GDBG) and Generalized Moving Peaks Benchmark (GMPB) problems and compared with seven state-of-the-art dynamic optimization algorithms. The experimental results validate the proposed algorithm performed competitively while solving DOPs.


ASB-CS: Adaptive sparse basis compressive sensing model and its application to medical image encryption

Abstract: Recent advances in intelligent wearable devices have brought tremendous chances for the development of healthcare monitoring system. However, the data collected by various sensors in it are user-privacy-related information. Once the individuals’ privacy is subjected to attacks, it can potentially cause serious hazards. For this reason, a feasible solution built upon the compression-encryption architecture is proposed. In this scheme, we design an Adaptive Sparse Basis Compressive Sensing (ASB-CS) model by leveraging Singular Value Decomposition (SVD) manipulation, while performing a rigorous proof of its effectiveness. Additionally, incorporating the Parametric Deformed Exponential Rectified Linear Unit (PDE-ReLU) memristor, a new fractional-order Hopfield neural network model is introduced as a pseudo-random number generator for the proposed cryptosystem, which has demonstrated superior properties in many aspects, such as hyperchaotic dynamics and multistability. To be specific, a plain medical image is subjected to the ASB-CS model and bidirectional diffusion manipulation under the guidance of the key-controlled cipher flows to yield the corresponding cipher image without visual semantic features. Ultimately, the simulation results and analysis demonstrate that the proposed scheme is capable of withstanding multiple security attacks and possesses balanced performance in terms of compressibility and robustness.


Role-oriented binary grey wolf optimizer using foraging-following and Lévy flight for feature selection

Abstract: Feature selection can effectively define the feature subset, remove redundant, irrelevant, and noisy features. In order to adapt the feature selection problem, this paper adopts role-oriented modeling paradigm to make up for the defects of binary grey wolf optimizer such as premature, declining diversity, and insufficient convergence, then comes up with novel V-shaped linear transfer functions. Ultimately, a role-oriented binary grey wolf optimizer is proposed. The top grey wolves combine the foraging-following to enhance convergence, while the ordinary grey wolves integrate Lévy flight to avoid premature and improve diversity. The outstanding ability of role-oriented binary grey wolf optimizer in balance of exploitation and exploration is verified by benchmarks and the effectiveness in solving feature selection is demonstrated through feature selection tasks. The results of benchmark and feature selection task have comprehensively demonstrated the competitiveness of role-oriented modeling paradigm in improving metaheuristic algorithms.


Emotional expression by artificial intelligence chatbots to improve customer satisfaction: Underlying mechanism and boundary conditions

Abstract: Artificial intelligence chatbots have invaded the tourism industry owing to their low cost and high efficiency. However, the influence of emotional expressions of chatbots on service outcomes has not received much attention from researchers. Drawing upon expectancy violations theory, we explored how emotional expressions of chatbots affect customer satisfaction using three experiments in the context of tourist attraction recommendations. Chatbots' expressions of concern for customers can improve customer satisfaction by reducing expectancy violations. In particular, customer's goal orientation, the human-likeness of chatbot's avatars, and the relationship type between customers and chatbots can moderate the negative relationship between emotional expression and expectancy violation. These findings advance research on the emotional expressions of chatbots and provide critical insights for deploying chatbots in customer service in the tourism industry.


Collaborative contrastive learning for hypergraph node classification

Abstract: Plenty of models have been presented to handle the hypergraph node classification. However, very few of these methods consider contrastive learning, which is popular due to its great power to represent instances. This paper makes an attempt to leverage contrastive learning to hypergraph representation learning. Specifically, we propose a novel method called Collaborative Contrastive Learning (CCL), which incorporates a generated standard graph with the hypergraph. The main technical contribution here is that we develop a collaborative contrastive schema, which performs contrast between the node views obtained from the standard graph and hypergraph in each network layer, thus making the contrast collaborative. To be precise, in the first layer, the view from the standard graph is used to augment that from the hypergraph. Then, in the next layer, the augmented features are adopted to train a new representation to augment the view from the standard graph conversely. With this setting, the learning procedure is alternated between the standard graph and hypergraph. As a result, the learning on the standard graph and hypergraph is collaborative and leads to the final informative node representation. Experimental results on several widely used datasets validate the effectiveness of the proposed model.


A cross Transformer for image denoising

Abstract: Deep convolutional neural networks (CNNs) depend on feedforward and feedback ways to obtain good performance in image denoising. However, how to obtain effective structural information via CNNs to efficiently represent given noisy images is key for complex scenes. In this paper, we propose a cross Transformer denoising CNN (CTNet) with a serial block (SB), a parallel block (PB), and a residual block (RB) to obtain clean images for complex scenes. A SB uses an enhanced residual architecture to deeply search structural information for image denoising. To avoid loss of key information, PB uses three heterogeneous networks to implement multiple interactions of multi-level features to broadly search for extra information for improving the adaptability of an obtained denoiser for complex scenes. Also, to improve denoising performance, Transformer mechanisms are embedded into the SB and PB to extract complementary salient features for effectively removing noise in terms of pixel relations. Finally, a RB is applied to acquire clean images. Experiments illustrate that our CTNet is superior to some popular denoising methods in terms of real and synthetic image denoising. It is suitable to mobile digital devices, i.e., phones. Codes can be obtained at https://github.com/hellloxiaotian/CTNet.


LFighter: Defending against the label-flipping attack in federated learning

Abstract: Federated learning (FL) provides autonomy and privacy by design to participating peers, who cooperatively build a machine learning (ML) model while keeping their private data in their devices. However, that same autonomy opens the door for malicious peers to poison the model by conducting either untargeted or targeted poisoning attacks. The label-flipping (LF) attack is a targeted poisoning attack where the attackers poison their training data by flipping the labels of some examples from one class (i.e., the source class) to another (i.e., the target class). Unfortunately, this attack is easy to perform and hard to detect, and it negatively impacts the performance of the global model. Existing defenses against LF are limited by assumptions on the distribution of the peers' data and/or do not perform well with high-dimensional models. In this paper, we deeply investigate the LF attack behavior. We find that the contradicting objectives of attackers and honest peers on the source class examples are reflected on the parameter gradients corresponding to the neurons of the source and target classes in the output layer. This makes those gradients good discriminative features for the attack detection. Accordingly, we propose LFighter, a novel defense against the LF attack that first dynamically extracts those gradients from the peers' local updates and then clusters the extracted gradients, analyzes the resulting clusters, and filters out potential bad updates before model aggregation. Extensive empirical analysis on three data sets shows the effectiveness of the proposed defense regardless of the data distribution or model dimensionality. Also, LFighter outperforms several state-of-the-art defenses by offering lower test error, higher overall accuracy, higher source class accuracy, lower attack success rate, and higher stability of the source class accuracy. Our code and data are available for reproducibility purposes at https://github.com/NajeebJebreel/LFighter.


Metaverse for service industries: Future applications, opportunities, challenges and research directions

Abstract: Although the metaverse is still in the early stages of development and implementation, it has the potential to revolutionize the way how businesses can interact with customers through both the virtual and real world. In particular, service industries are already exploring the opportunity to utilize the metaverse to provide more immersive, interactive and engaging customer experiences. However, the holistic overview of the future applications, opportunities, and challenges of a metaverse in the context of service industries from academic and expert perspectives is limited. By employing a multi-perspective approach, this study looks into these unexplored aspects of the metaverse in the context of service industries through informed and multifaceted narratives by leading academics and experts from cross-disciplinary backgrounds from media and communication, education, hospitality, financial services, retail, tourism and healthcare. The main opportunities identified include the development of new experiences, the introduction of novel inter-world interactions, and new business-consumer relations within the metaverse. The key challenges covered include current technological boundaries, limitations of the experiences in the metaverse, health issues, and data privacy, security, and legal issues. The paper concludes with formulating future research agendas and presenting contributions to literature and implications for practice.


Big data analytics deep learning techniques and applications: A survey

Abstract: Deep learning (DL), as one of the most active machine learning research fields, has achieved great success in numerous scientific and technological disciplines, including speech recognition, image classification, language processing, big data analytics, and many more. Big data analytics (BDA), where raw data is often unlabeled or uncategorized, can greatly benefit from DL because of its ability to analyze and learn from enormous amounts of unstructured data. This survey paper tackles a comprehensive overview of state-of-the-art DL techniques applied in BDA. The main target of this survey is intended to illustrate the significance of DL and its taxonomy and detail the basic techniques used in BDA. It also explains the DL techniques used in big IoT data applications as well as their various complexities and challenges. The survey presents various real-world data-intensive applications where DL techniques can be applied. In particular, it concentrates on the DL techniques in accordance with the BDA type for each application domain. Additionally, the survey examines DL benchmarked frameworks used in BDA and reviews the available benchmarked datasets, besides analyzing the strengths and limitations of each DL technique and their suitable applications. Further, a comparative analysis is also presented by comparing existing approaches to the DL methods used in BDA. Finally, the challenges of DL modeling and future directions are discussed.


Learning Flow-Based Disentanglement

Abstract: Face reenactment aims to generate the talking face images of a target person given by a face image of source person. It is crucial to learn latent disentanglement to tackle such a challenging task through domain mapping between source and target images. The attributes or talking features due to domains or conditions become adjustable to generate target images from source images. This article presents an information-theoretic attribute factorization (AF) where the mixed features are disentangled for flow-based face reenactment. The latent variables with flow model are factorized into the attribute-relevant and attribute-irrelevant components without the need of the paired face images. In particular, the domain knowledge is learned to provide the condition to identify the talking attributes from real face images. The AF is guided in accordance with multiple losses for source structure, target structure, random-pair reconstruction, and sequential classification. The random-pair reconstruction loss is calculated by means of exchanging the attribute-relevant components within a sequence of face images. In addition, a new mutual information flow is constructed for disentanglement toward domain mapping, condition irrelevance, and condition relevance. The disentangled features are learned and controlled to generate image sequence with meaningful interpretation. Experiments on mouth reenactment illustrate the merit of individual and hybrid models for conditional generation and mapping based on the informative AF.


ArtCap: A Dataset for Image Captioning of Fine Art Paintings

Abstract: The image captioning of fine art paintings aims at generating content descriptions for the paintings. Due to the complexity of modeling both image and language, this task usually needs sufficient training data. However, different from photographic image captioning, there are few satisfactory datasets for painting captioning. In this article, we introduce a painting captioning dataset (named the ArtCap dataset), which contains 3606 paintings and five descriptions for each painting. We present the carefully designed construction pipeline of our dataset and further evaluate our dataset from two aspects of annotation quality and application effectiveness, respectively. For the annotation quality, we compare the global characteristics, annotation content, and annotation consistency of our dataset with other painting descriptions datasets. For application effectiveness, we employ our dataset and other painting descriptions datasets to train image captioning models and analyze the captioning performances. The results demonstrate the promising annotation quality and application effectiveness of our dataset.


An Emotion Recognition Method Based on Eye Movement and Audiovisual Features in MOOC Learning Environment

Abstract: In recent years, more and more people have begun to use massive online open course (MOOC) platforms for distance learning.However, due to the space-time isolation between teachers and students, the negative emotional state of students in MOOC learning cannot be identified timely.Therefore, students cannot receive immediate feedback about their emotional states.In order to identify and classify learners' emotions in video learning scenarios, we propose a multimodal emotion recognition method based on eye movement signals, audio signals, and video images.In this method, two novel features are proposed: feature of coordinate difference of eyemovement (FCDE) and pixel change rate sequence (PCRS).FCDE is extracted by combining eye movement coordinate trajectory and video optical flow trajectory, which can represent the learner's attention degree.PCRS is extracted from the video image, which can represent the speed of image switching.A feature extraction network based on convolutional neural network (CNN) (FE-CNN) is designed to extract the deep features of the three modals.The extracted deep features are inputted into the emotion classification CNN (EC-CNN) to classify the emotions, including interest, happiness, confusion, and boredom.In single modal identification, the recognition accuracies corresponding to the three modals are 64.32%,74.67%, and 71.88%.The three modals are fused by feature-level fusion, decision-level fusion, and modellevel fusion methods, and the evaluation experiment results show that the method of decision-level fusion achieved the highest score of 81.90% of emotion recognition.Finally, the effectiveness of FCDE, FE-CNN, and EC-CNN modules is verified by ablation experiments.


Assessment of Wealth Distribution in Blockchain Online Social Media

Abstract: Online social networks (OSNs) revolutionized how people interact with each other, and nowadays, thanks to blockchain technology, new solutions are being considered, giving birth to blockchain online social media (BOSMs). BOSMs use the blockchain to redistribute with their users the wealth generated by the platform through a rewarding system, assigning better rewards to socially impactful users. Thus, these new systems are characterized by highly intertwined economical and social aspects and constitute a new scenario in the world of social networks. Many scenarios, economic and social alike, show a phenomenon known as “the rich-get-richer”, which states that the richest actors of a system tend to become richer over time. To the best of our knowledge, in the scenario of BOSMs, where users can acquire cryptocurrency through their social actions, this type of phenomenon was not yet studied. In this article, we propose a methodological framework composed of three hypotheses that can help study the rich-get-richer phenomenon through a set of measures and indices. In addition, we apply the proposed framework to the Steem case study, showing how unevenly wealth is distributed on its blockchain and comparing our results to other scenarios.


MTBullyGNN: A Graph Neural Network-Based Multitask Framework for Cyberbullying Detection

Abstract: Cyberbullying is a malady of social media, and its automatic detection is critically important considering its virulence, velocity of spreading, and the scale of the havoc it can wreak. However, the problem is challenging due to its disguised behavior, noise in the content, and, in recent times, introduction of code-mixing. In this work, we propose MTBullyGNN a novel graph neural network (GNN)-based multitask (MT) framework that solves sentiment-aided cyberbullying detection (CD) from code-mixed language. The GNN helps detect unlabelled or noisy label nodes (sentences) accurately by aggregating information from similarly labeled nodes. To connect nodes, we apply cosine similarity between sentences and create a single text graph for a benchmark code-mixed cyberbullying corpus, BullySent. Experimental results illustrate that MTBullyGNN outperforms the state-of-the-art (SOTA) methods for both the single (CD) and MT (CD and sentiment) settings by up to 4.46% and 4.92% in classification accuracy, respectively. Furthermore, another benchmark Hindi–English code-mixed single-task dataset has also been considered to illustrate the robustness of our proposed model. The code will be made publicly available in the camera-ready version.


Visual Analysis of Money Laundering in Cryptocurrency Exchange

Abstract: Blockchain-based cryptocurrencies, such as Bitcoin (BTC) and Ethereum (ETH), are newly emerging financial assets. Cryptocurrency exchanges are marketplaces for cryptocurrency circulation while becoming a new venue for money laundering. In this work, we cooperate with a cryptocurrency exchange to investigate new solutions for anti-money laundering in cryptocurrency exchanges. First, we learn the domain knowledge of cryptocurrency transactions and summarize data analytical requirements of transaction supervisors in their daily work of anti-money laundering. Then, we propose a visual analysis approach to support their daily work. The approach consists of a new algorithm that automatically detects suspicious money laundering accounts and a multiviewed user interface that visualizes the algorithm results and relevant transaction data. An abacus-inspired visualization is designed in the interface to depict transaction patterns contained in numerous cryptocurrency transactions, which can help supervisors find money laundering clues and deduce the trading tactic adopted by launderers. Finally, an algorithm performance experiment, a case study, and a field study are conducted with real-world data to demonstrate the effectiveness of our solution.


Blockchain Oriented Effective Charity Process During Pandemics and Emergencies

Abstract: During any emergency, a donation is considered a moral responsibility all over the globe. The lack of transparency and oversight in charity donations hurts people’s enthusiasm to donate. Donors are distrustful about how their funds are utilized. The use of blockchain technology (BCT) will provide a solution to make the donation procedure more viable. It is a distributed technology that offers a secure and transparent environment by avoiding the involvement of third parties between contributors and charities. This article proposed a blockchain-based donation mechanism for the convenience of charity organizations, donors, and beneficiaries during disasters, pandemics such as Covid-19, and other emergencies. All transactions can be traced in blockchain, giving donors visibility into where and how their funds are utilized. This article contributes to improving donations’ openness to strengthen public interest in donations and encourage BCT in charity. Ethereum blockchain is used to implement the proposed framework and provides a convenient donation platform. Smart contracts are used to make donations, which build trust between contributors, beneficiaries, and charity organizations. The blockchain-based donation method saves time, lowers donation costs, and eliminates the chances of dubious campaign funds. This study will contribute to improving emergency recovery efforts.


Zooming Into Video Conferencing Privacy

Abstract: The COVID-19 pandemic outbreak, with its related social distancing and shelter-in-place measures, has dramatically affected ways in which people communicate with each other, forcing people to find new ways to collaborate, study, celebrate special occasions, and meet with family and friends. One of the most popular solutions that have emerged is the use of video conferencing applications to replace face-to-face meetings with virtual meetings. This resulted in unprecedented growth in the number of video conferencing users. In this study, we explored privacy issues that may be at risk by attending virtual meetings. We extracted private information from collage images of meeting participants that are publicly posted on the Web. We used image processing, text recognition tools, as well as social network analysis to explore our web crawling curated dataset of over 15,700 collage images, and over 142,000 face images of meeting participants. We demonstrate that video conference users are facing prevalent security and privacy threats. Our results indicate that it is relatively easy to collect thousands of publicly available images of video conference meetings and extract personal information about the participants, including their face images, age, gender, usernames, and sometimes even full names. This type of extracted data can vastly and easily jeopardize people's security and privacy both in the online and real-world, affecting not only adults but also more vulnerable segments of society, such as young children and older adults. Finally, we show that cross-referencing facial image data with social network data may put participants at additional privacy risks they may not be aware of and that it is possible to identify users that appear in several video conference meetings, thus providing a potential to maliciously aggregate different sources of information about a target individual.


Hadoop-based secure storage solution for big data in cloud computing environment

Abstract: In order to address the problems of the single encryption algorithm, such as low encryption efficiency and unreliable metadata for static data storage of big data platforms in the cloud computing environment, we propose a Hadoop based big data secure storage scheme. Firstly, in order to disperse the NameNode service from a single server to multiple servers, we combine HDFS federation and HDFS high-availability mechanisms, and use the Zookeeper distributed coordination mechanism to coordinate each node to achieve dual-channel storage. Then, we improve the ECC encryption algorithm for the encryption of ordinary data, and adopt a homomorphic encryption algorithm to encrypt data that needs to be calculated. To accelerate the encryption, we adopt the dual-thread encryption mode. Finally, the HDFS control module is designed to combine the encryption algorithm with the storage model. Experimental results show that the proposed solution solves the problem of a single point of failure of metadata, performs well in terms of metadata reliability, and can realize the fault tolerance of the server. The improved encryption algorithm integrates the dual-channel storage mode, and the encryption storage efficiency improves by 27.6% on average.


A Novel DAO-Based Parallel Enterprise Management Framework in Web3 Era

Abstract: This article proposes a novel parallel management mode based on decentralized autonomous organizations (DAOs) for enterprises by utilizing the artificial systems, computational experiments, parallel execution (ACP) approach, parallel intelligence theory, and blockchain technologies, to realize the distributed management of an enterprise. The artificial enterprise DAO (EnDAO) corresponding to the actual enterprise is constructed, and they constitute a parallel system via virtual–real interaction and parallel execution. Through the non-fungible token (NFT)-based incentive mechanism, metaverse-based virtual learning and training, as well as DAO-based distributed management and decision-making, the management and control of the actual enterprise as well as its employees can be carried out. By virtue of the virtual–real interactions of three types of employees, as well as the virtual–real feedback of three closed loops in the parallel systems, DAO-based parallel management for enterprises can realize descriptive intelligence, predictive intelligence, and prescriptive intelligence. On this basis, this article takes the recruitment-oriented key performance indicator (KPI) management of a startup technology enterprise as the case to introduce the operation processes and illustrate the superiorities of the proposed DAO-based enterprise parallel management mode.


Transformer and GAN-Based Super-Resolution Reconstruction Network for Medical Images

Abstract: Super-resolution reconstruction in medical imaging has become more demanding due to the necessity of obtaining high-quality images with minimal radiation dose, such as in low-field magnetic resonance imaging (MRI). However, image super-resolution reconstruction remains a difficult task because of the complexity and high textual requirements for diagnosis purpose. In this paper, we offer a deep learning based strategy for reconstructing medical images from low resolutions utilizing Transformer and generative adversarial networks (T-GANs). The integrated system can extract more precise texture information and focus more on important locations through global image matching after successfully inserting Transformer into the generative adversarial network for picture reconstruction. Furthermore, we weighted the combination of content loss, adversarial loss, and adversarial feature loss as the final multi-task loss function during the training of our proposed model T-GAN. In comparison to established measures like peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), our suggested T-GAN achieves optimal performance and recovers more texture features in super-resolution reconstruction of MRI scanned images of the knees and belly.


Hybrid ultra-short-term PV power forecasting system for deterministic forecasting and uncertainty analysis

Abstract: The rapid development of the photovoltaic industry provides a new source of power for the continued operation of the over-consumed energy world. While providing new opportunities for global energy systems, it also poses challenges for power grids. Therefore, it is a priority to fully grasp the characteristics of photovoltaic changes and accurately forecast and analyze them. To enrich the existing research, a novel hybrid prediction system considering meteorological factors is constructed. First, a feature selection module is introduced to select features and assign weights to exogenous meteorological factors, which breaks through the limitations of single-data dimension prediction. Second, shallow and deep learning models are flexibly applied and multi-objective intelligent optimization strategies are introduced to construct deterministic combinatorial prediction models. The module can effectively increase the diversity of prediction models while fully weighing the accuracy and stability of prediction to meet the needs of different information users. Finally, an interval prediction model is constructed to further enrich the PV power prediction system from the perspective of uncertainty analysis. The empirical study is carried out with 5-minute interval data at three sites, and the results show that the hybrid system obtains superior out-of-sample forecasting performance with technical feasibility and general applicability.


Dense Text Retrieval Based on Pretrained Language Models: A Survey

Abstract: Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user’s queries in natural language. From heuristic-based retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn text representations and model the relevance matching. The recent success of pretrained language models (PLM) sheds light on developing more capable text-retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the semantic representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is called dense retrieval , since it employs dense vectors to represent the texts. Considering the rapid progress on dense retrieval, this survey systematically reviews the recent progress on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related studies by four major aspects, including architecture, training, indexing and integration, and thoroughly summarize the mainstream techniques for each aspect. We extensively collect the recent advances on this topic, and include 300+ reference papers. To support our survey, we create a website for providing useful resources, and release a code repository for dense retrieval. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval.


Fusion of transfer learning models with LSTM for detection of breast cancer using ultrasound images

Abstract: Breast Cancer (BC) is one of the top reasons for fatality in women worldwide. As a result, timely identification is critical for successful therapy and excellent survival rates. Transfer Learning (TL) approaches have recently shown promise in aiding in the early recognition of BC. In this work, three TL models, MobileNetV2, ResNet50, and VGG16, were combined with LSTM to extract the features from Ultrasound Images (USIs). Furthermore, the Synthetic Minority Over-sampling Technique (SMOTE) with Tomek (SMOTETomek) was employed to balance the extracted features. The proposed method with VGG16 achieved an F1 score of 99.0 %, Matthews Correlation Coefficient (MCC) and Kappa Coefficient of 98.9 % with an Area Under Curve (AUC) of 1.0. The K-fold method was applied for cross-validation and achieved an average F1 score of 96 %. Moreover, the Gradient-weighted Class Activation Mapping (Grad-CAM) method was applied for visualization, and the Local Interpretable Model-agnostic Explanations (LIME) method was applied for interpretability. The Normal Approximation Interval (NAI) and bootstrapping methods were used to calculate Confidence Intervals (CIs). The proposed method achieved a Lower CI (LCI), Upper CI (UCI), and Mean CI (MCI) of 96.50 %, 99.75 %, and 98.13 %, respectively, with the NAI, while 95 % LCI of 93.81 %, an UCI of 96.00 %, and a bootstrap mean of 94.90 % with the bootstrap method. Furthermore, the performance of the six state-of-the-art (SOTA) TL models, such as Xception, NASNetMobile, InceptionResNetV2, MobileNetV2, ResNet50, and VGG16, were compared with the proposed method.


A Novel Tensor Decomposition-Based Efficient Detector for Low-Altitude Aerial Objects With Knowledge Distillation Scheme

Abstract: Unmanned aerial vehicles (UAVs) have gained significant attention in practical applications, especially the low-altitude aerial (LAA) object detection imposes stringent requirements on recognition accuracy and computational resources. In this paper, the LAA images-oriented tensor decomposition and knowledge distillation-based network (TDKD-Net) is proposed, where the TT-format TD (tensor decomposition) and equal-weighted response-based KD (knowledge distillation) methods are designed to minimize redundant parameters while ensuring comparable performance. Moreover, some robust network structures are developed, including the small object detection head and the dual-domain attention mechanism, which enable the model to leverage the learned knowledge from small-scale targets and selectively focus on salient features. Considering the imbalance of bounding box regression samples and the inaccuracy of regression geometric factors, the focal and efficient IoU (intersection of union) loss with optimal transport assignment (F-EIoU-OTA) mechanism is proposed to improve the detection accuracy. The proposed TDKD-Net is comprehensively evaluated through extensive experiments, and the results have demonstrated the effectiveness and superiority of the developed methods in comparison to other advanced detection algorithms, which also present high generalization and strong robustness. As a resource-efficient precise network, the complex detection of small and occluded LAA objects is also well addressed by TDKD-Net, which provides useful insights on handling imbalanced issues and realizing domain adaptation.


Spherical autoregressive models, with application to distributional and compositional time series

Abstract: We introduce a new class of autoregressive models for spherical time series. The dimension of the spheres on which the observations of the time series are situated may be finite-dimensional or infinite-dimensional, where in the latter case we consider the Hilbert sphere. Spherical time series arise in various settings. We focus here on distributional and compositional time series. Applying a square root transformation to the densities of the observations of a distributional time series maps the distributional observations to the Hilbert sphere, equipped with the Fisher–Rao metric. Likewise, applying a square root transformation to the components of the observations of a compositional time series maps the compositional observations to a finite-dimensional sphere, equipped with the geodesic metric on spheres. The challenge in modeling such time series lies in the intrinsic non-linearity of spheres and Hilbert spheres, where conventional arithmetic operations such as addition or scalar multiplication are no longer available. To address this difficulty, we consider rotation operators to map observations on the sphere. Specifically, we introduce a class of skew-symmetric operators such that the associated exponential operators are rotation operators that for each given pair of points on the sphere map the first point of the pair to the second point of the pair. We exploit the fact that the space of skew-symmetric operators is Hilbertian to develop autoregressive modeling of geometric differences that correspond to rotations of spherical and distributional time series. Differences expressed in terms of rotations can be taken between the Fréchet mean and the observations or between consecutive observations of the time series. We derive theoretical properties of the ensuing autoregressive models and showcase these approaches with several motivating data. These include a time series of yearly observations of bivariate distributions of the minimum/maximum temperatures for a period of 120 days during each summer for the years 1990-2018 at Los Angeles (LAX) and John F. Kennedy (JFK) international airports. A second data application concerns a compositional time series with annual observations of compositions of energy sources for power generation in the U.S..


Career Mobility Analysis With Uncertainty-Aware Graph Autoencoders: A Job Title Transition Perspective

Abstract: Career mobility analysis aims at discovering the movement patterns of employees across different job positions or grades, which can benefit various human resource-related applications. Indeed, recent studies in this direction mainly focus on modeling individual career trajectories, while the macroguidance for labor market assessment has been largely ignored. To this end, in this article, we propose to study career mobility from a market-driven perspective based on large-scale online professional networks (OPNs). Specifically, we propose an uncertainty-aware graph autoencoders (UnGAEs) framework, which can simultaneously discover potential job title transition patterns and predict job durations. In this phase, we first construct a job title transition graph based on massive career trajectory data from OPNs. Then, considering the inherent uncertainty in career mobility, we introduce a novel uncertainty-aware graph encoder (UnGE) to represent job titles as Gaussian embeddings. Furthermore, we design two task-specific decoders that can preserve the asymmetric relationships between job titles, namely the gravity-inspired decoder (GID) and the energy-inspired decoder (EID), for predicting potential transition patterns and corresponding duration, respectively. In particular, both tasks are modeled through a specially designed multitask learning approach. Finally, extensive experiments on a real-world dataset clearly demonstrate the effectiveness of UnGAE compared with state-of-the-art baselines, as well as some potential applications such as job title benchmarking and career path planning.


CSC-Net: Cross-Color Spatial Co-Occurrence Matrix Network for Detecting Synthesized Fake Images

Abstract: Recently, the Generative Adversarial Networks (GAN) generated images have been spread over the social networks, which brings the new challenge in the the community of media forensics. Although some reliable forensic tools have advanced the study of detecting GAN generated images, while the detection accuracy cannot be guaranteed when facing the malicious post-processing attacks, especially in the practical social network scenario. Thus, in this context, we propose a novel well-designed deep neural network equipped with handed-crafted features for dealing with this problem. In particular, relying on the Cross-color Spatial Co-occurrence Matrix (CSCM), the discriminative features are extracted after carefully analyzing and selecting the most effective color channels. Next, the fused features are fed into the deep neutral network for training a high-efficient forensic detector. Extensive experimental results empirically verify that in most detection scenarios, our proposed detector performs superiorly to the prior arts, especially in the case of post-processing attacks. Moreover, we also highlight the relevance of the proposed detector over the realistic social network platforms, and its generalization capability in three different scenarios.


An intelligent active probing and trace-back scheme for IoT anomaly detection

Abstract: Due to their simple hardware, sensor nodes in IoT are vulnerable to attack, leading to data routing blockages or malicious tampering, which significantly disrupts secure data collection. An Intelligent Active Probing and Trace-back Scheme for IoT Anomaly Detection (APTAD) is proposed to collect integrated IoT data by recruiting Mobile Edge Users (MEUs). (a) An intelligent unsupervised learning approach is used to identify anomalous data from the collected data by MEUs and help to identify anomalous nodes. (b) Recruit MEUs to trace back and propose a series of trust calculation methods to determine the trust of nodes. (c) The last, the number of active detection packets and detection paths are designed, so as to accurately identify the trust of nodes in IoT at the minimum cost of the network. A large number of experimental results show that the recruiting cost and average anomaly detection time are reduced by 6.5 times and 34.33% respectively, while the accuracy of trust identification is improved by 20%.


Consensus Synergizes With Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes

Abstract: Anomaly segmentation is a critical task for safety-critical applications, such as autonomous driving in urban environments. Its objective is to detect out-of-distribution (OOD) samples with unseen categories, given a pre-trained segmentation model. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed in previous research. In this paper, we propose a simple yet effective approach named CosMe (Consensus Synergizes with Memory) to address this challenge. CosMe consists of two key components: 1) building a memory bank comprising seen prototypes extracted from multiple layers of the given segmentation model, and 2) training an auxiliary model that mimics the behavior of the given model and using the consensus of their mid-level features as complementary cues that synergize with the memory bank. The former serves as a baseline that can detect all potential outliers, including both OOD and hard in-distribution samples; the latter assists in distinguishing between these two types of outliers. Experimental results on several urban scene anomaly segmentation datasets demonstrate that CosMe outperforms previous approaches by a significant margin.


Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computation

Abstract: Federated learning (FL) enables mobile devices to collaboratively learn a shared prediction model while keeping data locally. However, there are two major research challenges to practically deploy FL over mobile devices: (i) frequent wireless updates of huge size gradients v.s. limited spectrum resources, and (ii) energy-hungry FL communication and local computing during training v.s. battery-constrained mobile devices. To address those challenges, in this paper, we propose a novel multi-bit over-the-air computation (M-AirComp) approach for spectrum-efficient aggregation of local model updates in FL and further present an energy-efficient FL design for mobile devices. Specifically, a high-precision digital modulation scheme is designed and incorporated in the M-AirComp, allowing mobile devices to upload model updates at the selected positions simultaneously in the multi-access channel. Moreover, we theoretically analyze the convergence property of our FL algorithm. Guided by FL convergence analysis, we formulate a joint transmission probability and local computing control optimization, aiming to minimize the overall energy consumption (i.e., iterative local computing + multi-round communications) of mobile devices in FL. Extensive simulation results show that our proposed scheme outperforms existing ones in terms of spectrum utilization, energy efficiency, and learning accuracy.


Training GAN with predictive centripetal acceleration

Abstract: To alleviate the issue of limit cycle behavior in training generative adversarial networks (GAN), in this paper, we draw inspiration from the centripetal acceleration algorithm and the modified predictive method (MPM) by Liang and Stokes (2019). Building upon geometric observation of uniform circular motion, we propose the predictive centripetal acceleration algorithm (PCA). First and foremost, we prove the last-iterate convergence of the PCA on the bilinear game, which is a special case of the GAN. Besides, by combining PCA with the stochastic gradient descent (SGD) algorithm and adaptive moment estimation (Adam) algorithm, we propose two variants, which are called stochastic PCA (SPCA) and PCA-Adam, for the practical training GAN. Last but not least, experiments conducted on the bilinear game, multivariate Gaussian distribution, and CIFAR10 and CelebA datasets validate the effectiveness of the proposed algorithm.


Application Prioritization Engine for Enhancing Real-Time Performance in Smartphones

Abstract: Smartphone consumers use various applications (apps), including online gaming, chat, streaming, video calling, and social networking. The smartphone relies on the network backhaul, such as a Wi-Fi access point, to provide the required Quality of Service (QoS). In its send-and-receive queues, the smartphone processes the packets in a first-in-first-out (FIFO) fashion. Many people worldwide started using video calling apps daily during the pandemic and post-pandemic periods. On the other hand, online gaming apps skyrocketed and continue to engage people. When real-time (RT) video calling and gaming apps race with non-real-time (NRT) traffic, we found a severe degradation in the Quality of Experience (QoE). In this work, we propose an Application Prioritization Engine (APE) framework that will improve user experience by dynamically allocating bandwidth to the different apps in the smartphone. APE helps improve the end-user experience by detecting and prioritizing real-time traffic over concurrent best-effort traffic. We introduced an eBPF (extended Berkeley Packet filter) that can control the NRT traffic to the extent that it does not affect the RT traffic. We evaluate the performance of APE with the top-chart video calling and gaming apps in a live-air scenario. APE enhances video calling performance in poor network conditions by improving the bit rate to 110%. Furthermore, it provides a four-fold gaming latency reduction despite NRT traffic. APE is a tech-transferred, app-agnostic, server-independent solution enabled in the latest Samsung flagship Smartphones with Android 13 OS.


See and Learn More: Dense Caption-Aware Representation for Visual Question Answering

Abstract: With the rapid development of deep learning models, great improvements have been achieved in the Visual Question Answering (VQA) field. However, modern VQA models are easily affected by language priors, which ignore image information and learn the superficial relationship between questions and answers, even in the optimal pre-training model. The main reason is that visual information is not fully extracted and utilized, which results in a domain gap between vision and language modalities to a certain extent. In order to mitigate the circumstances, we propose to extract dense captions (auxiliary semantic information) from images to enhance the visual information for reasoning and utilize them to release the gap between vision and language since the dense captions and the questions are from the same language modality (i.e., phrase or sentence). In this paper, we propose a novel dense caption-aware visual question answering model called DenseCapBert to enhance visual reasoning. Specifically, we generate dense captions for the images and propose a multimodal interaction mechanism to fuse dense captions, images, and questions in a unified framework, which makes the VQA models more robust. The experimental results on GQA, GQA-OOD, VQA v2, and VQA-CP v2 datasets show that dense captions are beneficial to improving the model generalization and our model effectively mitigates the language bias problem.


ViChaser: Chase Your Viewpoint for Live Video Streaming With Block-Oriented Super-Resolution

Abstract: The usage of live streaming services has led to a substantial increase in live video traffic. However, the perceived quality of experience of users is frequently limited by variations in the upstream bandwidth of streamers. To address this issue, several adaptive bitrate (ABR) algorithms have been developed to mitigate bandwidth variations. Nevertheless, the ability of users to enjoy high-quality live streams remains limited. While neural-enhanced approaches, such as super-resolution, offer significant quality improvements, frame-oriented super-resolution leads to excessive inference delay that violates the real-time feature of live streaming. In response, we propose ViChaser, which examines block-oriented super-resolution for live streaming. ViChaser performs neural super-resolution on potential blocks of interest in the media server, corresponding to the user’s viewpoint, and uses online learning to adapt to the dynamic content of the video. Additionally, ViChaser utilizes the Lyapunov framework to efficiently allocate uplink bandwidth for original low-quality live video and high-quality labels. The experimental results demonstrate that ViChaser achieves 1.2–1.5 dB higher video quality in Peak-Signal-to-Noise-Ratio than WebRTC and increases processing speed by 11–16 fps relative to LiveNAS.


Adversarial Attacks for Neural Network-Based Industrial Soft Sensors: Mirror Output Attack and Translation Mirror Output Attack

Abstract: Soft sensing using the neural network technique has been increasingly applied to industrial processes. Recently, the security and robustness of neural network based soft sensors have become primary concerns. Additionally, current studies indicated that neural networks are vulnerable to adversarial attacks. In other words, small perturbations imposed on the input can lead to significant deviations in the output. If a soft sensor for key process variables is attacked, considerable damage may be brought to industrial processes. This paper focuses on the attack methods for neural network based industrial soft sensors. Considering the characteristics of industrial soft sensors, this paper proposes two new adversarial attack methods. The first method, called the Mirror Output Attack (MOA), is a subtle attack method that flips the output curve to change the direction of outputs. The second method, called the Translation Mirror Output Attack (TMOA), is easy to make operators misoperate. TMOA translates the output curve while flipping the output curve to achieve the purpose of changing the output conditions. The effectiveness of MOA and TMOA is demonstrated in an industrial case study of the sulfur recovery unit (SRU) process. Simulation results show that the neural network based industrial soft sensors can be attacked by both of the two proposed adversarial attack methods. The study of adversarial attack methods can provide a basis for defending against attacks, thereby enhancing the security and robustness of soft sensors.


Information Transfer in Semi-Supervised Semantic Segmentation

Abstract: Enhancing the accuracy of dense classification with limited labeled data and abundant unlabeled data, known as semi-supervised semantic segmentation, is an essential task in vision comprehension. Due to the lack of annotation in unlabeled data, additional pseudo-supervised signals, typically pseudo-labeling, are required to improve the performance. Although effective, these methods fail to consider the internal representation of neural networks and the inherent class-imbalance in dense samples. In this work, we propose an information transfer theory, which establishes a theoretical relationship between shallow and deep representations. We further apply this theory at both the semantic and pixel levels, referred to as IIT-SP, to align different types of information. The proposed IIT-SP optimizes shallow representations to match the target representation required for segmentation. This limits the upper bound of deep representations to enhance segmentation performance. We also propose a momentum-based Cluster-State bar that updates class status online, along with a HardClassMix augmentation and a loss weighting technique to address class imbalance issues based on it. The effectiveness of the proposed method is demonstrated through comparative experiments on PASCAL VOC and Cityscapes benchmarks, where the proposed IIT-SP achieves state-of-the-art performance, reaching mIoU of 68.34% with only 2% labeled data on PASCAL VOC and mIoU of 64.20% with only 12.5% labeled data on Cityscapes.


Leveraging Graph Neural Networks for SLA Violation Prediction in Cloud Computing

Abstract: In this paper we examine different approaches for the prediction of Service Level Agreements (SLAs) violations that occur during the service provisioning between cloud customers and providers.Despite the fact that there are many network metrics that involve the server -client interaction, it is an open research question how these available metrics can be used by a SLA prediction mechanism.We study three different data representation models for the network characteristics, a time series, a content and a context representation.We see that a context approach using graph representations captures efficiently the associativity of clients and improves the performance of traditional SLA violation prediction models when it is combined with them.The prediction of the SLA violations takes place using neural networks, making us propose a composite SLA prediction model that leverages Graph Neural Networks (GNNs).In our research, we put special emphasis and try different variations on how we construct the graphs.We perform an extensive performance evaluation of 23 different SLA prediction models that can be grouped into the three representations categories, namely the vector models that are based on network features, sequential models that leverage the temporal evolution of QoS metrics and Graph models that take into consideration the associativity of the clients.The experimental results show that our proposed GNN-based model can significantly improve the accuracy of SLA violation prediction, making it a useful tool for Cloud and Service providers.


Encrypted network traffic classification based on machine learning

Abstract: Encrypted traffic is an essential part of maintaining the security and privacy of data transmission. It plays an important role in keeping our networks secure by preventing attackers from intercepting confidential information, which they may access without authorization; However, its effectiveness relies heavily on accurate classification techniques being applied correctly, so we can differentiate between legitimate users' activities versus those attempting malicious activity within the networks’ boundaries. Encrypted network traffic is becoming increasingly common in modern communication systems, presenting a challenge for effective network management and security. To address this challenge, machine learning models have been employed to classify encrypted traffic but with limited success due to the lack of clear visibility into packet contents and an inability to inspect their content. For the sake of tackling this issue, more effective research has begun on developing machine learning models for classifying encrypted payloads without relying on inspecting their contents directly. This research will investigate how features like packet length, time stamps or transport layer security (TLS) and encrypted payload information can be used as input features when attempting classification tasks, instead of analyzing unencrypted content directly from packets themselves which would otherwise be impossible given the current technology constraints. The evaluation process will focus on assessing different model architectures, as well as feature selection techniques that yield improved results over the existing approaches. In this paper, we proposed three approaches to identify encrypted traffic and classify different applications such as browsing, VOIP, file transfer and video streaming. The first two techniques consist of two stages: the first stage is either a neural network or a bi-directional LSTM, and the second stage is a selection of different classification techniques, namely Random Forest, Support vector machine, Linear regression, and K-nearest neighbor. The final result is achieved using an ensemble voting technique. As for the third technique, the network packets are grouped together by Source IP, destination IP and session time before feeding them into three different combinations of LSTM networks; either coupled with convolution 1D or 2D layers, or without. Like the first two techniques, the final result is achieved by means of ensemble voting. Through extensive comparison between the three approaches, The first approach yielded the highest accuracy. However, the performance of the second and third techniques in terms of time complexity was superior. The achieved accuracies were 96.8%, 95.2% and 96.5% for the proposed techniques, respectively.


RAFNet: Reparameterizable Across-Resolution Fusion Network for Real-Time Image Semantic Segmentation

Abstract: The demand to implement semantic segmentation networks on mobile devices has increased dramatically. However, existing real-time semantic segmentation methods still suffer from a large number of network parameters, unsuitable for mobile devices with limited memory resources. The reason mainly arises from the fact that most existing methods take the backbone networks (e.g., ResNet-18 and MobileNet) as an encoder. To alleviate this problem, we propose a novel Reparameterizable Channel & Dilation (RCD) block and construct a considerably lightweight yet effective encoder by stacking several RCD blocks according to three guidelines. The strengths of the proposed encoder result in the abilities not only to extract discriminative feature representations via channel convolutions and dilated convolutions, but also to reduce computational burdens while maintaining segmentation accuracy with the help of re-parameterization technique. Except for encoder, we also present a simple but effective decoder that adopts an across-resolution fusion strategy to fuse multi-scale feature maps generated from the encoder instead of a bottom-up pathway fusion. With such an encoder and a decoder, we provide a Reparameterizable Across-resolution Fusion Network (RAFNet) for real-time semantic segmentation. Extensive experiments demonstrate that our RAFNet achieves a promising trade-off between segmentation accuracy, inference speed and network parameters. Specifically, our RAFNet with only 0.96M parameters obtains 75.3% mIoU at 107 FPS and 75.8% mIoU at 195 FPS on Cityscapes and CamVid test sets for full-resolution inputs, respectively. After quantization and deployment on a Xilinx ZCU104 device, our RAFNet obtains a favorable segmentation performance with only 1.4W power.


HybridAD: A Hybrid Model-Driven Anomaly Detection Approach for Multivariate Time Series

Abstract: Anomaly detection, in recent years, has gained increasing attention in the research and practice of time series processing. However, the task is particularly challenging with multivariate time series which complicates the temporal dependency between observations and introduces complex inter-channel correlation. Meanwhile, in order to fit a broader range of applications, robustness during both training and detection is also a critical aspect. In this paper, we propose an unsupervised, hybrid model-driven anomaly detection scheme capable of (1) transforming sequences into a fused representation of temporal dependency embeddings and inter-channel correlation embeddings, and (2) achieving robust anomaly detection using a temporal prediction network for sample-wise posterior estimation combined with a data reconstruction network to assess the source of prediction. On this basis, we develop a probability density-based anomaly scoring mechanism for online detection in multivariate time series, where the anomaly score for each observation is rectified by the reliability of the prediction source. The results of extensive experiments on five publicly available datasets show that our proposed solution outperforms various state-of-the-art anomaly detection algorithms (including DL-based and non-DL-based), achieving a performance improvement (in F1-Score) by up to 10.42%.


Clustering asymmetrical data with outliers: Parsimonious mixtures of contaminated mean-mixture of normal distributions

Abstract: Mixture modeling has emerged as a statistical tool to perform unsupervised model-based clustering for heterogeneous data. A framework of using contaminated mean-mixture of normal distributions as the components of the mixture model is designed to accommodate asymmetric data with outliers. Fourteen parsimonious variants of the postulated model are introduced by employing an eigenvalue decomposition of the component scale matrices. Simultaneously clustering and outliers detection is an outstanding advantage of the proposed model in analyzing non-normally distributed data. A computationally feasible and flexible EM-type algorithm is outlined for obtaining maximum likelihood parameter estimates. Moreover, the score vector and empirical information matrix for calculating asymptotic standard errors of the parameter estimates are derived by offering an information-based approach. The applicability of the proposed method is demonstrated through the analysis of simulated and real datasets with varying proportions of outliers.


Multi-View Projection Based Joint Geometry and Color Hole Repairing Method for G-PCC Trisoup Encoded Color Point Cloud

Abstract: Efficient compression of color point clouds (CPC) is crucial for immersive visual systems. The G-PCC standard by Moving Picture Experts Group offers effective compression solutions for static CPCs. G-PCC Trisoup, one of the G-PCC solutions, achieves high coding performance using a pruned octree, combined with triangle soup (trisoup) surface encoding. However, encoding errors introduce distortions like triangular holes, degrading visual quality. This article proposes a multi-view projection based joint geometry and color hole repairing method for G-PCC trisoup encoded color point cloud, significantly improving reconstruction quality. Firstly, a G-PCC Trisoup encoded CPC is projected onto multi-view color and geometry maps. A multi-view projection-based scheme detects triangular holes by identifying geometry discontinuities. A geometric weighting factor, considering direction, distance, and level set distance, repairs geometry maps. Additionally, a novel gated hole repairing network repairs color maps by leveraging semantic information and reducing background interference. Finally, the joint geometry and color repaired results are inversely projected to obtain the final CPCs. The proposed method is validated on distorted CPCs encoded with G-PCC Trisoup at different levels. Experimental results demonstrate effective improvement in geometry and color objective quality, leading to enhanced visual perception.


CFTO: Communication-Aware Fairness in Blockchain Transaction Ordering

Abstract: Blockchain leader-based protocols elect leaders for proposing the next block of transactions. Proposed blocks need to pass a validation routine in order to be added to the blockchain. Proposers may prioritize certain transactions based on their fees or accounts, which enables attackers to gain profits through block building, while simultaneously causing negative impacts on other users. A fair block selection follows a random selection of pending transactions among those that a proposer is aware of. We propose CFTO, a protocol that aims at encouraging fair block selection in a leader-based blockchain network while taking into account real network conditions, such as the network’s topology structure and the forwarding protocol. CFTO offers two main contributions. First, it provides incentives for acting honestly and diminishing malicious and dishonest nodes. To accomplish this, we use a reputation system, whereby each node is given a reputation score based on its actions. Second, it consists of an algorithm that evaluates the proposed blocks based on the zone structure of the network. Furthermore, we adapt the evaluation algorithm to fit the additional order constraints implied in Ethereum transaction ordering. We demonstrate the improved accuracy of CFTO in detecting fair blocks, in terms of increasing the probability of approving fair blocks and decreasing the probability of approving unfair blocks, by implementing experiments and comparing them with Helix [1], a previously proposed consensus algorithm for fair block selection. As part of our experiments, we also compare certain features with those of previous studies.


FRAD: Free-Rider Attacks Detection Mechanism for Federated Learning in AIoT

Abstract: The rapid development of the Artificial Intelligence of Things (AIoT) opens up a new perspective for emerging service-based applications, and becomes a major driver of diverse Federated Learning (FL) applications. However, due to the heterogeneity of nodes and the existence of free-rider attacks, some device nodes may launch free-rider attacks to obtain the global model without any contribution, which not only dampens the enthusiasm of legitimate participants but also undermines the fairness of node contributions. In this paper, we propose a free-rider attack detection mechanism (FRAD) for federated learning with deep autoencoding gaussian mixture model based on contribution and reputation. Specifically, we first model the contribution values based on the computing resource, communication cost, and data quality of each device node. Then, based on PageRank algorithms, we design an optimal reputation-based model to fairly and precisely choose benign nodes to participate in federated training under information asymmetry. Furthermore, we develop FRAD mechanism via deep autoencoding gaussian mixture model that combines historical contribution with reputation scores. Simulation results validate that the proposed mechanism in this paper outperforms the state-of-the-art baselines ( ∼×2.14 and ∼×1.1 on MNIST and CIFAR, respectively) in defending against free-rider attacks, and when the main clients are free-riders, i.e., 50% or even up to 80% of the free-riders, FRAD can still maintain a high defense performance against free-rider attacks.


Low-Rank Sparse Generative Adversarial Unsupervised Domain Adaptation for Multi-Target Traffic Scene Semantic Segmentation

Abstract: Semantic segmentation in diverse real-world traffic scenes is a challenging task for autonomous vehicles to have a reliable understanding of the outside environment. Although deep visual models show a remarkable performance on traffic scene segmentation, they need significant improvements when the source and target scene distributions are different. The weather and illumination differences between source and target datasets result in dramatic distribution shifts which degrade the target segmentation performance. This paper develops an unsupervised domain adaptation (UDA) model to address complex domain shift problems where the dataset, weather, and illumination of source and target traffic scenes are different. We devise a sparse adversarial multi-target UDA approach to capture powerful domain-invariant features to segment traffic scenes in different conditions. First, a sparse representation of the source traffic scenes is captured via a new spectral low-rank dictionary learning technique in the latent space of a deep encoder-decoder segmentation architecture. The feature sparsity provides high generalization capacity for our deep feature extractor which enables us to compute complex visual patterns of source images. Then, the distribution of source sparse features is learned using a generative adversarial framework. Finally, the sparse representations of source and target scenes are aligned via a sparse domain-invariant feature extractor trained by min-max optimization. The aligned features serve as domain-invariant scene representations that can best describe both source and target scenes; hence providing deep domain adaptation for traffic scene semantic segmentation. Experiments on a real-world dataset indicate the superiority of the proposed model compared to the state-of-the-art methodologies.


Transmitter Identification With Contrastive Learning in Incremental Open-Set Recognition

Abstract: Radio frequency fingerprints are commonly exploited as a unique signature in the physical layer for distinguishing transmitters in Transmitter Identification Systems (TIS). In response to the growing demand for TIS in open dynamic scenarios, this paper proposes the Incremental Open-Set Recognition (IOSR) framework to address IOSR tasks, which involve changes in transmitter categories, component replacements, and open-set recognition. To overcome the limitations of traditional methods, the proposed framework focuses on enhancing the security, adaptability, reliability, and efficiency of TIS. Specifically, a well-designed data representation and a lightweight extractor based on supervised contrastive learning are introduced to improve inter-class discriminative ability and intra-class compactness for feature extraction. The incorporation of MobileNetV3 reduces the training parameters of the extractor while improving computational efficiency. Moreover, an adaptive evolved block is designed to mitigate catastrophic forgetting in incremental learning, preserving historical knowledge and enhancing system scalability and adaptability. Additionally, an enhanced open-set recognizer is proposed to establish a suitable open-set decision boundary through output calibration. The performance evaluation of the framework on the WiFi dataset showcases its superiority over existing approaches in the closed-set recognition, achieving an accuracy of over 99.6%. It also performs well in incremental tasks, with an accuracy exceeding 98.9%. In the open-set recognition, the framework achieves an accuracy improvement of approximately 8%. Moreover, it demonstrates superior accuracy in the IOSR task, outperforming other algorithms by more than 5.8%. Furthermore, ablation experiments provide further evidence of the effectiveness of the proposed framework, while a complexity comparison demonstrates its ability to balance computational load and accuracy.


A Secure Sidechain for Decentralized Trading in Internet of Things

Abstract: Sidechains allow transaction dissemination and execution outside the blockchain main network (i.e., the mainchain), enabling a scalable, efficient, and secure financial infrastructure for the Internet of Things (IoT) without trusting any central authority. Existing sidechains either have online requirements or rely on intensive computation on a central operator, which does not meet the needs of IoT for dynamic changes and high performance. This paper proposes an alternative sidechain construction, called Cumulus, which meets the needs of IoT by leveraging the classic Byzantine fault-tolerant (BFT) consensus protocols such as PBFT that have commonly been applied in permissioned blockchains. Cumulus builds BFT-based sidechains atop public blockchains (e.g., Ethereum) using smart contracts and ensures the bidirectional safety of users’ assets. Cumulus sidechains periodically interact with the mainchain and submit checkpoints through representatives selected in an efficient and decentralized manner. The experiments show that Cumulus sidechains outperform rollup-based sidechains, and state-of-the-art sidechain constructions, achieving two and three orders of magnitude improvement in throughput and latency while retaining comparable operational cost.


Multi-Party Privacy-Preserving Faster R-CNN Framework for Object Detection

Abstract: Faster region-based conventional neural network (Faster R-CNN) is a common algorithm for object detection that identifies the object and their location information through three steps: feature extraction, region proposal network and classification. However, there are data privacy issues in the training and prediction of Faster R-CNN. Thus we design a multi-party privacy-preserving Faster R-CNN framework for object detection named SPFR. Specifically, we extend the existing sub-protocols and achieve high-precision division, exponentiation and logarithm calculation through the idea of blinding and oblivious transfer (OT) protocols. Then we design a series of key privacy-preserving protocols that satisfy the secure computational requirements based on the above-mentioned sub-protocols and prove that these protocols are security in the semi-honest adversary model. Finally the proposed protocols are implemented in Pytorch, and the experimental findings demonstrate the efficiency of the protocols in Faster R-CNN.


ATAL: Active Learning Using Adversarial Training for Data Augmentation

Abstract: Active learning (AL) tries to maximize the model’s performance when the labeled dataset is limited, and the annotation cost is high. Although it can be efficiently implemented in deep neural networks (DNNs), it is questionable whether the model can maintain the ability to generalize well when there are significant distributional deviations between the labeled and unlabeled datasets. In this paper, we consider introducing adversarial training and adversarial samples into active learning to mitigate the problem of degraded generalization performance due to different data distributions. In particular, our proposed adversarial training active learning (ATAL) has two advantages, and one is that adversarial training by different networks enables the network to have better prediction performance and robustness with limited labeled samples. And the other is that the adversarial samples generated by the adversarial training can effectively expand the labeled dataset so that the designed query function can efficiently select the most informative unlabeled samples based on the expanded labeled dataset. Extensive experiments have been performed to verify the feasibility and efficiency of our proposed method, i.e., CIFAR-10 demonstrates the effectiveness of our method — new state-of-the-art robustness and accuracy are achieved.


Identity-Based Multiproxy Signature With Proxy Signing Key for Internet of Drones

Abstract: A multi-proxy signature scheme constructs a legitimate signature by a proxy group when an original signer is absent. In this paper, we newly propose an efficient and provably secure ID-Based Multi-Proxy Signature (IBMPS) from bilinear pairings based on a random proxy signing key. We observe that a proxy signing key from a Proxy Key Generation (PKG) makes IBMPS much more efficient and secure than directly using a private key for a multi-proxy signature. We show formal security results under the CDH assumption with PKG queries in the random oracle. Our scheme not only preserves provable security but also achieves the least computation cost in the signing and verification phase than other IBMPS schemes. To show its applicability in Internet-of-Drones (IoD), which usually demands greater efficiency due to the limited resources while the drones are in flight, we also present a practical on/offline IBMPS for an IoD setting. Our on/offline IBMPS scheme just requires one point scalar addition for generating a proxy signature in terms of each proxy signer. Considering that our IBMPS schemes can be practical in urgent situations, e.g., surveillance and rescue operations, we show our implementation of IBMPS schemes on a Raspberry Pi drone platform along with the relevant schemes. We demonstrate that our schemes are considerably more efficient than others with respect to computation time and energy consumption by the number of proxy signers.