GrassCaré: Visualizing the Grassmannian on the Poincaré Disk

Abstract: Abstract This paper introduces a novel method for visualizing high-dimensional Grassmannians through 2D embeddings on the Poincaré disk. The proposed approach involves the construction of affinity matrices on each manifold, followed by the minimization of KL-divergence between the geodesics affinity. This process enables the identification of an optimal projection that effectively preserves the geometry of the original high-dimensional Grassmannian. Our main theoretical contribution lies in bounding the embedding loss with respect to two factors: the logarithm of the number of subspaces and a term dependent on the distribution of subspaces within the Grassmannian. Notably, this term is smaller when subspaces exhibit well-defined clusters, and larger in the absence of any discernible structure. We complement our theoretical analysis with comprehensive experiments on both synthetic and real datasets. The experimental results showcase the superiority of our embedding in accurately visualizing Grassmannians compared to existing representations.


Tensions between the political, institutional, and project levels when developing professional digital competence in teacher education. A cultural historical activity theory analysis of inhibiting and facilitating factors

Abstract: This study is a Cultural Historical Activity Theory analysis of the tensions between object and motive at the political, institutional and project management levels for five digitalisation projects aimed at integrating professional digital competence (PDC) in Norwegian teacher education (TE). The study is inspired by ethnography and builds on a variety of qualitative data (including qualitative interviews (N = 14)) gathered through the author’s participation in different roles (2017–2021). Identifying objects and motives across levels allows for the identification of factors inhibiting and facilitating factors the development of PDC in TE. An analytical model was developed to identify the objects and motives at each level, showing a political object of preparing for the future, an institutional object of creating high quality TE, and a project management object of delivering a lasting impact on TE. These differing objects and motives guide the activities at each level and reveal tension, including between the political push for change, project management’s attempts to deliver impact, and a culture built on academic freedom which seems counterproductive by slowing things down. The study also reveals facilitating factors, such as funding and relevance in collaboration with schools, and inhibiting factors like academic freedom and the expected speed of change.


Dickson polynomial-based secure group authentication scheme for Internet of Things

Abstract: Internet of Things (IoT) paves the way for the modern smart industrial applications and cities. Trusted Authority acts as a sole control in monitoring and maintaining the communications between the IoT devices and the infrastructure. The communication between the IoT devices happens from one trusted entity of an area to the other by way of generating security certificates. Establishing trust by way of generating security certificates for the IoT devices in a smart city application can be of high cost and expensive. In order to facilitate this, a secure group authentication scheme that creates trust amongst a group of IoT devices owned by several entities has been proposed. The majority of proposed authentication techniques are made for individual device authentication and are also utilized for group authentication; nevertheless, a unique solution for group authentication is the Dickson polynomial based secure group authentication scheme. The secret keys used in our proposed authentication technique are generated using the Dickson polynomial, which enables the group to authenticate without generating an excessive amount of network traffic overhead. IoT devices' group authentication has made use of the Dickson polynomial. Blockchain technology is employed to enable secure, efficient, and fast data transfer among the unique IoT devices of each group deployed at different places. Also, the proposed secure group authentication scheme developed based on Dickson polynomials is resistant to replay, man-in-the-middle, tampering, side channel and signature forgeries, impersonation, and ephemeral key secret leakage attacks. In order to accomplish this, we have implemented a hardware-based physically unclonable function. Implementation has been carried using python language and deployed and tested on Blockchain using Ethereum Goerli's Testnet framework. Performance analysis has been carried out by choosing various benchmarks and found that the proposed framework outperforms its counterparts through various metrics. Different parameters are also utilized to assess the performance of the proposed blockchain framework and shows that it has better performance in terms of computation, communication, storage and latency.


Activity scenarios simulation by discovering knowledge through activities of daily living datasets

Abstract: Efficiently recognizing Activities of Daily Living (ADLs) requires overcoming challenges in collecting datasets through innovative approaches. Simultaneously, it involves adapting to the demand for interpreting human activities amidst temporal sequences of actions and interactions with objects, considering real-life scenarios and resource constraints. This study investigates the potential of generating synthetic training data for ADLs recognition using the VirtualHome2KG framework. Furthermore, we investigate the transformative potential of simulating activities in virtual spaces, as evidenced by our survey of real-world activity datasets and exploration of synthetic datasets in virtual environments. Our work explicitly simulates activities in the 3D Unity platform, affording seamless transitions between environments and camera perspectives. Furthermore, we meticulously construct scenarios not only for regular daily activities but also for abnormal activities to detect risky situations for independent living, ensuring the incorporation of critical criteria. We incorporate one contemporary method for abnormal activity detection to demonstrate the efficacy of our simulated activity data. Our findings suggest that our activity scenario preparation accomplishes the intended research objective while paving the way for an interesting research avenue.


A heterogeneous 3-D stacked PIM accelerator for GCN-based recommender systems

Abstract: Abstract Modern recommendation systems integrate graph convolution neural networks (GCN) for enhancing embedding representation. Compared with widely deployed neural network-based models, the extra message propagation layer of GCN-based recommendation is featured with extensive computations and irregular memory access. However, architecture designs for prevailing deep neural network recommendation models assume simple pooling in the embedding layer. ReRAM-based GCN accelerators are specialized for graph-related operations. However, they are designed for general graphs, while GCN-based recommendation models mainly operate on the user-item graph. In this paper, we proposed a resistive random accessed memory (ReRAM) based processing-in-memory (PIM) accelerator, ReGCNR, for GCN-based recommendation. ReGCNR is featured with three key innovations. First, we exploit the 3-dimensional (3-D) stacked heterogeneous ReRAM to fit with the large-size embedding table and user-item graph. Then, we propose a joint degree mapping schema that maximizes the efficiency of the execution pipeline. After that, ReGCNR assembles a well-coordinated pipeline and hardware scheduling design to boost overall system performance. Results show that ReGCNR outperforms GPU by 69.83 $$\times$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> and 56.67 $$\times$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> in terms of average speedup and energy saving, respectively. In addition, ReGCNR outperforms state-of-the-art ReRAM-based solutions by 11.13 $$\times$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> speedups and 7.22 $$\times$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>×</mml:mo> </mml:math> energy savings on average.


Understanding the Micro, Meso, and Macro Worlds of User Experience

Abstract: Overview: The article presents a framework for holistically understanding a product’s user experience (UX). Informed by the author’s experience as a UX researcher and manager at Google, it is intended to demystify the competencies that UX researchers can bring to organizations to help propel successful innovation. The framework outlines how users simultaneously inhabit three nested scale dimensions of experience with products (Worlds of UX) and demonstrates that engendering user-centered thinking within organizations requires integrating knowledge from each. The Micro World of UX comprises moment-by-moment experiences: direct sensory and motor phenomena, tasks, tools, and decision-making moments users encounter when using a product to accomplish lower-order goals. The Meso World of UX consists of user journeys: sequences of lower-order goals that form pathways toward higher-order goals. The Macro World of UX is the realm of user–product relationships: how users experience successive user journeys and how user segments orient toward products longitudinally.


Academician Alexander Konstantinovich Krylov as a teacher and educator

Abstract: The subject of this article is the activities of Academician Alexander Konstantinovich Krylov - Academician of the Russian Academy of Arts, Honored Artist of the Russian Federation, painter, laureate of the International Prize named after N.K. Roerich, as a teacher and educator. This is a study of the contribution of Academician Alexander Konstantinovich Krylov to the development of education and science, his teaching methods, pedagogical and educational principles. The author examines in detail the scientific, teaching, social, and creative activities of A.K. Krylova and pays special attention to the educational activities of the academician. The author also presents a complete list of institutions that, according to the State Catalog of the Museum Fund of the Russian Federation and data provided by the artist himself, store his works. The research methodology is presented by the historical method, biographical method, and comparative method. Also, during the preparation of the article, a study was conducted on the basis of archival documents, publications, and pedagogical works of A.K. Krylov and his students. The main conclusions of the study are the creative portrait of the artist presented in this message, which has independent cultural and historical significance and expresses the deeply personal attitude of the author, who hopes that the information and characteristics he collected will be useful to future biographers of the outstanding Russian artist, academician of painting A. K. Krylov. The novelty of the research lies in the generalization of the creative materials of Academician A.K. Krylov, which can be in demand by aspiring artists, art historians, cultural experts, historians and everyone who cherishes the tragic and happy milestones of the steady spiritual growth of Russia - in the past, present and future.


Intelligent 3D garment system of the human body based on deep spiking neural network

Abstract: Intelligent garments, a burgeoning class of wearable devices, have extensive applications in domains such as sports training and medical rehabilitation. Nonetheless, existing research in the smart wearables domain predominantly emphasizes sensor functionality and quantity, often skipping crucial aspects related to user experience and interaction. To address this gap, this study introduces a novel real-time 3D interactive system based on intelligent garments. The system utilizes lightweight sensor modules to collect human motion data and introduces a dual-stream fusion network based on pulsed neural units to classify and recognize human movements, thereby achieving real-time interaction between users and sensors. Additionally, the system in- corporates 3D human visualization functionality, which visualizes sensor data and recognizes human actions as 3D models in realtime, providing accurate and comprehensive visual feedback to help users better understand and analyze the details and features of human motion. This system has significant potential for applications in motion detection, medical monitoring, virtual reality, and other fields. The accurate classification of human actions con- tributes to the development of personalized training plans and injury prevention strategies. This study has substantial implications in the domains of intelligent garments, human motion monitoring, and digital twin visualization. The advancement of this system is expected to propel the progress of wearable technology and foster a deeper comprehension of human motion.


Importance-aware 3D volume visualization for medical content-based image retrieval-a preliminary study

Abstract: A medical content-based image retrieval (CBIR) system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image. CBIR is widely used in evidence- based diagnosis, teaching, and research. Although the retrieval accuracy has largely improved, there has been limited development toward visualizing important image features that indicate the similarity of retrieved images. Despite the prevalence of3D volumetric data in medical imaging such as computed tomography (CT), current CBIR systems still rely on 2D cross-sectional views for the visualization of retrieved images. Such 2D visualization requires users to browse through the image stacks to confirm the similarity of the retrieved images and often involves mental reconstruction of 3D information, including the size, shape, and spatial relations of multiple structures. This process is time-consuming and reliant on users’ experience. In this study, we proposed an importance-aware 3D volume visualization method. The rendering parameters were automatically optimized to maximize the visibility of important structures that were detected and prioritized in the retrieval process. We then integrated the proposed visualization into a CBIR system, thereby complementing the 2D cross-sectional views for relevance feedback and further analyses. Our preliminary results demonstrate that 3D visualization can provide additional information using multimodal positron emission tomography and computed tomography (PET- CT) images of a non-small cell lung cancer dataset.


An improved quantum payment protocol based on group signature without entanglement

Abstract: With the popularization of E-payment, more and more electronic payment protocols have been proposed. In 2021, Lin et al. proposed a trusted third-party E-Payment protocol based on a set of three-particle local indistinguishable orthogonal product (X-LIOP) states. In this paper, we first point out that there is a security flaw where the merchant can forge the purchase goods information to sell something that the customer does not want to buy in Lin et al. protocol. Besides, the X-LIOP states used by Lin are not orthogonal and can be discriminated by Local Operation and Classical Communication (LOCC). Second, we propose a new electronic payment protocol that combines group signature and the three-particle X-LIOP states. Compared with Lin et al. protocol, customers can purchase goods anonymously in our protocol and the unforgeability of messages and signatures is guaranteed.


Semi-supervised Federated Learning for Digital Twin 6G-enabled IIoT: A Bayesian Estimated Approach

Abstract: In recent years, the proliferation of Industrial Internet of Things (IIoT) devices has resulted in a substantial increase in data generation across various domains, including the nascent 6G networks. Digital Twins (DTs), serving as virtual replicas of physical entities, have gained popularity within the realm of IoT due to their capacity to simulate and optimize physical systems in a cost-effective manner. Nonetheless, the security of DTs and the safeguarding of the sensitive data they generate have emerged as paramount concerns. Fortunately, the Federated Fearning (FL) system has emerged as a promising solution to address the challenge of data privacy within DTs. Nonetheless, the requisite acquisition of a significant volume of labeled data for training purposes poses a formidable challenge, particularly in a DT environment that blends real and virtual data.To tackle this challenge, this study presents an innovative Semi-supervised FL (SSFL) framework designed to overcome the scarcity of labeled data through the strategic utilization of pseudo-labels.Specifically, our proposed SSFL algorithm, named SSFL-MBE, introduces a novel approach by combining Mix data augmentation and Bayesian Estimation consistency regularization loss, thereby integrating robust augmentation techniques to enhance model generalization. Furthermore, we introduce a Bayesian-estimated pseudo-label loss that leverages prior probabilistic knowledge to enhance model performance. Our investigation focuses particularly on a demanding scenario where labeled and unlabeled data are segregated across disparate locations, specifically, the server and various clients.Comprehensive evaluations conducted on CIFAR-10 and MNIST datasets conclusively demonstrate that our proposed algorithm consistently surpasses mainstream SSFL baseline models, exhibiting an enhancement in model performance ranging from 0.5% to 1.5%.Overall, this work contributes to the development of more efficient and secure approaches for model training in DT-empowered FL settings, which is crucial for the deployment of IIoTs in 6G-enabled environments.


Expanding the cloud-to-edge continuum to the IoT in serverless federated learning

Abstract: Serverless computing enables greater flexibility and efficiency in the cloud-to-edge continuum. Artificial Intelligence and Machine Learning (AI/ML) applications benefit greatly from this paradigm, as they need to gather, preprocess, aggregate and analyze data at various scales. In such contexts, the increasing hardware/software resource availability of Internet of Things (IoT) devices provides the opportunity to exploit them not only as data sources in AI/ML infrastructures, but also as computational nodes for model training and inference; nevertheless, comprehensive frameworks are still mostly missing. This work introduces an innovative serverless computing architecture which expands the cloud-to-edge continuum toward IoT devices. The same functions can run on IoT, edge and cloud nodes with minimal to no code modification and they can be invoked through a uniform interface. A federated learning framework is defined based on the proposed architecture, exploiting an existing IoT-oriented ML algorithm in a novel way. Notably, IoT nodes are used for both federated training and local inference tasks. A full prototype implementation has been built with off-the-shelf technologies and devices. A case study on federated machine learning for activity recognition and experiments have been conducted to validate key elements of the proposal.


Multi-behavior recommendation with SVD graph neural networks

Abstract: Graph Neural Networks (GNNs) have been extensively employed in the field of recommendation systems, offering users personalized recommendations and yielding remarkable outcomes. Recently, GNNs incorporating contrastive learning have demonstrated promising performance in handling the sparse data problem of recommendation systems. However, existing contrastive learning methods still have limitations in resisting noise interference, especially for multi-behavior recommendation. To mitigate the aforementioned issues, this paper proposes a GNN-based multi-behavior recommendation model called MB-SVD that utilizes Singular Value Decomposition (SVD) graphs to enhance model performance. In particular, MB-SVD considers user preferences across different behaviors, improving recommendation effectiveness. First, MB-SVD integrates the representation of users and items under different behaviors with learnable weight scores, which efficiently considers the influence of different behaviors. Then, MB-SVD generates augmented graph representation with global collaborative relations. Next, we simplify the contrastive learning framework by directly contrasting original representation with the enhanced representation using the InfoNCE loss. Through extensive experimentation, the remarkable performance of our proposed MB-SVD approach in multi-behavior recommendation endeavors across diverse real-world datasets is exhibited.


Two-step ensemble under-sampling algorithm for massive imbalanced data classification

Abstract: Imbalanced data classification is a challenging problem in the field of machine learning. Class imbalance, class overlap, and large data volume significantly affect classification performance. Focusing on the impact of class overlap on classification effectiveness, we propose a two-step ensemble under-sampling algorithm based on boundary information mining (TSSE-BIM) with the goal of reducing the information loss from under-sampling methods on large-scale imbalanced data. In the first stage, the proposed method applies an improved equalization under-sampling strategy to mine sample contribution information and quickly obtains the distribution information of data relative to the decision boundary. In the second stage, based on the boundary information, a weighted boundary sampling is performed to remove noisy and highly overlapping samples. It is easy to retain samples with high contribution and effectively suppress the information loss caused by under-sampling. Then, the overall framework is designed based on a serial ensemble similar to boosting, where the weights of each base classifier are assigned to achieve a more powerful performance based on the false positive rate and false negative rate on the original data. Finally, extensive experiments indicate that TSSE-BIM outperforms state-of-the-art methods and ranks first on average under four metrics, especially F1 and MMC.


Non-iterative reversible information hiding in the secret sharing domain

Abstract: Long running time due to multiple iterations is the main drawback of existing information hiding in the sharing domain (IHSD) algorithm. To address this problem, we propose non-iterative reversible information hiding in the secret sharing domain (NIIHSD). We calculate the coefficients of the polynomial that make the least significant bit of the shadow pixel equal to ‘0’ or ‘1’ and store them in two forms (A0 and A1). According to the information to be embedded, coefficients are randomly selected in A0 and A1 to generate qualified shares by once calculation. The comparison with IHSD demonstrates our scheme improves the running speed of the algorithm by a factor of 2 to 4 for different (k,n) thresholds and secret image with different sizes. The coefficient forms only need to be created once to be reused. Our scheme has a greater advantage when sharing a large number of images.


High-speed encrypted traffic classification by using payload features

Abstract: Traffic encryption techniques facilitate cyberattackers to hide their presence and activities. Traffic classification is an important method to prevent network threats. However, due to the tremendous traffic volume and limitations of computing, most existing traffic classification techniques are inapplicable to the high-speed network environment. In this paper, we propose a High-speed Encrypted Traffic Classification (HETC) method containing two stages. First, to efficiently detect whether traffic is encrypted, HETC focuses on randomly sampled short flows and extracts aggregation entropies with chi-square test features to measure the different patterns of the byte composition and distribution between encrypted and unencrypted flows. Second, HETC introduces binary features upon the previous features and performs fine-grained traffic classification by combining these payload features with a Random Forest model. The experimental results show that HETC can achieve a 94% F-measure in detecting encrypted flows and a 85%–93% F-measure in classifying fine-grained flows for a 1-KB flow-length dataset, outperforming the state-of-the-art comparison methods. Meanwhile, HETC does not need to wait for the end of the flow and can extract mass computing features. The average time for HETC to process each flow is only 2 or 16 ms, which is lower than the flow duration in most cases, making it a good candidate for high-speed traffic classification.


Deepfake face discrimination based on self-attention mechanism

Abstract: With the rapid progress of deepfake technology, the improper use of manipulated images and videos presenting synthetic faces has arisen as a noteworthy concern, thereby posing threats to both daily life and national security. While numerous CNN based deepfake face detection methods were proposed, most of the existing approaches encounter challenges in effectively capturing the image contents across different scales and positions. In this paper, we present a novel two-branch structural network, referred to as the Self-Attention Deepfake Face Discrimination Network (SADFFD). Specifically, a branch incorporating cascaded multi self-attention mechanism (SAM) modules, is parallelly integrated with EfficientNet-B4 (EffB4). The multi SAM branch supplies additional features that concentrate on image regions essential for discriminating between real and fake. The EffB4 network is adopted because of its efficiency by adjusting the resolution, depth, and width of the network. According to our comprehensive experiments conducted on FaceForensics++, Celeb-DF, and our self-constructed SAMGAN3 datasets, the proposed SADFFD demonstrated the highest detection accuracy, averaging 99.01% in FaceForensics++, 98.65% in Celeb-DF, and an impressive 99.99% in SAMGAN3, surpassing other state-of-the-art (SOTA) methods.


Continual learning for adaptive social network identification

Abstract: The popularity of social networks as primary mediums for sharing visual content has made it crucial for forensic experts to identify the original platform of multimedia content. Various methods address this challenge, but the constant emergence of new platforms and updates to existing ones often render forensic tools ineffective shortly after release. This necessitates the regular updating of methods and models, which can be particularly cumbersome for techniques based on neural networks which cannot quickly adapt to new classes without sacrificing performance on previously learned ones – a phenomenon known as catastrophic forgetting. Recently, researchers aimed at mitigating this problem via a family of techniques known as continual learning. In this paper we study the applicability of continual learning techniques to the social network identification task by evaluating two relevant forensic scenarios: Incremental Social Platform Classification, for handling newly introduced social media platforms, and Incremental Social Version Classification, for addressing updated versions of a set of existing social networks. We perform an extensive experimental evaluation of a variety of continual learning approaches applied to these two scenarios. Experimental results demonstrate that, although Continual Social Network Identification remains a difficult problem, catastrophic forgetting can be significantly mitigated in both scenarios by retaining only a fraction of the image patches from past task training samples or by employing previous tasks prototypes.


Space Limited Linear-Time Graph Algorithms on Big Data

Abstract: We study algorithms for graph problems in which the graphs are of extremely large size N so that super-linear time ω(N) or linear space Θ(N) would become impractical. We use a parameter k to characterize the computational power of a normal computer that can provide additional time and space bounded by polynomials of k in dealing with the large graphs. In particular, we are interested in strict linear-time algorithms using space O(kO(1)). In our case studies, as examples, we present (1) a randomized greedy algorithm of time O(N) and space O(k2) for a parameterized version of the Maximal Matching problem; and (2) randomized kernelization algorithms of time O(N) and space O(kO(1)) for a number of well-known NP-hard problems. Our kernelization algorithms have their kernel sizes match the best kernel sizes by known polynomial-time kernelization algorithms with no space constraints for the problems. We also study the relationship between our proposed model and the streaming model. This study motivates a new streaming kernelization algorithm for the famous Vertex Cover problem that has an optimal update time complexity while matches the best known space complexity of streaming algorithms for the problem.


Cultivating connectedness: Effects of an app-based compassion meditation course on changes in global identity, nature connectedness, and pro-environmental behaviour

Abstract: Individual behaviour change and political measures are required to solve pressing global environmental crises such as climate change. Identification with an ingroup that includes all humanity (i.e., a global identity) was positively related to pro-environmental engagement in various studies, but little is known about how to foster a global identity. We argue that meditation with a focus on cultivating compassion might increase global identity because connectedness with people all over the world is a traditional goal of such practice. In a randomised controlled intervention study with pre-post design (N = 164, Germany), participants received an app-based 7-day meditation course. Controlling for pre-survey levels, compared to the waiting control group, the meditation group reported stronger changes in global identity and pro-environmental behaviour but not climate policy support. We found that participating in the meditation course indirectly predicted changes in pro-environmental behaviour through a strengthened global self-investment, as one dimension of global identity. Moreover, we also found stronger changes in nature connectedness in the meditation group compared to the control group. Hence, a 7-day app-based compassion meditation course indeed seems to cultivate connectedness – with both humans and the broader natural world. Future research should compare different course curricula. Moreover, it should aim at diverse samples, longitudinal assessments, and investigate reasons for self-selecting in and dropping out of meditation-related studies.


Neighborhood evaluator for efficient super-resolution reconstruction of 2D medical images

Abstract: Deep learning-based super-resolution (SR) algorithms aim to reconstruct low-resolution (LR) images into high-fidelity high-resolution (HR) images by learning the low- and high-frequency information. Experts’ diagnostic requirements are fulfilled in medical application scenarios through the high-quality reconstruction of LR digital medical images. Medical image SR algorithms should satisfy the requirements of arbitrary resolution and high efficiency in applications. However, there is currently no relevant study available. Several SR research on natural images have accomplished the reconstruction of resolutions without limitations. However, these methodologies provide challenges in meeting medical applications due to the large scale of the model, which significantly limits efficiency. Hence, we suggest a highly effective method for reconstructing medical images at any desired resolution. Statistical features of medical images exhibit greater continuity in the region of neighboring pixels than natural images. Hence, the process of reconstructing medical images is comparatively less challenging. Utilizing this property, we develop a neighborhood evaluator to represent the continuity of the neighborhood while controlling the network’s depth. The suggested method has superior performance across seven scales of reconstruction, as evidenced by experiments conducted on panoramic radiographs and two external public datasets. Furthermore, the proposed network significantly decreases the parameter count by over 20× and the computational workload by over 10× compared to prior researches. On large-scale reconstruction, the inference speed can be enhanced by over 5×. The novel proposed SR strategy for medical images performs efficient reconstruction at arbitrary resolution, marking a significant breakthrough in the field. The given scheme facilitates the implementation of SR in mobile medical platforms.


Global Feature-Based Multimodal Semantic Segmentation

Abstract: Incorporating complementary modality into RGB branch can significantly improve the effectiveness of semantic segmentation. However, fusion between the two modalities faces huge challenge due to the difference of their optical dimensions. Existed fusion methods can't keep a balance between performance and efficiency in aggregating detailed features. To address this problem, we propose a global feature-based network (GFBN) for semantic segmentation that establishes mapping function and extraction relationship among the multi-modalities. The GFBN contains three important modules, which are used for feature correction, fusion and edge enhancement. Firstly, the cross-attention rectification module (CARM) adaptively extracts mapping relationships and rectifies the RGB and complementary features. Secondly, the cross-field fusion module (CFM) integrates long-range rectified features of two branches to obtain an optimal fusion feature. Finally, the boundary guidance module (BGM) sharpens the boundary information of the fused features to effectively improve the segmentation accuracy of object boundaries. We make the experiments of GFBN on the challenging MCubeS and ZJU-RGB-Ps datasets. The results show that GFBN outperforms state-of-the-art methods by at least 0.64% and 0.7% on mean intersection over union (mIoU), respectively. It demonstrates the performance and efficiency of our proposed method.


Background no more: Action recognition across domains by causal interventions

Abstract: We aim to recognize actions under an appearance distribution shift between a source training domain and a target test domain. To enable such video domain generalization, our key idea is to intervene on the action to remove the confounding effect of the domain background on the class label using causal inference. Towards this, we propose to learn a causally debiased model on a source domain that intervenes on the action through three possible Do-operators, which separate the action and background. To better align the source and target distributions, we also introduce a test-time action intervention. Experiments on two challenging video domain generalization benchmarks reveal that causal inference is a promising tool for action recognition as it already achieves state-of-the-art results on Kinetics2Mimetics, the benchmark with the largest domain shift.


TIE- text information extraction from natural scene images using SVM

Abstract: Detection of text and its localisation from natural scenic imagery plays a vital role in in Content Based Imagery Analysis. The major and unavoidable challenges are illumination effects, different orientations in the lines, typical backgrounds, different font styles and sizes. This paper presents a novel methodology Text Information Extraction (TIE) using Support Vector Machine (SVM) which robustly detects and localises the text from natural scene images. Major focus of this paper lies in detection of text object from the whole scene. Initially the image will be pre-processed for noise removal and contrast enhancement. Later, all the objects of scene will be marked and extracted in order to form an object pool. SVM technique is used to locate text object among the object pool. The SVM will be trained with supervised parameter learning. At last the well trained model will perform binary classification to differentiate between text and non-text object from the object pool. Experiments have been performed on ICDAR dataset and achieved results determine that proposed approach leads to utmost Precision and Recall performance when compared to existing state-of-the-art techniques.


Audio-visual saliency prediction for movie viewing in immersive environments: Dataset and benchmarks

Abstract: In this paper, an eye-tracking dataset of movie viewing in the immersive environment is developed, which contains 256 movie clips with 2K QHD resolution and corresponding movie genre labels from IMDb (Internet Movie Database). The dataset provides the audio-visual clues for studying the human visual attention when watching movie using a VR headset, by recording the eye movements using integrated eye tracker. To provide benchmarks for a saliency prediction for movie viewing in the immersive environment, fifteen computational models are evaluated on the dataset, including a newly developed multi-stream audio-visual saliency prediction model based on deep neural networks, named as MSAV. Detailed quantitative and qualitative comparisons and analyses are also provided. The developed dataset and benchmarks could help to facilitate the studies of visual saliency prediction for movie viewing in the immersive environments.


Stochastic reconstruction of digital cores using two-discriminator VAE-GAN

Abstract: The internal structure of rocks has a great influence on the refined reservoir exploitation. The digital core has been extensively studied to reflect the underlying characteristics of rock structures. Physical experimental methods and numerical reconstruction methods are two mainstreams in the reconstruction of digital cores. The former requires expensive equipment and is probably very time-consuming because of the complex preparation process. In contrast, numerical reconstruction methods are less expensive but have problems such as slow processes and potentially poor results. Nowadays, deep learning based on extracting features from training images has opened a new door for reconstructing digital cores. Variational autoencoder (VAE) and generative adversarial network (GAN) are two well-known deep learning models. The training process of VAE is stable but the generated image is easily blurred while GAN can generate clear images but have an unstable process. Although VAE-GAN combines VAE and GAN to hopefully completely address the above issue, it cannot solve the error problem from variational inference originated from VAE. Therefore, we add a code discriminator in VAE-GAN to solve this problem, replacing variational inference by adversarial learning. By comparing with a traditional method and classical deep generative models, the effectiveness and practicality of the new model in reconstructing digital cores is verified.


Trustworthy Artificial Intelligence -based federated architecture for symptomatic disease detection

Abstract: The recent viral outbreaks have had a significant impact on interpersonal relationships, particularly in enclosed spaces. Detecting and preventing the transmission of diseases such as COVID-19 has become a top priority. These diseases are typically identifiable through the symptoms they cause in humans. However, the collection of personal and health data for use in Artificial Intelligence models can give rise to ethical, security, and privacy issues. Therefore, it is necessary to have architectures that maintain the principles of Trustworthy Artificial Intelligence by design. This work proposes a decentralized architecture based on Federated Learning for symptomatic disease detection using the edge computing paradigm, storing the information in the device that collected it, and the foundations of Trustworthy Artificial Intelligence. The architecture is designed to be robust, secure, transparent, and responsible while maintaining data privacy. The proposed approach can be used with medical information capture systems with different user profiles.


Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes

Abstract: The broad adoption of electronic health record (EHR) systems brings us a tremendous amount of clinical data and thus provides opportunities to conduct data-based healthcare research to solve various clinical problems in the medical domain. Machine learning and deep learning methods are widely used in the medical informatics and healthcare domain due to their power to mine insights from raw data. When adapting deep learning models for EHR data, it is essential to consider its heterogeneous nature: EHR contains patient records from various sources including medical tests (e.g. blood test, microbiology test), medical imaging, diagnosis, medications, procedures, clinical notes, etc. Those modalities together provide a holistic view of patient health status and complement each other. Therefore, combining data from multiple modalities that are intrinsically different is challenging but intuitively promising in deep learning for EHR. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in EHR for enhanced performance in clinical risk prediction. Early, joint, and late fusion strategies are employed to combine data from various modalities effectively. We test the model with three predictive tasks: in-hospital mortality, long length of stay, and 30-day readmission. Experimental results show that multimodal models outperform uni-modal models in the tasks involved. Additionally, by training models with different input modality combinations, we calculate the Shapley value for each modality to quantify their contribution to multimodal performance. It is shown that temporal variables tend to be more helpful than CXR images and clinical notes in the three explored predictive tasks.


Dataset of Sentiment Tagged Language Resources for Bosnian Language

Abstract: The Bosnian language holds significant importance as a member of the West-South Slavic subgroup within the Slavic branch of the Indo-European linguistic family. With approximately 2.5 million speakers in Europe, including 1.87 million individuals in Bosnia and Herzegovina alone, the Bosnian language constitutes the mother tongue for a considerable portion of the population. In Natural Language Processing (NLP) tasks related to the Bosnian language, besides removing stop words, it is important to consider the influence of other linguistic elements. Bosnian text contains words derived from diminishers, relative intensifiers, minimizers, maximizers, boosters, and approximators. These words contribute to the overall meaning and sentiment analysis of the text. By including these elements in NLP models and algorithms, researchers can achieve more accurate and nuanced analysis of Bosnian language data, enhancing the effectiveness of NLP applications. The two lists of sentiment annotated words that present the core of the Bosnian sentiment-annotated lexicon, a list of the stopwords, and a list of Affirmative and non-Affrimative words (AnAwords) composed mostly of intensifiers and diminishers, were used to construct a dataset that presents the base for sentiment analysis in the Bosnian language.


LPMSAEF: Lightweight Process Mining-Based Software Architecture Evaluation Framework for Security and Performance Analysis

Abstract: The article discusses the need for a lightweight software architecture evaluation framework that can address practitioners' concerns. Specifically, the proposed framework uses process mining and Petri nets to analyze security and performance in software development's early and late stages. Moreover, the framework has been implemented in six case studies, and the results show that it is a feasible and effective solution that can detect security and performance issues in complex and heterogeneous architecture with less time and effort. Furthermore, the article provides a detailed explanation of the framework's features, factors, and evaluation criteria. Additionally, this article discusses the challenges associated with traditional software architecture documentation methods using Unified Modeling Language diagrams and the limitations of code alone for creating comprehensive Software Architecture models. Various methods have been developed to extract implicit Software Architecture from code artifacts, but they tend to produce code-oriented diagrams instead of Software Architecture diagrams. Therefore, to bridge the model-code gap, the article proposes a framework that considers existing Software Architecture in the source code as architectural components and focuses on Software Architecture behaviors for analyzing performance and security. The proposed framework also suggests comparing Software Architecture extracted by different Process Mining algorithms to achieve consensus on architecture descriptions, using visualizations to understand differences and similarities. Finally, the article suggests that analyzing the previous version of a system's Software Architecture can lead to improvements and deviations from planned Software Architecture can be detected using traceability approaches to aid software architects in detecting inconsistencies.


Deep learning CNN-APSO-LSSVM hybrid fusion model for feature optimization and gas-bearing prediction

Abstract: Conventional machine learning (CML) methods have been successfully applied for gas reservoir prediction. Their prediction accuracy largely depends on the quality of the sample data; therefore, feature optimization of the input samples is particularly important. Commonly used feature optimization methods increase the interpretability of gas reservoirs; however, their steps are cumbersome, and the selected features cannot sufficiently guide CML models to mine the intrinsic features of sample data efficiently. In contrast to CML methods, deep learning (DL) methods can directly extract the important features of targets from raw data. Therefore, this study proposes a feature optimization and gas-bearing prediction method based on a hybrid fusion model that combines a convolutional neural network (CNN) and an adaptive particle swarm optimization-least squares support vector machine (APSO-LSSVM). This model adopts an end-to-end algorithm structure to directly extract features from sensitive multicomponent seismic attributes, considerably simplifying the feature optimization. A CNN was used for feature optimization to highlight sensitive gas reservoir information. APSO-LSSVM was used to fully learn the relationship between the features extracted by the CNN to obtain the prediction results. The constructed hybrid fusion model improves gas-bearing prediction accuracy through two processes of feature optimization and intelligent prediction, giving full play to the advantages of DL and CML methods. The prediction results obtained are better than those of a single CNN model or APSO-LSSVM model. In the feature optimization process of multicomponent seismic attribute data, CNN has demonstrated better gas reservoir feature extraction capabilities than commonly used attribute optimization methods. In the prediction process, the APSO-LSSVM model can learn the gas reservoir characteristics better than the LSSVM model and has a higher prediction accuracy. The constructed CNN-APSO-LSSVM model had lower errors and a better fit on the test dataset than the other individual models. This method proves the effectiveness of DL technology for the feature extraction of gas reservoirs and provides a feasible way to combine DL and CML technologies to predict gas reservoirs.


Evaluating the regulation of social media: An empirical study of the German NetzDG and Facebook

Abstract: This study compiles an original data set of Facebook posts and comments to analyze potential overblocking and chilling effects of a German law that aims to reduce hate speech on social media platforms (Netzwerkdurchsetzungsgesetz, NetzDG). The analysis is based on a sample of 10 popular public Facebook pages. While critics of the law have feared an undue increase in the deletion of user-generated content (“overblocking”), we find no robust evidence in support of this claim in the data during our investigation period. Although there has been a slight increase in the proportion of deleted comments per post, the effect size is small. Furthermore, neither do people abstain from commenting on Facebook nor does the tonality of comments become more positive after the law was in full effect (“chilling effects”). In addition, this study demonstrates the difficulties of the empirical evaluation of platform regulation. This raises the question how politicians can hold social media platforms accountable to protect users and their rights in the digital sphere.


Enhanced Route navigation control system for turtlebot using human-assisted mobility and 3-D SLAM optimization

Abstract: An autonomous, power-assisted Turtlebot is presented in this paper in order to enhance human mobility. The turtlebot moves from its initial position to its final position at a predetermined speed and acceleration. We propose an intelligent navigation system that relies solely on individual instructions. When there is no individual present, the Turtlebot remains stationary. Turtlebot utilizes a rotating Kinect sensor in order to perceive its path. Various angles were examined in order to demonstrate the effectiveness of the system in experiments conducted on a U-shaped experimental pathway. The Turtlebot was used as an experimental device during these trials. Based on the U-shaped path, deviations from different angles were measured to evaluate its performance. SLAM (Simultaneous Localization and Mapping) experiments were also explored. We divided the SLAM problem into components and implemented the Kalman filter on the experimental path to address it. The Kalman filter focused on localization and mapping challenges, utilizing mathematical processes considering both the system's knowledge and the measurement tool. This approach allowed us to achieve the most accurate system state estimation possible. The significance of this work extends beyond the immediate application, as it lays the groundwork for advancements in wheelchair navigation research by Dynamic Control. The experiments conducted on a U-shaped pathway not only validate the efficacy of our algorithm but also provide valuable insights into the intricacies of navigating in both forward and reverse directions. These insights are pivotal for refining the navigation algorithm, ultimately contributing to the development of more robust and user-friendly systems for individuals with mobility challenges. The data used for this purpose included actuator input, vehicle location, robot movement sensors, and sensor readings representing the world state. The study provides a strong foundation for future wheelchair navigation research by Dynamic Control. Consequently, we found that navigating the Turtlebot in the reverse direction resulted in a 5%-6% increase in diversion compared to forward navigation, providing valuable insight into further improvement of the navigation algorithm.


SPACE: Senti-Prompt As Classifying Embedding for sentiment analysis

Abstract: In natural language processing, the general approach to sentiment analysis involves a pre-training and fine-tuning paradigm using pre-trained language models combined with classifier models. Recently, numerous studies have applied prompts not only to downstream generation but also to classification tasks as well. However, to fully utilize the advantages of prompts and incorporate the context-dependent meaning of class representations into the prompts, it is necessary for the prompts to be learned similarly to the sentiment representations of each class. To achieve this, We introduce a novel method to learn soft prompts during fine-tuning. In this method, the class prompts are initialized with sentiment-related embeddings and are trained by a denoising task, which replaces them with masked tokens, just like the conventional masked language model (MLM) approach. Furthermore, a novel attention pattern is designed to tune attention between class prompts effectively. As a result, we demonstrate that our approach outperforms state-of-the-art models through experiments on four common datasets, achieving superior performance on sentiment analysis.


Extracting key topics from massive COVID-19 information on social networks: An integrated deep learning and LDA framework

Abstract: Quickly discovering and processing unmeaningful information is especially important in the current social network, where public opinion is rapidly fermenting. In this study, a novel topic extraction framework is proposed to extract key topics for the special event of COVID-19 on social networks. First, huge and noisy COVID-19 information is pre-processed for our framework. Second, a model of key information identification is designed for COVID-19 information, which combines the deep learning model of Text Convolutional Neural Networks (TextCNN) with Bi-directional Long Short-Term Memory (BiLSTM) to perform a multi-class classification task. Finally, a model of key topic extraction is developed with the Latent Dirichlet Allocation (LDA) model, which is able to summarize topics and keywords from a large amount of information. we crawled the public opinion information of the COVID-19 on weibo as our dataset. experimental results demonstrate that our model outperforms the other benchmark models in identifying important information related to COVID-19. the key topic extraction model based on LDA can further divide key information into multiple topics ►And describe the topics through keywords for fast tracking of COVID-19 pandemic on social networks.


FL-OTCSEnc: Towards secure federated learning with deep compressed sensing

Abstract: In recent years, federated learning has made significant progress in preserving data privacy. In this paradigm, clients train local models without sharing their raw data, thereby substantially mitigating the vulnerability to private data exposure. However, it is still possible to infer clients’ raw data by leveraging the gradient parameters exchanged between the clients and the server. To address this problem, this paper proposes a novel algorithm that introduces deep compressed sensing into federated learning to support one time encryption, called FL-OTCSEnc, to secure the communication data exchanged between the clients and the server. The process starts by creating a dataset of deep learning model parameters and training a system for both encryption and decryption using deep compressed sensing. This system is then used to secure the communication between clients and the server in federated learning, by encrypting and decrypting the data exchanged. To enhance the security of the proposed algorithm, we introduce an assessment method for evaluating the security level of the clients, facilitating the selection of suitable candidates for deployment within distributed training encryption and decryption models that are updated in real time. To enhance the accuracy of the decrypted deep network model, we introduce a tandem loss function in the training process. Moreover, this paper proves that the proposed end-to-end encryption method satisfies additive homomorphic encryption properties. Extensive experiments demonstrate that the deep compressed sensing encryption in federated learning achieves promising results without increasing the computational complexity.


Learning disentangled representations in signed directed graphs without social assumptions

Abstract: Signed graphs can represent complex systems of positive and negative relationships such as trust or preference in various domains. Learning node representations is indispensable because they serve as pivotal features for downstream tasks on signed graphs. However, most existing methods often oversimplify the modeling of signed relationships by relying on social theories, while real-world relationships can be influenced by multiple latent factors. This hinders those methods from effectively capturing the diverse factors, thereby limiting the expressiveness of node representations. In this paper, we propose DINES, a novel method for learning disentangled node representations in signed directed graphs without social assumptions. We adopt a disentangled framework that separates each embedding into distinct factors, allowing for capturing multiple latent factors, and uses signed directed graph convolutions that focus solely on sign and direction, without depending on social theories. Additionally, we propose a new decoder that effectively classifies an edge's sign by considering correlations between the factors. To further enhance disentanglement, we jointly train a self-supervised factor discriminator with our encoder and decoder. Throughout extensive experiments on real-world signed directed graphs, we show that DINES effectively learns disentangled node representations, and significantly outperforms its competitors in predicting link signs.


Towards trustworthy remaining useful life prediction through multi-source information fusion and a novel LSTM-DAU model

Abstract: Remaining useful life (RUL) prediction is a key part of the prognostic and health management of machines, which can effectively prevent catastrophic faults and decrease expensive unplanned maintenance. A good health indicator (HI) can ensure the accuracy and reliability of RUL prediction. However, most of the existing HI construction methods use only a single signal and rely heavily on prior knowledge, making it difficult to capture critical information about mechanical degradation, which in turn affects the performance of RUL prediction. To solve the above problems, a novel adaptive multi-source fusion method based on genetic programming is proposed for building a HI that can effectively reflect the health state of machines. Subsequently, a new LSTM model with a dual-attention mechanism is developed, which differentially handles the network input information and the recurrent information to improve the prediction performance and reduce the time complexity at the same time. Experimental validation is carried out on the real PRONOSTIA bearing dataset. The comparative results validate that the constructed fusion HI has better comprehensive performance than other fusion HIs, and the proposed prediction method is competitive with the current state-of-the-art methods.


Blind image quality assessment with semi-supervised learning

Abstract: Blind image quality assessment (BIQA) aims to automatically predict the perceptual quality of an image without requiring access to its pristine reference counterpart. BIQA models are typically developed through supervised learning, optimizing and testing them by comparing their predictions to human ratings, usually expressed as mean opinion scores (MOS), which can be labor-intensive to collect. The performance of these BIQA models is significantly reliant on the amount of labeled training data. When there is a shortage of human-rated data, these BIQA models may perform inadequately. In this study, we investigate the potential of incorporating unlabeled data to mitigate this issue and enhance the performance of BIQA models. To achieve this, we propose a deep ensemble-based BIQA model (referred to as the “target model”) with two heads: one for quality estimation and the other for pseudo-label generation. Initially, we train it on a small set of human-rated images where the supervisory signals are binary labels indicating the pairwise ranking of perceptual quality for image pairs. Then, the head responsible for pseudo-label generation assigns pseudo-binary labels to unlabeled pairs. Subsequently, we re-train the target model using a combination of labeled and pseudo-labeled datasets. This process can be iterated, allowing for the progressive improvement of the target model’s performance. We conduct comprehensive case studies to illustrate the advantages of utilizing unlabeled data for BIQA, particularly in terms of model generalization and identifying cases of model failure.


Voice-based Age, Gender, and Language Recognition Based on ResNet Deep model and Transfer learning in Spectro-Temporal Domain

Abstract: In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals.


Sliding mode control for discrete interval type-2 fuzzy semi-Markov jump models with delay in controller mode switching

Abstract: The problem of asynchronous sliding mode control (SMC) for discrete nonlinear semi-Markov jump models is investigated by the semi-Markov kernel strategy, where asynchronous mechanism means that there is a one-step delay in the controller mode switching related to the system mode. Unlike traditional T-S fuzzy model, the interval type-2 fuzzy model is applied to represent discrete semi-Markov jump systems with uncertain parameters. The main novelty is to design a suitable asynchronous SMC mechanism for the attainment of the quasi-sliding mode, overcoming the difficulty caused by parameter uncertainties. According to the fuzzy Lyapunov function related to the elapsed time and the current mode, the stability criterion of nonlinear model is developed by additional matrix variables, realizing the σ-error mean-square stability of the underlying augmented system. An appropriate fuzzy asynchronous SMC mechanism is established to achieve the accessibility of the quasi-sliding mode. Finally, a tunnel diode circuit model validates the proposed control algorithm.


Learning-Augmented Maximum Flow

Abstract: We propose a framework for speeding up maximum flow computation by using predictions. A prediction is a flow, i.e., an assignment of non-negative flow values to edges, which satisfies the flow conservation property, but does not necessarily respect the edge capacities of the actual instance (since these were unknown at the time of learning). We present an algorithm that, given an m-edge flow network and a predicted flow, computes a maximum flow in O(mη) time, where η is the ℓ1 error of the prediction, i.e., the sum over the edges of the absolute difference between the predicted and optimal flow values. Moreover, we prove that, given an oracle access to a distribution over flow networks, it is possible to efficiently PAC-learn a prediction minimizing the expected ℓ1 error over that distribution. Our results fit into the recent line of research on learning-augmented algorithms, which aims to improve over worst-case bounds of classical algorithms by using predictions, e.g., machine-learned from previous similar instances. So far, the main focus in this area was on improving competitive ratios for online problems. Following Dinitz et al. (NeurIPS 2021), our results are among the firsts to improve the running time of an offline problem.


Knowledge graph-based image classification

Abstract: This paper introduces a deep learning method for image classification that leverages knowledge formalised as a graph created from information represented by pairs attribute/value. The proposed method investigates a loss function that adaptively combines the classical cross-entropy commonly used in deep learning with a novel penalty function. The novel loss function is derived from the representation of nodes after embedding the knowledge graph and incorporates the proximity between class and image nodes. Its formulation enables the model to focus on identifying the boundary between the most challenging classes to distinguish. Experimental results on several image databases demonstrate improved performance compared to state-of-the-art methods, including classical deep learning algorithms and recent algorithms that incorporate knowledge represented by a graph.


Textural-compositional evolution of pyrite and metal remobilization during low-grade metamorphism of metapelite: Contribution to gold mineralization in the Luk Ulo Complex, Central Java, Indonesia

Abstract: The Cretaceous Luk Ulo Complex (LUC) in Central Java, Indonesia, hosts orogenic gold-type ores with S derived from metapelite. The host rock consists of accreted metapelite, metamorphosed under temperature up to 510 ± 30 °C. We investigated the textural and chemical evolution of pyrite and carbonaceous matter (CM), and analyzed the hosts phases for trace elements (TEs) such as As, Pb, Cu, Sb, Te, Ag, Zn, Bi, Se, Hg, and Au, with the aim of documenting the behavior of these elements during prograde metamorphism and evaluate the implications on gold mineralization. With increasing metamorphic grade, the texture and chemical composition of pyrite evolved through different stages: 1) framboidal pyrite and pyrite nodules in the sedimentary–diagenetic facies, 2) anhedral–subhedral pyrite in sub-greenschist facies (SGSF), 3) euhedral pyrite in lower greenschist facies (LGSF), 4) elongated pyrite in middle greenschist facies (MGSF), and 5) replacement of pyrite by pyrrhotite in upper greenschist facies (UGSF). The framboidal pyrite and pyrite nodules contain measurable TEs. Recrystallization of such pyrite into euhedral pyrite in LGSF resulted in significant depletion of TEs, except for Se, by pressure-solution. With increasing metamorphic grade, the recrystallization of organic matter to “cooked” CM in LGSF was accompanied by significant depletion in most TEs, with only Fe and S being retained. The transformation of cooked CM into graphite in MGSF, and of elongated pyrite into pyrrhotite in UGSF resulted in further removal of remaining TEs. These observations and the absence of para-amphibolite in the LUC suggest that remobilization of TEs and S from sedimentary pyrite and CM might have occurred in the LGSF, much earlier than previously proposed (UGSF-amphibolite transition).


A preliminary study on few-shot knowledge reasoning mechanism based on three-way partial order structure

Abstract: Few-shot knowledge reasoning, as a novel theory that attempts to combine few-shot learning with knowledge reasoning, is a challenging subject. This article intends to explore few-shot knowledge reasoning mechanism based on three-way partial order structure. First, from the perspective of granular computing, three knowledge reasoning operators are defined: granule splitting, granule combination, and granule isomorphism; then, based on the knowledge reasoning operators, the data augmentation operation and knowledge reasoning algorithm are presented; finally, some specific experiments on five datasets are conducted to further illustrate the few-shot knowledge reasoning algorithm. Experiment results show that the method proposed in this article can effectively realize knowledge reasoning based on small sample size data. It can be concluded that the proposed few-shot knowledge reasoning method is effective at some extent, and is expected to bring inspiration to the development of other concept-based knowledge reasoning models.


Revocable certificateless Provable Data Possession with identity privacy in cloud storage

Abstract: Provable Data Possession (PDP) has gained widespread adoption for ensuring the integrity of data in remote cloud storage, where a data owner can delegate a third party auditor (TPA) to perform data auditing. To eliminate key escrow problem or complicated certificate management in classic solutions, numerous certificateless PDP schemes have been proposed while they failed to achieve efficient user revocation and protect user identity privacy. Therefore, we propose ReCIP, a revocable certificateless PDP scheme with identity privacy, where a TPA can perform public data integrity batch verification for a user while learning no useful knowledge about user identity privacy. Technically, we introduce a new user revocation strategy that directly revokes users’ secret keys, with no correlation to the number of data blocks in place for revocation time cost. To further boost the efficiency of ReCIP, we employ a semi-generic online-offline strategy to obtain an online-offline ReCIP (ReCIPoo) to reduce the time cost of tag generation. Moreover, we conduct a formal security proof of ReCIP, where the security is reduced to simple computational Diffie–Hellman problem and discrete logistic problem. Compared to state-of-the-art solutions, our ReCIPoo achieves comparable computation and communication cost while still achieving user revocation and protecting user identity privacy.


Occlusion-Aware Visual Object Tracking Based on Multi-template Updating Siamese Network

Abstract: Visual object tracking is a crucial area of computer vision research. It aims to accurately track objects in videos with challenges such as occlusion, deformation, and lighting variations. Existing algorithms face difficulties when objects leave the camera or reappear after being occluded, and they also struggle to track objects with significant appearance changes. To address these issues, this study proposed a novel tracking algorithm. It combines tracking and detection, enabling global searching when objects are occluded or disappear and redetecting them when they reappear. A multi-template updating mechanism was used to adapt to changes in appearance. The study proposed the OTB100-AB and UAV123-AB datasets to evaluate the tracker's ability to handle target disappearance, along with the TNR metric. The proposed algorithm was evaluated on these datasets, as were OTB50 and UAV20L, which outperformed state-of-the-art algorithms and significantly improved tracking performance.


The influence of spatial dimensions of virtual environments on attitudes and nonverbal behaviors during social interactions

Abstract: Research on physical-world environments has shown that the spatial properties of built worlds are consequential for shaping psychological states and social behavior. However, it has been difficult to empirically test this in natural settings in the physical world. This study uses immersive virtual reality (VR) environments, which have shown to have comparable effects to physical-world environments, to investigate the influence of two spatial dimensions (ceiling height and floor area) on individuals’ attitudes and nonverbal behaviors during social interactions. In the present study, groups of three to four physically remote participants wore VR headsets (n = 110) and took part in discussions every week for four weeks in one of four virtual environments that varied in their spatial dimensions (low or high ceilings, small or large floor areas). Results showed that, when in a virtual environment with a high ceiling, participants reported feeling greater perceived restorativeness, awe, and momentary affective well-being, compared to when they were in virtual environments with low ceilings. Participants also paid more social attention (i.e., looked at other group members), when they were in virtual environments with high ceilings. When in a virtual environment with a large floor area, participants reported having a greater sense of awe, compared to environments with small floor areas. Furthermore, when in a large environment with a high ceiling, participants physically moved their heads more slowly and virtually stood further apart from their group members, compared to the other three conditions. We discuss implications for theoretical work on context and behaviors as well as design of social VR environments.


Federated learning with a Balanced Heterogeneous-Yoke and Loose restriction

Abstract: The most critical challenges of federated learning for joint modeling via model sharing are the problem of model drift due to the heterogeneous distribution of local data, and the cooperation of a subset of random devices due to communication constraints. To address these dilemmas, we propose a novel Federated Learning algorithm with a Balanced Heterogeneous-Yoke and Loose restriction(FedByL), which chooses to exploit more historical gradients from the client to accelerate model convergence; in addition, a more relaxed local restriction is chosen in local updates, and an incentive clause is proposed to encourage cooperative clients to find the global optimum. Experiments and analyses on 95 sets of five real-world datasets and one synthetic dataset show that FedByL achieves better in terms of convergence speed and accuracy compared to superior algorithms (e.g. FedDyn, SCAFFOLD). In particular, FedByL communication overhead is the same as FedAvg, being lightweight and requiring only one additional storage variable locally, which greatly reduces the communication burden and the risk of information being hijacked.


SelectE: Multi-scale adaptive selection network for knowledge graph representation learning

Abstract: Most knowledge graphs in the real world suffer from incompleteness which can be addressed through knowledge graph representation learning (KGRL) techniques that use known facts to infer missing links. In this paper, a novel multi-scale adaptive selection network for KGRL, namely SelectE, is developed to learn richer multi-scale interactive features and automatically select important features, thereby achieving promising KGRL performance. Specifically, first, the input feature matrix is redesigned to better cooperate with multi-scale convolution to improve the interaction of entities and relations. Second, a multi-scale learning module is designed to learn richer multi-scale features from the input matrix using multiple branches with different kernel sizes. Third, to automatically strengthen the important features and suppress the useless features, a multi-scale adaptive selection mechanism is proposed to dynamically allocate the weights of the obtained features based on their contained information. The core of SelectE is to maximize interactions while also considering how to better utilize features. Finally, the outstanding performance of SelectE is validated by a series of comparison experiments on seven benchmark datasets (FB15k-237, WN18RR, FB15k, WN18, YAGO3-10, KINSHIP, UMLS). The experimental results show that SelectE outperforms other state-of-the-art models, demonstrating its remarkable performance and generalization.


Extending User Control for Image Stylization using Hierarchical Style Transfer Networks

Abstract:
Abstract
 The field of neural style transfer refers to the re-rendering of content image while fusing the features of a style image. The recent studies either focus on multiple style transfer or arbitrary style transfer while using perceptual and fixpoint content losses in their respective network architectures. The aforementioned losses provide notable stylization results but lack the liberty of style control to the user. Consequently, the stylization results also compromise the preservation of details with respect to the content image. This work proposes the hierarchical style transfer network (HSTN) for the image stylization task that could provide the user with the liberty to control the degree of incurred style via denoising parameter. The HSTN incorporates the proposed fixpoint control loss that preserves details from the content image and the addition of denoising CNN network (DnCNN) and denoising loss for allowing the user to control the level of stylization. The encoder-decoder block, the DnCNN block, and the loss network block make the basic building blocks of HSTN. Extensive experiments have been carried out, and the results are compared with existing works to demonstrate the effectiveness of HSTN. The subjective user evaluation shows that the HSTN's stylization represents the best fusion of style and generates unique stylization results while preserving the content image details, which is evident by acquiring 12% better results than the second-best performing method. It has also been observed that the proposed work is amongst the studies that achieve the best trade-off regarding content and style classification scores, i.e. 37.64% and 60.27%, respectively.


An extended proximity relation and quantified aggregation for designing robust fuzzy query engine

Abstract: In this article, we propose a novel model of a robust fuzzy query engine that addresses vagueness in data and users’ requirements. It aims to assist users in recommending similar products or services by retrieving the most suitable entities when the limitations of queries and recommendation approaches are recognized. The proposed fuzzy engine model considers various complex aspects, including imprecise preferences explained by linguistic terms, uncertain data in datasets, connections among elementary requirements, and the lack of historical data necessary for personalized recommendations. To achieve this goal, we propose a similarity matching based on the extended proximity relation and an adapted conformance measure for elementary requirements. In this direction, a new monotonicity property for proximity relation is introduced to ensure consistency in similarities among ordinal categorical data (including binary data) expressed by crisp or fuzzy numbers. Therefore, the conformance measure used to evaluate the similarity between user requirements and attribute values is expressed as a fuzzy number. Next, we propose a quantified aggregation of elementary requirements by strictly monotone fuzzy relative quantifiers. The flexibility is further extended by a convex combination of possibility and necessity measures. A hotel selection experiment is being carried out to explore the potential of the proposed fuzzy query engine. Finally, the limitations and usefulness of the proposed approach are addressed.


Constructing the three-dimensional extinction density maps using V-net

Abstract: ABSTRACT One of the major challenges we face is how to quickly and accurately create the three-dimensional (3D) density distributions of interstellar dust in the Milky Way using extinction and distance measurements of large samples of stars. In this study, we introduce a novel machine-learning approach that utilizes a convolution neural network, specifically a V-net, to infer the 3D distribution of dust density. Experiments are performed within two regions located towards the Galactic anticentre. The neural network is trained and tested using 10 000 simulations of dust density and line-of-sight extinction maps. Evaluation of the test sample confirms the successful generation of dust density maps from extinction maps by our model. Additionally, the performance of the trained network is evaluated using data from the literature. Our results demonstrate that our model is capable of capturing detailed dust density variations and can recover dust density maps while reducing the ‘fingers of god’ effect. Moving forward, we plan to apply this model to real observational data to obtain the fine distribution of dust at large and small scales in the Milky Way.


CFAuditChain: Audit BlockChain Based On Cuckoo Filter

Abstract: Abstract Log blockchain can be used to ensure the integrity of log data. However, current methods are facing the problems of throughput mismatch and rough audit granularity. Packaging multiple logs to generate integrity proofs improves throughput but reduces audit granularity, and the auditor can only locate the tampering log packet instead of specific records. This paper proposes CFAuditChain, an audit blockchain based on a cuckoo filter, where the proof of existence (PoE) of the single log is calculated while calculating the integrity proof of packet logs. The PoE is saved in the cuckoo filter and stored using blockchain for immutability. Therefore, the auditor can verify the credibility of each log based on the filter when the integrity proof of a log packet does not match. The theoretical analysis and experimental results show that CFAuditChain provides the granularity of audit logs down to the records level at an acceptable cost.


Identity-Based Encryption with (Almost) Tight Security in the Multi-instance, Multi-ciphertext Setting

Abstract: Abstract We construct an identity-based encryption (IBE) scheme that is tightly secure in a very strong sense. Specifically, we consider a setting with many instances of the scheme and many encryptions per instance. In this setting, we reduce the security of our scheme to a variant of a simple assumption used for a similar purpose by Chen and Wee (CRYPTO 2013, Springer, 2013). The security loss of our reduction is $$\textbf{O} (k)$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>O</mml:mi> <mml:mo>(</mml:mo> <mml:mi>k</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> (where $$k $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>k</mml:mi> </mml:math> is the security parameter). Our scheme is the first IBE scheme to achieve this strong flavor of tightness under a simple assumption. Technically, our scheme is a variation of the IBE scheme by Chen and Wee. However, in order to “lift” their results to the multi-instance, multi-ciphertext case, we need to develop new ideas. In particular, while we build on (and extend) their high-level proof strategy, we deviate significantly in the low-level proof steps.


To the 75th Anniversary of Academician of the Russian Academy of Sciences Evgeny Lvovich Nasonov

Abstract: On September 4, 2023, the 75th anniversary of the largest scientist, clinician and innovator, a talented teacher in the field of rheumatology, an outstanding healthcare organizer, scientific director of the Federal State Budgetary Scientific Institution “V.A. Nasonova Research Institute of Rheumatology”, Honored Scientist of the Russian Federation, Academician of the Russian Academy of Sciences Evgeny Lvovich Nasonov.


Line segment detection algorithm in image extraction improvement study

Abstract: In recent years, image processing technology has been developing and maturing, but due to the influence of many interfering factors in the acquisition process, there is a large amount of redundant information in the images obtained. The line segment detection algorithm in image extraction needs to be improved. This study utilizes computer technology to improve the line segment detection technology, and designs a line segment detection algorithm based on the linear detection improvement. Firstly, based on the basic principle of straight line detection algorithm, for the problems of line segment breakage and missing in straight line detection, RGB three-channel grayscale map is applied to detect line segments. Then the detected line segments are connected, merged and deleted. The test results show that the line segment detection algorithm improved based on straight line detection has the highest accuracy rate of 94.50 %, and the average processing time per image is also the lowest at 0.2 s. The algorithm runs faster at 0.25 s and has a higher F-value. It is able to detect the boundaries of a variety of rectangular targets, using the improved line segment detection algorithm has a wide range of applicability, lower error rate, and strong anti-interference ability. The improved line segment detection algorithm has a greater advantage in rectangular target extraction for document, text and book type images.


A penalized criterion for selecting the number of clusters for K-medians

Abstract: Clustering is a usual unsupervised machine learning technique for grouping the data points into groups based upon similar features. We focus here on unsupervised clustering for contaminated data, i.e in the case where K-medians algorithm should be preferred to K-means because of its robustness. More precisely, we concentrate on a common question in clustering: how to chose the number of clusters? The answer proposed here is to consider the choice of the optimal number of clusters as the minimization of a penalized criterion. In this paper, we obtain a suitable penalty shape for our criterion and derive an associated oracle-type inequality. Finally, the performance of this approach with different types of K-medians algorithms is compared on a simulation study with other popular techniques. All studied algorithms are available in the R package Kmedians on CRAN.


Taiyi: a bilingual fine-tuned large language model for diverse biomedical tasks

Abstract: Most existing fine-tuned biomedical large language models (LLMs) focus on enhancing performance in monolingual biomedical question answering and conversation tasks. To investigate the effectiveness of the fine-tuned LLMs on diverse biomedical natural language processing (NLP) tasks in different languages, we present Taiyi, a bilingual fine-tuned LLM for diverse biomedical NLP tasks.We first curated a comprehensive collection of 140 existing biomedical text mining datasets (102 English and 38 Chinese datasets) across over 10 task types. Subsequently, these corpora were converted to the instruction data used to fine-tune the general LLM. During the supervised fine-tuning phase, a 2-stage strategy is proposed to optimize the model performance across various tasks.Experimental results on 13 test sets, which include named entity recognition, relation extraction, text classification, and question answering tasks, demonstrate that Taiyi achieves superior performance compared to general LLMs. The case study involving additional biomedical NLP tasks further shows Taiyi's considerable potential for bilingual biomedical multitasking.Leveraging rich high-quality biomedical corpora and developing effective fine-tuning strategies can significantly improve the performance of LLMs within the biomedical domain. Taiyi shows the bilingual multitasking capability through supervised fine-tuning. However, those tasks such as information extraction that are not generation tasks in nature remain challenging for LLM-based generative approaches, and they still underperform the conventional discriminative approaches using smaller language models.


Sustainable deployment of clinical prediction tools—a 360° approach to model maintenance

Abstract: As the enthusiasm for integrating artificial intelligence (AI) into clinical care grows, so has our understanding of the challenges associated with deploying impactful and sustainable clinical AI models. Complex dataset shifts resulting from evolving clinical environments strain the longevity of AI models as predictive accuracy and associated utility deteriorate over time.Responsible practice thus necessitates the lifecycle of AI models be extended to include ongoing monitoring and maintenance strategies within health system algorithmovigilance programs. We describe a framework encompassing a 360° continuum of preventive, preemptive, responsive, and reactive approaches to address model monitoring and maintenance from critically different angles.We describe the complementary advantages and limitations of these four approaches and highlight the importance of such a coordinated strategy to help ensure the promise of clinical AI is not short-lived.


What is a blink? Classifying and characterizing blinks in eye openness signals

Abstract: Blinks, the closing and opening of the eyelids, are used in a wide array of fields where human function and behavior are studied. In data from video-based eye trackers, blink rate and duration are often estimated from the pupil-size signal. However, blinks and their parameters can be estimated only indirectly from this signal, since it does not explicitly contain information about the eyelid position. We ask whether blinks detected from an eye openness signal that estimates the distance between the eyelids (EO blinks) are comparable to blinks detected with a traditional algorithm using the pupil-size signal (PS blinks) and how robust blink detection is when data quality is low. In terms of rate, there was an almost-perfect overlap between EO and PS blink (F1 score: 0.98) when the head was in the center of the eye tracker's tracking range where data quality was high and a high overlap (F1 score 0.94) when the head was at the edge of the tracking range where data quality was worse. When there was a difference in blink rate between EO and PS blinks, it was mainly due to data loss in the pupil-size signal. Blink durations were about 60 ms longer in EO blinks compared to PS blinks. Moreover, the dynamics of EO blinks was similar to results from previous literature. We conclude that the eye openness signal together with our proposed blink detection algorithm provides an advantageous method to detect and describe blinks in greater detail.


Fuzzy cognitive maps for municipal governance improvement

Abstract: This paper applies Fuzzy Cognitive Maps (FCMs) to understand the diverse behavior of municipal governments in Ecuador to find common elements that influence the well-being of citizens in the short and long term. Information gathering was conducted in two stages: in the first one, a group of 16 national experts was consulted to develop the initial FCM; in the second stage, local experts from 220 municipalities were interviewed to collect information on the general validity of initial FCMs and specific values given to concepts and relationships in their municipalities. Results show the importance of certain concepts for long-term municipal performance, such as the need for a competitive entrepreneurial sector, improving human resources in the municipality, and, particularly, having a competent mayor with leadership skills and a forward-looking vision that enables the development of municipal projects required to reach an efficient and equitable coverage of goods and services throughout the city. Through the application of genetic algorithms, the FCM was calibrated to ascertain the long-term dynamics of municipal development and the optimal values of the concepts that would optimize the attainment of the set objectives. The derived outcomes suggest the desirability of the maintenance of, in principle, unwanted structures like financial transfers from the central government and the need to exploit natural resources to attain urban development.


Klasik ve Bulanık Kenar Algılama Yöntemlerinin Karşılaştırılması

Abstract: Edge detection is one of the challenging problems in image processing. Four different classical edge detection methods—Sobel, Prewitt, Roberts, and Canny—and type-1 and type-2 fuzzy logic-based edge detection methods are applied to analyze two separate datasets with various properties. The datasets are STARE which contains medical images of the retina and BIPED which contains images of the street. Furthermore, two separate hybrid fuzzy logic methods are implemented. The type-1 and type-2 fuzzy inference techniques are combined to produce the hybrid-1 and hybrid-2 approaches, using the "AND" and "OR" logic operators. We compare the simulation results for each technique using three different image quality metrics. These are Mean Square Error (MSE), Peak Signal Noise Ratio (PSNR), and Structural Similarity Index (SSIM). The type-2 fuzzy technique outperformed the hybrid-1 fuzzy method in visual quality metrics comparison, demonstrating superior blood vessel recognition on the STARE retinal image dataset—a dataset that more closely resembles the human visual system. Using the BIPED street image dataset, the hybrid-1 fuzzy approach outperformed the Roberts method. The hybrid-1 fuzzy technique showed good results in the second order for both kinds of datasets. Any data and general applications can take advantage of it.


Generalization of Deep-Learning Models for Classification of Local Distance Earthquakes and Explosions across Various Geologic Settings

Abstract: Abstract Although accurately classifying signals from earthquakes and explosions at local distance (&lt;250 km) remains an important task for seismic network operations, the growing volume of available seismic data presents a challenge for analysts using traditional source discrimination techniques. In recent years, deep-learning models have proven effective at discriminating between low-magnitude earthquakes and explosions measured at local distances, but it is not clear how well these models are capable of generalizing across different geological settings. To address the issue of generalization between regions, we train deep-learning models (convolutional neural networks [CNNs]) on time–frequency representations (scalograms) of three-component earthquake and explosion signals from eight different regions in the continental United States. We explore scenarios where models are trained on data from all regions, individual regions, or all but one region. We find that although CNN models trained on individual regions do not necessarily generalize well across different settings, models trained on multiple regions that include diverse path coverage generalize to new regions, with station-level accuracy of up to 90% or more for data sets from unseen regions. In general, CNN-based discrimination models significantly outperform models based on uncorrected P/S ratio (measured in the 10–18 Hz frequency band), even when CNN models are tested on data from entirely unseen regions.


To the 75th Anniversary of Academician of the Russian Academy of Sciences Gennady Petrovich Kotelnikov

Abstract: On January 11, 2024, the largest scientist in the field of traumatology and orthopedics, a talented organizer of healthcare and higher education, a public figure, laureate of the State Prize of the Russian Federation in the field of science and technology, three times laureate of the Government of the Russian Federation, Honored Scientist of the Russian Federation, academician, turned 75 RAS to Gennady Petrovich Kotelnikov.


Constructing polynomial libraries for reservoir computing in nonlinear dynamical system forecasting

Abstract: Reservoir computing is an effective model for learning and predicting nonlinear and chaotic dynamical systems; however, there remains a challenge in achieving a more dependable evolution for such systems. Based on the foundation of Koopman operator theory, considering the effectiveness of the sparse identification of nonlinear dynamics algorithm to construct candidate nonlinear libraries in the application of nonlinear data, an alternative reservoir computing method is proposed, which creates the linear Hilbert space of the nonlinear system by including nonlinear terms in the optimization process of reservoir computing, allowing for the application of linear optimization. We introduce an implementation that incorporates a polynomial transformation of arbitrary order when fitting the readout matrix. Constructing polynomial libraries with reservoir-state vectors as elements enhances the nonlinear representation of reservoir states and more easily captures the complexity of nonlinear systems. The Lorenz-63 system, the Lorenz-96 system, and the Kuramoto-Sivashinsky equation are used to validate the effectiveness of constructing polynomial libraries for reservoir states in the field of state-evolution prediction of nonlinear and chaotic dynamical systems. This study not only promotes the theoretical study of reservoir computing, but also provides a theoretical and practical method for the prediction of nonlinear and chaotic dynamical system evolution.


Readout-Induced Suppression and Enhancement of Superconducting Qubit Lifetimes

Abstract: It has long been known that the lifetimes of superconducting qubits suffer during readout, increasing readout errors. We show that this degradation is due to the anti-Zeno effect, as readout-induced dephasing broadens the qubit so that it overlaps "hot spots" of strong dissipation, likely due to two-level systems in the qubit's bath. Using a flux-tunable qubit to probe the qubit's frequency-dependent loss, we accurately predict the change in lifetime during readout with a new self-consistent master equation that incorporates the modification to qubit relaxation due to measurement-induced dephasing. Moreover, we controllably demonstrate both the Zeno and anti-Zeno effects, which can explain both suppression and the rarer enhancement of qubit lifetimes during readout.


Image Division Using Threshold Schemes with Privileges

Abstract: Threshold schemes are used among cryptographic techniques for splitting visual data. Such methods allow the generation of a number of secret shares, a certain number of which need to be assembled in order to reconstruct the original image. Traditional techniques for partitioning secret information generate equal shares, i.e., each share has the same value when reconstructing the original secret. However, it turns out that it is possible to develop and use partitioning protocols that allow the generation of privileged shares, i.e., those that allow the reconstruction of secret data in even smaller numbers. This paper will therefore describe new information sharing protocols that create privileged shares, which will also use visual authorization codes based on subject knowledge to select privileged shares for secret restoration. For the protocols described, examples of their functioning will be presented, and their complexity and potential for use in practical applications will be determined.