On the Out-Of-Distribution Generalization of Multimodal Large Language  Models

Abstract: We investigate the generalization boundaries of current Multimodal Large Language Models (MLLMs) via comprehensive evaluation under out-of-distribution scenarios and domain-specific tasks. We evaluate their zero-shot generalization across synthetic images, real-world distributional shifts, and specialized datasets like medical and molecular imagery. Empirical results indicate that MLLMs struggle with generalization beyond common training domains, limiting their direct application without adaptation. To understand the cause of unreliable performance, we analyze three hypotheses: semantic misinterpretation, visual feature extraction insufficiency, and mapping deficiency. Results identify mapping deficiency as the primary hurdle. To address this problem, we show that in-context learning (ICL) can significantly enhance MLLMs' generalization, opening new avenues for overcoming generalization barriers. We further explore the robustness of ICL under distribution shifts and show its vulnerability to domain shifts, label shifts, and spurious correlation shifts between in-context examples and test data.


Rapid quantum approaches for combinatorial optimisation inspired by optimal state-transfer

Abstract: We propose a new design heuristic to tackle combinatorial optimisation problems, inspired by Hamiltonians for optimal state-transfer. The result is a rapid approximate optimisation algorithm. We provide numerical evidence of the success of this new design heuristic. We find this approach results in a better approximation ratio than the Quantum Approximate Optimisation Algorithm at lowest depth for the majority of problem instances considered, while utilising comparable resources. This opens the door to investigating new approaches for tackling combinatorial optimisation problems, distinct from adiabatic-influenced approaches.


Musical Systems with $\mathbb{Z}_n$ -- Cayley Graphs

Abstract: We apply geometric group theory to study and interpret known concepts from Western music. We show that chords, the circle of fifths, scales and certain aspects of the first species of counterpoint are encoded in the Cayley graph of the group $\mathbb{Z}_{12}$, generated by $3$ and $4$. Using $\mathbb{Z}_{12}$ as a model, we extend the above music concepts to a particular class of groups $\mathbb{Z}_{n}$, which displays geometric and algebraic features similar to $\mathbb{Z}_{12}$. We identify a weaker form of counterpoint which, in particular leads to Fux's dichotomy in $\mathbb{Z}_{12}$, and to consonant sets in $\mathbb{Z}_n$. Using Maple software, we implement these new constructions and show how to experiment with them musically.


Designing NLP-based solutions for requirements variability management:  experiences from a design science study at Visma

Abstract: Context and motivation: In this industry-academia collaborative project, a team of researchers, supported by a software architect, business analyst, and test engineer explored the challenges of requirement variability in a large business software development company. Question/problem: Following the design science paradigm, we studied the problem of requirements analysis and tracing in the context of contractual documents, with a specific focus on managing requirements variability. This paper reports on the lessons learned from that experience, highlighting the strategies and insights gained in the realm of requirements variability management. Principal ideas/results: This experience report outlines the insights gained from applying design science in requirements engineering research in industry. We show and evaluate various strategies to tackle the issue of requirement variability. Contribution: We report on the iterations and how the solution development evolved in parallel with problem understanding. From this process, we derive five key lessons learned to highlight the effectiveness of design science in exploring solutions for requirement variability in contract-based environments.


EmoWear: Exploring Emotional Teasers for Voice Message Interaction on  Smartwatches

Abstract: Voice messages, by nature, prevent users from gauging the emotional tone without fully diving into the audio content. This hinders the shared emotional experience at the pre-retrieval stage. Research scarcely explored "Emotional Teasers"-pre-retrieval cues offering a glimpse into an awaiting message's emotional tone without disclosing its content. We introduce EmoWear, a smartwatch voice messaging system enabling users to apply 30 animation teasers on message bubbles to reflect emotions. EmoWear eases senders' choice by prioritizing emotions based on semantic and acoustic processing. EmoWear was evaluated in comparison with a mirroring system using color-coded message bubbles as emotional cues (N=24). Results showed EmoWear significantly enhanced emotional communication experience in both receiving and sending messages. The animated teasers were considered intuitive and valued for diverse expressions. Desirable interaction qualities and practical implications are distilled for future design. We thereby contribute both a novel system and empirical knowledge concerning emotional teasers for voice messaging.


3D Gaussian as a New Vision Era: A Survey

Abstract: 3D Gaussian Splatting (3D-GS) has emerged as a significant advancement in the field of Computer Graphics, offering explicit scene representation and novel view synthesis without the reliance on neural networks, such as Neural Radiance Fields (NeRF). This technique has found diverse applications in areas such as robotics, urban mapping, autonomous navigation, and virtual reality/augmented reality, just name a few. Given the growing popularity and expanding research in 3D Gaussian Splatting, this paper presents a comprehensive survey of relevant papers from the past year. We organize the survey into taxonomies based on characteristics and applications, providing an introduction to the theoretical underpinnings of 3D Gaussian Splatting. Our goal through this survey is to acquaint new researchers with 3D Gaussian Splatting, serve as a valuable reference for seminal works in the field, and inspire future research directions, as discussed in our concluding section.


Unsupervised Discovery of Object-Centric Neural Fields

Abstract: We study inferring 3D object-centric scene representations from a single image. While recent methods have shown potential in unsupervised 3D object discovery from simple synthetic images, they fail to generalize to real-world scenes with visually rich and diverse objects. This limitation stems from their object representations, which entangle objects' intrinsic attributes like shape and appearance with extrinsic, viewer-centric properties such as their 3D location. To address this bottleneck, we propose Unsupervised discovery of Object-Centric neural Fields (uOCF). uOCF focuses on learning the intrinsics of objects and models the extrinsics separately. Our approach significantly improves systematic generalization, thus enabling unsupervised learning of high-fidelity object-centric scene representations from sparse real-world images. To evaluate our approach, we collect three new datasets, including two real kitchen environments. Extensive experiments show that uOCF enables unsupervised discovery of visually rich objects from a single real image, allowing applications such as 3D object segmentation and scene manipulation. Notably, uOCF demonstrates zero-shot generalization to unseen objects from a single real image. Project page: https://red-fairy.github.io/uOCF/


Parameterizations for Gradient-based Markov Chain Monte Carlo on the  Stiefel Manifold: A Comparative Study

Abstract: Orthogonal matrices play an important role in probability and statistics, particularly in high-dimensional statistical models. Parameterizing these models using orthogonal matrices facilitates dimension reduction and parameter identification. However, establishing the theoretical validity of statistical inference in these models from a frequentist perspective is challenging, leading to a preference for Bayesian approaches because of their ability to offer consistent uncertainty quantification. Markov chain Monte Carlo methods are commonly used for numerical approximation of posterior distributions, and sampling on the Stiefel manifold, which comprises orthogonal matrices, poses significant difficulties. While various strategies have been proposed for this purpose, gradient-based Markov chain Monte Carlo with parameterizations is the most efficient. However, a comprehensive comparison of these parameterizations is lacking in the existing literature. This study aims to address this gap by evaluating numerical efficiency of the four alternative parameterizations of orthogonal matrices under equivalent conditions. The evaluation was conducted for four problems. The results suggest that polar expansion parameterization is the most efficient, particularly for the high-dimensional and complex problems. However, all parameterizations exhibit limitations in significantly high-dimensional or difficult tasks, emphasizing the need for further advancements in sampling methods for orthogonal matrices.


From Uncertainty to Precision: Enhancing Binary Classifier Performance  through Calibration

Abstract: The assessment of binary classifier performance traditionally centers on discriminative ability using metrics, such as accuracy. However, these metrics often disregard the model's inherent uncertainty, especially when dealing with sensitive decision-making domains, such as finance or healthcare. Given that model-predicted scores are commonly seen as event probabilities, calibration is crucial for accurate interpretation. In our study, we analyze the sensitivity of various calibration measures to score distortions and introduce a refined metric, the Local Calibration Score. Comparing recalibration methods, we advocate for local regressions, emphasizing their dual role as effective recalibration tools and facilitators of smoother visualizations. We apply these findings in a real-world scenario using Random Forest classifier and regressor to predict credit default while simultaneously measuring calibration during performance optimization.


Aya Model: An Instruction Finetuned Open-Access Multilingual Language  Model

Abstract: Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages. What does it take to broaden access to breakthroughs beyond first-class citizen languages? Our work introduces Aya, a massively multilingual generative language model that follows instructions in 101 languages of which over 50% are considered as lower-resourced. Aya outperforms mT0 and BLOOMZ on the majority of tasks while covering double the number of languages. We introduce extensive new evaluation suites that broaden the state-of-art for multilingual eval across 99 languages -- including discriminative and generative tasks, human evaluation, and simulated win rates that cover both held-out tasks and in-distribution performance. Furthermore, we conduct detailed investigations on the optimal finetuning mixture composition, data pruning, as well as the toxicity, bias, and safety of our models. We open-source our instruction datasets and our model at https://hf.co/CohereForAI/aya-101


Secured Auto Encryption and Authentication Process for Cloud Computing Security

Abstract: In the IT sector, cloud computing is a rapidly expanding technology. Users are being offered more and more flexible and appropriate services by cloud service providers (CSP). Cloud computing users are increasing at a steady pace. Users and CSP are looking for appropriate security measures as cloud computing becomes a need in modern life. A large number of researchers are studying cloud computing security issues. This work introduces an appropriate cryptography and secure user authentication system, which includes auto encryption and a mechanism for updating keys on a cloud site that is not initially accessible to users. After a predetermined amount of time, this operation will recur frequently on the cloud site. This protocol can be carried out manually by CSP at any time, or it can run automatically after a predetermined amount of time. This will guarantee additional data security. Additionally, this procedure adds a level to the user's credentials. Even with legitimate credentials, these feathers will ensure that hackers cannot obtain original files or data. The suggested protocol is emulated by CloudSim, and we saw that there is an additional step requiring user credentials, requiring some time to access files or data for the first time. Although this small amount of time is insignificant and can be disregarded, it significantly improves security.


Forecasting high-impact research topics via machine learning on evolving  knowledge graphs

Abstract: The exponential growth in scientific publications poses a severe challenge for human researchers. It forces attention to more narrow sub-fields, which makes it challenging to discover new impactful research ideas and collaborations outside one's own field. While there are ways to predict a scientific paper's future citation counts, they need the research to be finished and the paper written, usually assessing impact long after the idea was conceived. Here we show how to predict the impact of onsets of ideas that have never been published by researchers. For that, we developed a large evolving knowledge graph built from more than 21 million scientific papers. It combines a semantic network created from the content of the papers and an impact network created from the historic citations of papers. Using machine learning, we can predict the dynamic of the evolving network into the future with high accuracy, and thereby the impact of new research directions. We envision that the ability to predict the impact of new ideas will be a crucial component of future artificial muses that can inspire new impactful and interesting scientific ideas.


Group Decision-Making among Privacy-Aware Agents

Abstract: How can individuals exchange information to learn from each other despite their privacy needs and security concerns? For example, consider individuals deliberating a contentious topic and being concerned about divulging their private experiences. Preserving individual privacy and enabling efficient social learning are both important desiderata but seem fundamentally at odds with each other and very hard to reconcile. We do so by controlling information leakage using rigorous statistical guarantees that are based on differential privacy (DP). Our agents use log-linear rules to update their beliefs after communicating with their neighbors. Adding DP randomization noise to beliefs provides communicating agents with plausible deniability with regard to their private information and their network neighborhoods. We consider two learning environments one for distributed maximum-likelihood estimation given a finite number of private signals and another for online learning from an infinite, intermittent signal stream. Noisy information aggregation in the finite case leads to interesting tradeoffs between rejecting low-quality states and making sure all high-quality states are accepted in the algorithm output. Our results flesh out the nature of the trade-offs in both cases between the quality of the group decision outcomes, learning accuracy, communication cost, and the level of privacy protections that the agents are afforded.


Learning How To Ask: Cycle-Consistency Refines Prompts in Multimodal  Foundation Models

Abstract: When LLMs perform zero-shot inference, they typically use a prompt with a task specification, and generate a completion. However, there is no work to explore the possibility of the reverse - going from completion to task specification. In this paper, we employ both directions to perform cycle-supervised learning entirely in-context. Our goal is to create a forward map f : X -> Y (e.g. image -> generated caption), coupled with a backward map g : Y -> X (e.g. caption -> generated image) to construct a cycle-consistency "loss" (formulated as an update to the prompt) to enforce g(f(X)) ~= X. The technique, called CyclePrompt, uses cycle-consistency as a free supervisory signal to iteratively craft the prompt. Importantly, CyclePrompt reinforces model performance without expensive fine-tuning, without training data, and without the complexity of external environments (e.g. compilers, APIs). We demonstrate CyclePrompt in two domains: code generation and image captioning. Our results on the HumanEval coding benchmark put us in first place on the leaderboard among models that do not rely on extra training data or usage of external environments, and third overall. Compared to the GPT4 baseline, we improve accuracy from 80.5% to 87.2%. In the vision-language space, we generate detailed image captions which outperform baseline zero-shot GPT4V captions, when tested against natural (VQAv2) and diagrammatic (FigureQA) visual question-answering benchmarks. To the best of our knowledge, this is the first use of self-supervised learning for prompting.


Combining Insights From Multiple Large Language Models Improves  Diagnostic Accuracy

Abstract: Background: Large language models (LLMs) such as OpenAI's GPT-4 or Google's PaLM 2 are proposed as viable diagnostic support tools or even spoken of as replacements for "curbside consults". However, even LLMs specifically trained on medical topics may lack sufficient diagnostic accuracy for real-life applications. Methods: Using collective intelligence methods and a dataset of 200 clinical vignettes of real-life cases, we assessed and compared the accuracy of differential diagnoses obtained by asking individual commercial LLMs (OpenAI GPT-4, Google PaLM 2, Cohere Command, Meta Llama 2) against the accuracy of differential diagnoses synthesized by aggregating responses from combinations of the same LLMs. Results: We find that aggregating responses from multiple, various LLMs leads to more accurate differential diagnoses (average accuracy for 3 LLMs: $75.3\%\pm 1.6pp$) compared to the differential diagnoses produced by single LLMs (average accuracy for single LLMs: $59.0\%\pm 6.1pp$). Discussion: The use of collective intelligence methods to synthesize differential diagnoses combining the responses of different LLMs achieves two of the necessary steps towards advancing acceptance of LLMs as a diagnostic support tool: (1) demonstrate high diagnostic accuracy and (2) eliminate dependence on a single commercial vendor.


Can Text-to-image Model Assist Multi-modal Learning for Visual  Recognition with Visual Modality Missing?

Abstract: Multi-modal learning has emerged as an increasingly promising avenue in vision recognition, driving innovations across diverse domains ranging from media and education to healthcare and transportation. Despite its success, the robustness of multi-modal learning for visual recognition is often challenged by the unavailability of a subset of modalities, especially the visual modality. Conventional approaches to mitigate missing modalities in multi-modal learning rely heavily on algorithms and modality fusion schemes. In contrast, this paper explores the use of text-to-image models to assist multi-modal learning. Specifically, we propose a simple but effective multi-modal learning framework GTI-MM to enhance the data efficiency and model robustness against missing visual modality by imputing the missing data with generative transformers. Using multiple multi-modal datasets with visual recognition tasks, we present a comprehensive analysis of diverse conditions involving missing visual modality in data, including model training. Our findings reveal that synthetic images benefit training data efficiency with visual data missing in training and improve model robustness with visual data missing involving training and testing. Moreover, we demonstrate GTI-MM is effective with lower generation quantity and simple prompt techniques.


Primitive elements of finite fields $\mathbf{F}_{q^r}$ avoiding affine  hyperplanes for $q=4$ and $q=5$

Abstract: For a finite field $\mathbf{F}_{q^r}$ with fixed $q$ and $r$ sufficiently large, we prove the existence of a primitive element outside of a set of $r$ many affine hyperplanes for $q=4$ and $q=5$. This complements earlier results by Fernandes and Reis for $q\ge 7$. For $q=3$ the analogous result can be derived from a very recent bound on character sums of Iyer and Shparlinski. For $q=2$ the set consists only of a single element, and such a result is thus not possible.


Cryo Chiplets for Quantum Applications

Abstract: that degrades QC performance is referred to as a defect. Here, we focus on a specific defect known as a frequency collision. When transmon frequencies collide, their difference falls within a range that limits two-qubit gate fidelity. Frequency collisions occur with greater probability on larger QCs, causing collision-free yields to decline as the number of on-chip qubits increases. As a solution, we propose exploiting the higher yields associated with smaller QCs by integrating quantum chiplets within quantum multi-chip modules (MCMs). Yield, gate performance, and application-based analysis show the feasibility of QC scaling through modularity. Our results demonstrate that chiplet architectures, relative to monolithic designs, benefit from average yield improvements ranging from 9.6 – 92.6 × for ≲5 qubit machines. In addition, our simulations explore the design space of chiplet systems and discover configurations that demonstrate average two-qubit gate infidelity reductions that are at best 0.815 × their monolithic counterpart. Lastly, we observe that carefully-selected modular systems achieve fidelity improvements on a range of benchmark circuits.


Factors Influencing Acceptance of ILMU E-Learning Among Lecturers: An Empirical Study Based on UTAUT Model

Abstract: E-learning is a form of innovation in technology used in educational field, including higher education. University of Pembangunan Nasional “Veteran” Jawa Timur is one of many universities that have implemented e-learning called ILMU to support the teaching-learning process. The application of ILMU as e-learning has yet to be utilised by lecturers, due to some challenges in implementation of ILMU regarding accessibility and features of ILMU. Meanwhile, successful implementation of a technology requires acceptance from its users. This research was acquited to define what acceptance factors that influence lecturers while accessing ILMU. This study is measured using UTAUT model. The research was carried on by quantitatively distributing questionnaires to 60 lecturers. Data were analyzed and processed using SEM-PLS technique and SMARTPLS 3.0 application. Factors that influence users to receive ILMU e-learning and significantly are effort expectancy, social influence, facilitating conditions, and behavioral intention. Meanwhile, performance expectancy does not influence users significantly to accept ILMU e-learning. These factors are key indicators to of the implementation and improvement of ILMU e-learning, thus it will develop a better implementation for the lecturers to use and accept it.


An exploration system to effectively analyze the visual metaphor used in sentiment visualization

Abstract: In the field of data visualization, there has been a recent trend of using a complex type of visualization with a multidimensional structure or using several visualizations in parallel when summarizing the results of sentiment analysis. Although this trend may be useful for sophisticated sentiment analysis, such analysis is difficult for the general public and novice researchers. To address this issue, there has recently been a trend of visualizing sentiments using visual metaphors. To facilitate the understanding of related cases, it is necessary to have a systematic means of grasping the sentiment target, the purpose and motivation of research, and the representations used as substitutes for visual metaphors. Therefore, the objective of the present study was to develop an exploration system that can analyze the visual metaphors used in the case of sentiment visualization. For this study, (1) sentiment visualization cases in which visual metaphors are used were collected. (2) After a taxonomy composed of the categories of “target, intermediation, representation, visual variable, and visualization technique” was constructed, it was used to analyze sentences of visual metaphors appearing in sentiment visualization cases. (3) An exploration system capable of grasping the semantic relationships of sub-elements within the five categories of the taxonomy and intuitively interpreting visual metaphors was developed so that appropriate cases can be recommended to sentiment visualization researchers. (4) The approach and usefulness of the exploration system were explained using user scenarios. (5) A case study was conducted to show that the provided system can be analyzed from various perspectives. (6) The usability of the exploration system was demonstrated through a verification targeting experts. The proposed system allows researchers and analysts to intuitively grasp “what types of visual metaphor method and idea should be equipped to visualize sentiment data in an easier way to understand.”


Blockchain Technology in Supply Chain Management

Abstract: Blockchain is a promising technology with its features such as immutability and decentralized database. It has applications in various fields such as pharmaceutical, finance, & food industry. At the core of its heart lies its feature, traceability which is the most desired key in supply chains. However, supply chains have always been hit rock bottom by scandals and controversies. In this review paper, we have explored the advancement and research gaps, of blockchain technology (BT) in supply chain management (SCM). We have used the Prisma framework for systematic literature review (SLR) and included a minuscule amount of grey literature to reduce publication bias. We found that supply chain traceability and transparency is the most researched objective in SCM. There was hardly any research in supply chain resilience. Further, we found that 40 % of the papers were application based. Most articles have focused on the advantages of BT, rather than analyzing it critically. This study will help identify gaps and, suitable actions to be followed for an efficient implementation of BT in SCM.


Insights and caveats from mining local and global temporal motifs in  cryptocurrency transaction networks

Abstract: Distributed ledger technologies have opened up a wealth of fine-grained transaction data from cryptocurrencies like Bitcoin and Ethereum. This allows research into problems like anomaly detection, anti-money laundering, pattern mining and activity clustering (where data from traditional currencies is rarely available). The formalism of temporal networks offers a natural way of representing this data and offers access to a wealth of metrics and models. However, the large scale of the data presents a challenge using standard graph analysis techniques. We use temporal motifs to analyse two Bitcoin datasets and one NFT dataset, using sequences of three transactions and up to three users. We show that the commonly used technique of simply counting temporal motifs over all users and all time can give misleading conclusions. Here we also study the motifs contributed by each user and discover that the motif distribution is heavy-tailed and that the key players have diverse motif signatures. We study the motifs that occur in different time periods and find events and anomalous activity that cannot be seen just by a count on the whole dataset. Studying motif completion time reveals dynamics driven by human behaviour as well as algorithmic behaviour.


LL-GABR: Energy Efficient Live Video Streaming Using Reinforcement  Learning

Abstract: Over the recent years, research and development in adaptive bitrate (ABR) algorithms for live video streaming have been successful in improving users' quality of experience (QoE) by reducing latency to near real-time levels while delivering higher bitrate videos with minimal rebuffering time. However, the QoE models used by these ABR algorithms do not take into account that a large portion of live video streaming clients use mobile devices where a higher bitrate does not necessarily translate into higher perceived quality. Ignoring perceived quality results in playing videos at higher bitrates without a significant increase in perceptual video quality and becomes a burden for battery-constrained mobile devices due to higher energy consumption. In this paper, we propose LL-GABR, a deep reinforcement learning approach that models the QoE using perceived video quality instead of bitrate and uses energy consumption along with other metrics like latency, rebuffering events, and smoothness. LL-GABR makes no assumptions about the underlying video, environment, or network settings and can operate flexibly on different video titles, each having a different bitrate encoding ladder without additional re-training, unlike existing learning-based ABRs. Trace-driven experimental results show that LL-GABR outperforms the state-of-the-art approaches by up to 44% in terms of perceptual QoE and a 73% increase in energy efficiency as a result of reducing net energy consumption by 11%.


Seagull: Privacy preserving network verification system

Abstract: The current routing protocol used in the internet backbone is based on manual configuration, making it susceptible to errors. To mitigate these configuration-related issues, it becomes imperative to validate the accuracy and convergence of the algorithm, ensuring a seamless operation devoid of problems. However, the process of network verification faces challenges related to privacy and scalability. This paper addresses these challenges by introducing a novel approach: leveraging privacy-preserving computation, specifically multiparty computation (MPC), to verify the correctness of configurations in the internet backbone, governed by the BGP protocol. Not only does our proposed solution effectively address scalability concerns, but it also establishes a robust privacy framework. Through rigorous analysis, we demonstrate that our approach maintains privacy by not disclosing any information beyond the query result, thus providing a comprehensive and secure solution to the intricacies associated with routing protocol verification in large-scale networks.


DoRA: Weight-Decomposed Low-Rank Adaptation

Abstract: Among the widely used parameter-efficient finetuning (PEFT) methods, LoRA and its variants have gained considerable popularity because of avoiding additional inference costs. However, there still often exists an accuracy gap between these methods and full fine-tuning (FT). In this work, we first introduce a novel weight decomposition analysis to investigate the inherent differences between FT and LoRA. Aiming to resemble the learning capacity of FT from the findings, we propose Weight-Decomposed LowRank Adaptation (DoRA). DoRA decomposes the pre-trained weight into two components, magnitude and direction, for fine-tuning, specifically employing LoRA for directional updates to efficiently minimize the number of trainable parameters. By employing DoRA, we enhance both the learning capacity and training stability of LoRA while avoiding any additional inference overhead. DoRA consistently outperforms LoRA on fine-tuning LLaMA, LLaVA, and VL-BART on various downstream tasks, such as commonsense reasoning, visual instruction tuning, and image/video-text understanding.


Get More with LESS: Synthesizing Recurrence with KV Cache Compression  for Efficient LLM Inference

Abstract: Many computational factors limit broader deployment of large language models. In this paper, we focus on a memory bottleneck imposed by the key-value (KV) cache, a computational shortcut that requires storing previous KV pairs during decoding. While existing KV cache methods approach this problem by pruning or evicting large swaths of relatively less important KV pairs to dramatically reduce the memory footprint of the cache, they can have limited success in tasks that require recollecting a majority of previous tokens. To alleviate this issue, we propose LESS, a simple integration of a (nearly free) constant sized cache with eviction-based cache methods, such that all tokens can be queried at later decoding steps. Its ability to retain information throughout time shows merit on a variety of tasks where we demonstrate LESS can help reduce the performance gap from caching everything, sometimes even matching it, all while being efficient.


Applications of Artificial Intelligence in Education

Abstract: The article examines current trends and prospects for the implementation of artificial intelligence (AI) technologies in educational processes. The authors analyze various aspects of the use of AI, such as individualized learning, automated assessment of knowledge, the creation of intelligent educational platforms and personalized educational plans. The article also analyzes the ethical and social issues associated with the use of artificial intelligence in education and offers recommendations for the development of effective strategies for integrating technology into educational systems. Overall, the article seeks to shed light on the role of artificial intelligence in modern education and its potential in improving the quality of learning and developing educational practices.


Assessing Factors Influencing Customers’ Adoption of AI-Based Voice Assistants

Abstract: The study aims to assess the factors that engage and accentuate usage pertaining to consumers with artificial intelligence-based voice assistants. Drawing on the Behavioral Reasoning Theory (BRT), the paper establishes relationships between the enablers and inhibitors on the adoption intention of voice assistants. To examine the hypothesized relationship, a quantitative approach employing PLS-SEM was utilized to evaluate a representative sample of 1189 Indian consumers and examining the psychological and functional constructs leading to adoption intention for voice assistants. The study reveals that the “value of openness to change” poses a significant effect toward “reasons for” adoption, among which “performance expectancy” and “hedonic motivation” play the most significant role in establishing a favorable attitude toward voice assistants’ adoption. Among the “reasons against”, the “value barrier” and “image barrier” come out to be major determinants leading to inhibitions regarding the usage of voice assistants. Besides this, the findings reclaim that “reasons” (for/against) shape to form the attitude of consumers toward voice assistants. The current research offers preliminary evidence with regard to the investigation of voice assistant adoption using the BRT that jointly examines enabling and inhibiting elements leading to consumers’ attitude formation. The study findings contribute to Human-Computer interaction literature by utilizing BRT framework which provides incremental evidence demonstrating a much broader and organic explanation of consumers’ cognition for the adoption of voice assistant technology. It also provides implications for tech-managers and algorithm designers to build effective voice technology for superior user experience.


Pengaruh Komunikasi Interpersonal Pustakawan terhadap Kepuasan Pemustaka di Perpustakaan Era Digital (Studi Kasus di Poltekkes Kemenkes Padang)

Abstract: The purpose of this study was to determine (1) describe the interpersonal communication of librarians in the Library of Poltekkes Kemenkes Padang. (2) describe the level of user satisfaction at the Poltekkes Kemenkes Padang Library. (3) describe the effect of librarian interpersonal communication on the level of library satisfaction at the Poltekkes Kemenkes Padang Library in the digital era. This research is a type of quantitative research with descriptive methods, data obtained through distributing questionnaires at the Poltekkes Kemenkes Padang Library. The population in this study was visitor data from January to December 2022 with a sample size of 99 respondents. Sampling is determined based on the Simple Random Sampling Technique which is a sampling technique that is carried out randomly without regard to the strata in the population. The results of this study are (1) interpersonal communication of librarians at the Poltekkes Kemenkes Padang Library is classified as high with an average value of 3.07 on a good interval scale. (2) user satisfaction at the Poltekkes Kemenkes Padang Library is high with an average of 3.19 on a good interval scale. (3) Librarian interpersonal communication affects library satisfaction at the Poltekkes Kemenkes Padang Library, where the Fhitung value (39.747)> Ftabel (3.94) at a significant level of 5% which states that there is a significant influence between librarian interpersonal communication on library satisfaction at the Poltekkes Kemenkes Padang Library.


Trimodal extension based on the flexible generalized skew-normal distribution

Abstract: We propose a novel class of generalized skew-normal densities that improvesthe flexibility of empirical distributions and can systematically capture skewness,heavy tails, and multimodality. We extend the so-called flexible generalized skewnormal(FGSN) density developed by Y. Ma and M.G. Genton in 2004. The mainnovelty is the existence of a fifth-order degree term in the polynomial that appearsin the cumulative distribution function of such a density. In this case, we provethat our density has at most three modes under certain conditions for the parameters.Leveraging this new approach eases the modeling of data consisting of threesubpopulations. For validation, we present examples of both univariate and multivariatecases using demographic data from the Human Mortality Data Base (HMD).


Score: Mechanical Asynchronicity – audio album

Abstract: This exposition is the online, open-access audio album of 'Score: Mechanical Asynchronicity' by Danica Maier and Martin Scheuregger. The album allows you to play full ensemble versions of the project's two pieces, and also mix your own versions of each piece by combining and balancing the individual instruments. It accompanies the publication of a related 40-page book on the project from Beam Editions.


The Data Availability Policies of Political Science Journals

Abstract: This paper describes the data availability policies posted on the websites of a broad collection of political science journals. We code the policies of 221 political science journals, and we find that only 20% of these journals require researchers to share their data and/or code as a condition of publication. While requirements to share data and code are common among the most visible political science journals, these policies remain relatively rare in the discipline more broadly. In prioritizing reforms to improve openness and reproducibility in political science, we suggest remaining mindful of the relative rarity and simple effectiveness of requiring authors to share their data and code.


Generative AI: An AI paradigm shift in the making?

Abstract: Abstract It is sometimes difficult to evaluate progress in Generative AI, that is, image generation and large language models. This may be because they represent a paradigm shift in AI, and the traditional ways of developing, evaluating, understanding, and deploying AI systems no longer apply. Instead, we need to develop new such approaches, possibly by extending those currently in use in cognitive neuroscience and psychology. In this manner, a new AI paradigm can be created, providing a significant leap in AI research and practice.


REVITALIZING THE SCHOOL LIBRARY: EMBRACING TECHNOLOGY AND INNOVATION

Abstract: This innovation revolves in the relaunching of the new library with the aid of modern technology. Installation of new library facilities such as the multimedia corner, educational board games corner and printing corner motivated learners to visit the school library that made them more excited in learning, reading and discovering things around them. This innovation was made possible through the unwavering support from various stakeholders extending their assistance by providing the financial needs of this project. KEYWORDS: Library, Innovation,Technology


MENINGKATKAN KETERAMPILAN MENULIS DIALOG PADA MATA PELAJARAN BAHASA INDONESIA MELALUI PENDEKATAN COOPERATIVE LEARNING SISTEM STAD PADA SISWA KELAS III MI PULOERANG LAKBOK KABUPATEN CIAMIS

Abstract: One of the problems faced by the teacher is the passivity of students in class, in group study in class to express opinions, it can be said that the students' abilities are not yet fluent. not optimal, Indonesian learning achievement is still low, especially in formative tests, summative students are not yet brave enough to answer questions or express opinions, the use of models or approaches in learning is not maximized. The research model is a procedure that describes how this research is carried out. In this class action research, the researcher used the action research model developed by Kemmis and McTaggart (1990:11). Based on the success indicator, namely 75% of the class population scored above the KKM, the action in cycle II was successful. As many as 16 students (88.89%) scored above the KKM. The learning process is further improved, namely students have done peer tutoring and students are more active in learning activities with the teacher. Giving material that has been repeated makes students understand more about the material presented by the teacher. Students who better understand the meaning of the activities carried out by the teacher help friends who do not understand. Thus cooperative learning is achieved. In the pre-action, the dialogue writing skills of class III MI Puloerang Lakbok students were still low. Students who can achieve KKM only 3 children (16.67%). After carrying out the STAD type cooperative learning actions in the first cycle, it produced 8 students (44.44%). Furthermore, in cycle II, there were 16 students (88.89%) who succeeded in achieving the KKM. BIBLIOGRAPHY Anita Lie. 2002. Cooperative Learning. Jakarta: Gramedia. Dimyati dan Mudjiono.2009. Belajar dan pembelajaran, Jakarta Rineka Cipta Haryadi dan Zamzani. 1996/ 1997. Peningkatan Keterampilan Berbahasa Indonesia, Jakarta. DepDikBud Dirjen Dikti PPTK Isjoni. 2009. Cooperative Learning. Bandung : Alfabeta Kemmis, San Mc. Taggart, R.1990. The Action Reseach Planner. Deakin University Larry King Bill GILBERT. 2007 Seni berbicara, Jakarta : PT Gramedia Pustaka Utama Slamet, 2008. Keterampilan-Keterampilan Berbahasa Indonesia. Surakarta: UNS Press. Suharjono, 1995. Pedoman Karya Tulis Ilmiah, Jakarta : Dep. P & K Suharjono. 2007. PTK sebagai Kegiatan Pengembangan Profesi Guru, Jakarta : Bumi Aksara Suharsimi Arikunto. 2006 Penelitian Tindakan Kelas, Jakarta : PT Bumi Aksara. UU RI No. 20 tahun 2003 tentang SISDIKNAS. Bandung : Fokus Media UU RI No 14 tahun 2005 tentang guru dan dosen. Jakarta : Novindo Pustaka Mandiri


Exchangeability, prediction and predictive modeling in Bayesian  statistics

Abstract: Prediction is a central problem in Statistics, and there is currently a renewed interest for the so-called predictive approach in Bayesian statistics. What is the latter about? One has to return on foundational concepts, which we do in this paper, moving from the role of exchangeability and reviewing forms of partial exchangeability for more structured data, with the aim of discussing their use and implications in Bayesian statistics. There we show the underlying concept that, in Bayesian statistics, a predictive rule is meant as a learning rule - how one conveys past information to information on future events. This concept has implications on the use of exchangeability and generally invests all statistical problems, also in inference. It applies to classic contexts and to less explored situations, such as the use of predictive algorithms that can be read as Bayesian learning rules. The paper offers a historical overview, but also includes a few new results, presents some recent developments and poses some open questions.


Hidden Traveling Waves bind Working Memory Variables in Recurrent Neural  Networks

Abstract: Traveling waves are a fundamental phenomenon in the brain, playing a crucial role in short-term information storage. In this study, we leverage the concept of traveling wave dynamics within a neural lattice to formulate a theoretical model of neural working memory, study its properties, and its real world implications in AI. The proposed model diverges from traditional approaches, which assume information storage in static, register-like locations updated by interference. Instead, the model stores data as waves that is updated by the wave's boundary conditions. We rigorously examine the model's capabilities in representing and learning state histories, which are vital for learning history-dependent dynamical systems. The findings reveal that the model reliably stores external information and enhances the learning process by addressing the diminishing gradient problem. To understand the model's real-world applicability, we explore two cases: linear boundary condition and non-linear, self-attention-driven boundary condition. The experiments reveal that the linear scenario is effectively learned by Recurrent Neural Networks (RNNs) through backpropagation when modeling history-dependent dynamical systems. Conversely, the non-linear scenario parallels the autoregressive loop of an attention-only transformer. Collectively, our findings suggest the broader relevance of traveling waves in AI and its potential in advancing neural network architectures.


Align before Attend: Aligning Visual and Textual Features for Multimodal  Hateful Content Detection

Abstract: Multimodal hateful content detection is a challenging task that requires complex reasoning across visual and textual modalities. Therefore, creating a meaningful multimodal representation that effectively captures the interplay between visual and textual features through intermediate fusion is critical. Conventional fusion techniques are unable to attend to the modality-specific features effectively. Moreover, most studies exclusively concentrated on English and overlooked other low-resource languages. This paper proposes a context-aware attention framework for multimodal hateful content detection and assesses it for both English and non-English languages. The proposed approach incorporates an attention layer to meaningfully align the visual and textual features. This alignment enables selective focus on modality-specific features before fusing them. We evaluate the proposed approach on two benchmark hateful meme datasets, viz. MUTE (Bengali code-mixed) and MultiOFF (English). Evaluation results demonstrate our proposed approach's effectiveness with F1-scores of $69.7$% and $70.3$% for the MUTE and MultiOFF datasets. The scores show approximately $2.5$% and $3.2$% performance improvement over the state-of-the-art systems on these datasets. Our implementation is available at https://github.com/eftekhar-hossain/Bengali-Hateful-Memes.


Identification with Posterior-Separable Information Costs

Abstract: I provide a model of rational inattention with heterogeneity and prove it is observationally equivalent to a state-dependent stochastic choice model subject to attention costs. I demonstrate that additive separability of unobservable heterogeneity, together with an independence assumption, suffice for the empirical model to admit a representative agent. Using conditional probabilities, I show how to identify: how covariates affect the desirability of goods, (a measure of) welfare, factual changes in welfare, and bounds on counterfactual market shares.


BLOCKCHAIN IN U.S. ACCOUNTING: A REVIEW: ASSESSING ITS TRANSFORMATIVE POTENTIAL FOR ENHANCING TRANSPARENCY AND INTEGRITY

Abstract:
This study critically examines the transformative potential of blockchain technology in enhancing transparency ansd integrity within U.S. accounting practices. Employing a systematic literature review and content analysis methodology, the research draws on a range of peer-reviewed articles, industry reports, and authoritative databases. The study's objectives include understanding the evolution of blockchain in accounting, assessing its impact on transparency and integrity, identifying challenges in adoption, and exploring future trends and stakeholder implications. Key findings reveal that blockchain acts as a catalyst for transparency and integrity in accounting, offering a decentralized and immutable record-keeping system that significantly reduces the likelihood of fraud and errors. The technology's integration into accounting practices is evolving, indicating a future of automated, real-time auditing processes and enhanced financial reporting. However, challenges such as regulatory ambiguity, technical complexities, and scalability issues persist, necessitating strategic solutions. The study recommends that industry leaders and policymakers proactively embrace blockchain technology through education, regulatory framework development, and collaboration between technology developers and the accounting industry. Standard-setting bodies are also encouraged to update accounting standards in light of blockchain's implications. Future research directions include exploring the long-term impact of blockchain on various accounting aspects, empirical assessments of its real-world impact, and strategies to overcome adoption barriers. In conclusion, the study highlights blockchain's significant role in revolutionizing accounting practices, emphasizing the need for continued exploration and strategic integration of this technology in the accounting sector.
 Keywords: Blockchain Technology, Accounting Transparency, Regulatory Frameworks, Financial Integrity.


Struggles and Strategies in Understanding Information Visualizations

Abstract: While the visualization community is increasingly aware that people often find visualizations difficult to understand, there is less information about what we need to do to create comprehensible visualizations. To help visualization creators and designers improve their visualizations, we need to better understand what kind of support people are looking for in their sensemaking process. Empirical studies are needed to tease apart the details of what makes the process of understanding difficult for visualization viewers. We conducted a qualitative study with 14 participants, observing them as they described how they were trying to make sense of 20 information visualizations. We identified the challenges participants faced throughout their sensemaking process and the strategies they employed to help themselves in overcoming the challenges. Our findings show how details and nuances within visualizations can impact comprehensibility and offer research suggestions to help us move toward more understandable visualizations.


Knowledge Sharing Behavior Shaped by Organizational Climate, Social Network, Perception, and Achievement Motivation

Abstract: To date, knowledge sharing commonly discussed just as a part of knowledge management. However, knowledge sharing as a behavior is imperative to be investigated and managed carefully at a firm level. This study was aimed to examine employee knowledge sharing behavior could be shaped by organizational climate, social network, individual perception on knowledge itself, and achievement motivation. Data collected from questionnaires filled by 121 respondents who work at a multinational Japanese company operating in Indonesia (PT MII). Path analysis was used to evaluate causal model among variables. Result indicates that achievement motivation acts well as an intervening variable influences knowledge sharing behavior. Organization climate has a strongest path among variables with good practices performed by a supervisor; corporate policy to deploy knowledge; a knowledgeable feedback and corrective actions; and an assignment with a considerable time to deadline. Employees perceived that sharing behavior encouraged first in the network, promoted and managed by the organization, and lastly as a personal initiative. Motivated employees sharing their knowledge for self’s empowered, a sense of self merit, and share first at the situation of immediate and necessity. Idea for a better work more often shared than to find solutions for instant actions, tacit knowledge tends to be shared when requested, self’s refined knowledge shared individually rather than at a workgroup, and still there are some reluctances of senior owners to contribute their precious tacit knowledge at once. Keywords: Knowledge Sharing Behavior, Organizational Climate, Social Network, Individual Perception, Achievement Motivation


Wasserstein GAN-based architecture to generate collaborative filtering synthetic datasets

Abstract: Abstract Currently, generative applications are reshaping different fields, such as art, computer vision, speech processing, and natural language. The computer science personalization area is increasingly relevant since large companies such as Spotify, Netflix, TripAdvisor, Amazon, and Google use recommender systems. Then, it is rational to expect that generative learning will increasingly be used to improve current recommender systems. In this paper, a method is proposed to generate synthetic recommender system datasets that can be used to test the recommendation performance and accuracy of a company on different simulated scenarios, such as large increases in their dataset sizes, number of users, or number of items. Specifically, an improvement in the state-of-the-art method is proposed by applying the Wasserstein concept to the generative adversarial network for recommender systems (GANRS) seminal method to generate synthetic datasets. The results show that our proposed method reduces the mode collapse, increases the sizes of the synthetic datasets, improves their ratings distributions, and maintains the potential to choose the desired number of users, number of items, and starting size of the dataset. Both the baseline GANRS and the proposed Wasserstein-based WGANRS deep learning architectures generate fake profiles from dense, short, and continuous embeddings in the latent space instead of the sparse, large, and discrete raw samples that previous GAN models used as a source. To enable reproducibility, the Python and Keras codes are provided in open repositories along with the synthetic datasets generated to test the proposed architecture ( https://github.com/jesusbobadilla/ganrs.git ). Graphical abstract


Moving beyond the challenges of learning through technologies: The current status of ICT integration in South African schools

Abstract: The purpose of this study was to understand the current status of ICT integration in South African schools. Yet, this study focused on the Tshwane West District schools in Gauteng Province, South Africa. This study employed a descriptive research design to understand the aforesaid phenomenon. Data was collected through interviews and open-ended questionnaires with 10 teachers and one curriculum specialist. This study employed thematic analysis as a qualitative research method. Furthermore, Roger's Diffusion of Innovations is used as a framework. Despite the existence of policies, this study revealed that the integration of ICT remained a significant challenge for several instructors in South Africa. The most interesting part was that schools had improved on most ICT tools to enable access to learning with technologies, but to a greater extent, insufficient internet coverage in classrooms made it impossible for most teachers to appreciate the benefits that come with using tools such as laptops, whiteboards, and online videos. This research proposes the installation of pocket Wi-Fi devices in every classroom in schools to provide comprehensive internet access. This will facilitate interaction between instructors and students through up-to-date learning management systems subscribed to by schools, as well as films. This will promote a digitally viable learning environment that promotes creative opportunities for learners.


Deep Learning Strategies for 5G and LTE Spectrum Sensing Communication

Abstract: The idea of 5G innovations is a prevalent instrument for the pace of transmission and gathering of data and the accessibility of permitting all over the place. Notwithstanding that the fifth era convergences will embrace a keen procedure for the data transmission process. Sending and getting signals work in high coordination in 5G networks, since this innovation arranges flexible, geostationary earthbound correspondence with other medium and little circuit correspondences with short steering in straight correspondences, and the correspondence incorporates signal processing as well as way finding. In this study the responsiveness improvement of the correspondence range will be tested by applying blended deep learning methods, in which the data cross-over will be diminished with the upgraded smart control. Utilizing blended deep learning methods, this study exhibits the huge difficulties presented by 5G transmissions in keenly detecting the LTE signal range and different data in 5G remote sensor networks. Way obstructions are recognized as the essential hindrance. The states of the correspondence framework ought to be considered while plotting the network and sensors for the fifth era.


Building Trust Takes Time: Limits to Arbitrage for Blockchain-Based Assets

Abstract: Abstract A blockchain replaces central counterparties with time-consuming consensus protocols to record the transfer of ownership. This settlement latency slows cross-exchange trading, exposing arbitrageurs to price risk. Off-chain settlement, instead, exposes arbitrageurs to costly default risk. We show with Bitcoin network and order book data that cross-exchange price differences coincide with periods of high settlement latency, asset flows chase arbitrage opportunities, and price differences across exchanges with low default risk are smaller. Blockchain-based trading thus faces a dilemma: Reliable consensus protocols require time-consuming settlement latency, leading to arbitrage limits. Circumventing such arbitrage costs is possible &lt;italic&gt;only&lt;/italic&gt; by reinstalling trusted intermediation, which mitigates default risk.


Examining Pathological Bias in a Generative Adversarial Network  Discriminator: A Case Study on a StyleGAN3 Model

Abstract: Generative adversarial networks generate photorealistic faces that are often indistinguishable by humans from real faces. We find that the discriminator in the pre-trained StyleGAN3 model, a popular GAN network, systematically stratifies scores by both image- and face-level qualities and that this disproportionately affects images across gender, race, and other categories. We examine the discriminator's bias for color and luminance across axes perceived race and gender; we then examine axes common in research on stereotyping in social psychology.


NutePrune: Efficient Progressive Pruning with Numerous Teachers for  Large Language Models

Abstract: The considerable size of Large Language Models (LLMs) presents notable deployment challenges, particularly on resource-constrained hardware. Structured pruning, offers an effective means to compress LLMs, thereby reducing storage costs and enhancing inference speed for more efficient utilization. In this work, we study data-efficient and resource-efficient structure pruning methods to obtain smaller yet still powerful models. Knowledge Distillation is well-suited for pruning, as the intact model can serve as an excellent teacher for pruned students. However, it becomes challenging in the context of LLMs due to memory constraints. To address this, we propose an efficient progressive Numerous-teacher pruning method (NutePrune). NutePrune mitigates excessive memory costs by loading only one intact model and integrating it with various masks and LoRA modules, enabling it to seamlessly switch between teacher and student roles. This approach allows us to leverage numerous teachers with varying capacities to progressively guide the pruned model, enhancing overall performance. Extensive experiments across various tasks demonstrate the effectiveness of NutePrune. In LLaMA-7B zero-shot experiments, NutePrune retains 97.17% of the performance of the original model at 20% sparsity and 95.07% at 25% sparsity.


Sequential Recommendation on Temporal Proximities with Contrastive  Learning and Self-Attention

Abstract: Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Although traditional deep-learning-based models and modern transformer-based models in previous studies capture unidirectional and bidirectional patterns within user-item interactions, the importance of temporal contexts, such as individual behavioral and societal trend patterns, remains underexplored. Notably, recent models often neglect similarities in users' actions that occur implicitly among users during analogous timeframes-a concept we term vertical temporal proximity. These models primarily adapt the self-attention mechanisms of the transformer to consider the temporal context in individual user actions. Meanwhile, this adaptation still remains limited in considering the horizontal temporal proximity within item interactions, like distinguishing between subsequent item purchases within a week versus a month. To address these gaps, we propose a sequential recommendation model called TemProxRec, which includes contrastive learning and self-attention methods to consider temporal proximities both across and within user-item interactions. The proposed contrastive learning method learns representations of items selected in close temporal periods across different users to be close. Simultaneously, the proposed self-attention mechanism encodes temporal and positional contexts in a user sequence using both absolute and relative embeddings. This way, our TemProxRec accurately predicts the relevant items based on the user-item interactions within a specific timeframe. We validate this work through comprehensive experiments on TemProxRec, consistently outperforming existing models on benchmark datasets as well as showing the significance of considering the vertical and horizontal temporal proximities into sequential recommendation.


Online Food Ordering System

Abstract: Online Food Ordering App is an application designed primarily for use in the food delivery industry. This system will allow hotels, restaurants and food courts to increase the scope of business by reducing the labour cost involved. The system also allows to quickly and easily manage an online menu which customers can browse and use to place orders with just few clicks. Admin employees then use these orders through an easy to navigate graphical interface for efficient processing. The online food ordering system provides convenience for the customers. This system increases the takeaway of foods than visitors. The online food ordering app set up menu online and the customers easily places the order with a simple click. Also, with a food menu online you can easily track the orders, maintain customer s database and improve your food delivery service. The proposed system leverages the latest in mobile technology to bridge the gap between restaurants, food enthusiasts, and delivery services. It offers a user-friendly interface that allows users to browse a wide range of cuisine options, explore restaurant menus, customize orders, and place them seamlessly. The system is powered by trending Artificial Intelligence Technologies like Machine Learning, Recommendation Engine, Route optimization for delivery and Image recognition for meal searching. This system will be a culinary companion that will connect users to their favourite food experiences without leaving the comfort of their homes or offices. Its dynamic features make it an indispensable tool for food enthusiasts and a valuable asset for the restaurant industry, creating a win-win situation for all stakeholders involved in the modern gastronomy ecosystem.


Quickest Detection of False Data Injection Attack in Distributed Process  Tracking

Abstract: This paper addresses the problem of detecting false data injection (FDI) attacks in a distributed network without a fusion center, represented by a connected graph among multiple agent nodes. Each agent node is equipped with a sensor, and uses a Kalman consensus information filter (KCIF) to track a discrete time global process with linear dynamics and additive Gaussian noise. The state estimate of the global process at any sensor is computed from the local observation history and the information received by that agent node from its neighbors. At an unknown time, an attacker starts altering the local observation of one agent node. In the Bayesian setting where there is a known prior distribution of the attack beginning instant, we formulate a Bayesian quickest change detection (QCD) problem for FDI detection in order to minimize the mean detection delay subject to a false alarm probability constraint. While it is well-known that the optimal Bayesian QCD rule involves checking the Shriyaev's statistic against a threshold, we demonstrate how to compute the Shriyaev's statistic at each node in a recursive fashion given our non-i.i.d. observations. Next, we consider non-Bayesian QCD where the attack begins at an arbitrary and unknown time, and the detector seeks to minimize the worst case detection delay subject to a constraint on the mean time to false alarm and probability of misidentification. We use the multiple hypothesis sequential probability ratio test for attack detection and identification at each sensor. For unknown attack strategy, we use the window-limited generalized likelihood ratio (WL-GLR) algorithm to solve the QCD problem. Numerical results demonstrate the performances and trade-offs of the proposed algorithms.


System-level Impact of Non-Ideal Program-Time of Charge Trap Flash (CTF)  on Deep Neural Network

Abstract: Learning of deep neural networks (DNN) using Resistive Processing Unit (RPU) architecture is energy-efficient as it utilizes dedicated neuromorphic hardware and stochastic computation of weight updates for in-memory computing. Charge Trap Flash (CTF) devices can implement RPU-based weight updates in DNNs. However, prior work has shown that the weight updates (V_T) in CTF-based RPU are impacted by the non-ideal program time of CTF. The non-ideal program time is affected by two factors of CTF. Firstly, the effects of the number of input pulses (N) or pulse width (pw), and secondly, the gap between successive update pulses (t_gap) used for the stochastic computation of weight updates. Therefore, the impact of this non-ideal program time must be studied for neural network training simulations. In this study, Firstly, we propose a pulse-train design compensation technique to reduce the total error caused by non-ideal program time of CTF and stochastic variance of a network. Secondly, we simulate RPU-based DNN with non-ideal program time of CTF on MNIST and Fashion-MNIST datasets. We find that for larger N (~1000), learning performance approaches the ideal (software-level) training level and, therefore, is not much impacted by the choice of t_gap used to implement RPU-based weight updates. However, for lower N (<500), learning performance depends on T_gap of the pulses. Finally, we also performed an ablation study to isolate the causal factor of the improved learning performance. We conclude that the lower noise level in the weight updates is the most likely significant factor to improve the learning performance of DNN. Thus, our study attempts to compensate for the error caused by non-ideal program time and standardize the pulse length (N) and pulse gap (t_gap) specifications for CTF-based RPUs for accurate system-level on-chip training.


Using Geographic Information Systems as a Practical Way of Teaching Mathematics

Abstract: Geographic Information Systems (GIS) are programs that connect math, science, and social science concepts with many resources for teachers to provide and integrate activities with math and science. This paper reflects on proposals to merge Mathematics and Geography when teaching undergraduates from inter- and trans-disciplinary spatial models using mathematical modeling to enable spatial analysis and visualization through GIS—Geographic Information Systems. Mathematics Education incorporates facts from real-world reality with models that use new geotechnologies. GIS brings forward an approach to Geography with new technologies integrating it with Statistics and Computer Science. Thus, it describes some practical cases that are examples of geographical problems solved mathematically. As an example, a case study about COVID-19 infection in a city is presented. The results show that GIS can be an educational tool for teaching both Geography and Mathematics based on real problems. Integrating this tool enables active and constant participation of the students, which can make them realize the applicability of mathematics in solving real problems related to geographical space, is required by any school committed to education.


Text Summarization: Taking Advantage of Structural Syntax, Term Expansion and Process Refinement

Abstract: This chapter is dedicated to the development and application of an extractive summarization procedure and is a detailed account of the set of experiments performed to study the utility of applying a combined structural property of a text’s sentences and term expansion using WordNet along with a local thesaurus all combined and used in the selection of the most appropriate extractive text summarization in a particular document.  Sentences were tagged and normalized. Next, they were subjected to the well-known Longest Common Subsequence (LCS) algorithm for the selection of the most similar subset of sentences. Calculated similarity was based on LCS of pairs of sentences making up the document. A normalized score was calculated and used to rank the sentences. A selected top subset of the most similar sentences was then tokenized to produce a set of important keywords or terms. The produced terms were further expanded into two subsets using 1) WorldNet; and 2) a local electronic dictionary/thesaurus.  The three sets obtained (the original and the expanded two) were then re-cycled to further refine and expand the list of selected sentences from the original document. The process was repeated a number of times in order to find the best representative set of sentences. A final set of the top (best) sentences was selected as candidate sentences for summarization.  In order to verify the utility of the procedure, a number of experiments were conducted using an email corpus. The results were compared to those produced by human annotators as well as to those results produced using basic sentences similarity calculation method. Produced results were very encouraging and compared well to those of human annotators and Jacquard sentences similarity.


Lester: rotoscope animation through video object segmentation and  tracking

Abstract: This article introduces Lester, a novel method to automatically synthetise retro-style 2D animations from videos. The method approaches the challenge mainly as an object segmentation and tracking problem. Video frames are processed with the Segment Anything Model (SAM) and the resulting masks are tracked through subsequent frames with DeAOT, a method of hierarchical propagation for semi-supervised video object segmentation. The geometry of the masks' contours is simplified with the Douglas-Peucker algorithm. Finally, facial traits, pixelation and a basic shadow effect can be optionally added. The results show that the method exhibits an excellent temporal consistency and can correctly process videos with different poses and appearances, dynamic shots, partial shots and diverse backgrounds. The proposed method provides a more simple and deterministic approach than diffusion models based video-to-video translation pipelines, which suffer from temporal consistency problems and do not cope well with pixelated and schematic outputs. The method is also much most practical than techniques based on 3D human pose estimation, which require custom handcrafted 3D models and are very limited with respect to the type of scenes they can process.


A Quantum Approach to News Verification from the Perspective of a News  Aggregator

Abstract: In the dynamic landscape of digital information, the rise of misinformation and fake news presents a pressing challenge. This paper takes a completely new approach to verifying news, inspired by how quantum actors can reach agreement even when they are spatially spread out. We propose a radically new, to the best of our knowledge, algorithm that uses quantum ``entanglement'' (think of it as a special connection) to help news aggregators sniff out bad actors, whether they be other news sources or even fact-checkers trying to spread misinformation. This algorithm doesn't rely on quantum signatures, it just uses basic quantum technology we already have, in particular, special pairs of particles called ``EPR pairs'' that are much easier to create than other options. More complex entangled states are like juggling too many balls - they're hard to make and slow things down, especially when many players are involved. For instance, bigger, more complex states like ``GHZ states'' work for small groups, but they become messy with larger numbers. So, we stick with Bell states, the simplest form of entanglement, which are easy to generate no matter how many players are in the game. This means our algorithm is faster to set up, works for any number of participants, and is more practical for real-world use. Bonus points: it finishes in a fixed number of steps, regardless of how many players are involved, making it even more scalable. This new approach may lead to a powerful and efficient way to fight misinformation in the digital age, using the weird and wonderful world of quantum mechanics.


OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large  Language Models

Abstract: Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. This paper introduces OptiMUS, a Large Language Model (LLM)-based agent designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. OptiMUS can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve its model and code based on these evaluations. OptiMUS utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS outperforms existing state-of-the-art methods on easy datasets by more than $20\%$ and on hard datasets (including a new dataset, NLP4LP, released with this paper that features long and complex problems) by more than $30\%$.


Bridging the Empirical-Theoretical Gap in Neural Network Formal Language  Learning Using Minimum Description Length

Abstract: Neural networks offer good approximation to many tasks but consistently fail to reach perfect generalization, even when theoretical work shows that such perfect solutions can be expressed by certain architectures. Using the task of formal language learning, we focus on one simple formal language and show that the theoretically correct solution is in fact not an optimum of commonly used objectives -- even with regularization techniques that according to common wisdom should lead to simple weights and good generalization (L1, L2) or other meta-heuristics (early-stopping, dropout). However, replacing standard targets with the Minimum Description Length objective (MDL) results in the correct solution being an optimum.


Towards Safer Large Language Models through Machine Unlearning

Abstract: The rapid advancement of Large Language Models (LLMs) has demonstrated their vast potential across various domains, attributed to their extensive pretraining knowledge and exceptional generalizability. However, LLMs often encounter challenges in generating harmful content when faced with problematic prompts. To address this problem, existing work attempted to implement a gradient ascent based approach to prevent LLMs from producing harmful output. While these methods can be effective, they frequently impact the model utility in responding to normal prompts. To address this gap, we introduce Selective Knowledge negation Unlearning (SKU), a novel unlearning framework for LLMs, designed to eliminate harmful knowledge while preserving utility on normal prompts. Specifically, SKU is consisted of two stages: harmful knowledge acquisition stage and knowledge negation stage. The first stage aims to identify and acquire harmful knowledge within the model, whereas the second is dedicated to remove this knowledge. SKU selectively isolates and removes harmful knowledge in model parameters, ensuring the model's performance remains robust on normal prompts. Our experiments conducted across various LLM architectures demonstrate that SKU identifies a good balance point between removing harmful information and preserving utility.


Few-Shot Object Detection with Sparse Context Transformers

Abstract: Few-shot detection is a major task in pattern recognition which seeks to localize objects using models trained with few labeled data. One of the mainstream few-shot methods is transfer learning which consists in pretraining a detection model in a source domain prior to its fine-tuning in a target domain. However, it is challenging for fine-tuned models to effectively identify new classes in the target domain, particularly when the underlying labeled training data are scarce. In this paper, we devise a novel sparse context transformer (SCT) that effectively leverages object knowledge in the source domain, and automatically learns a sparse context from only few training images in the target domain. As a result, it combines different relevant clues in order to enhance the discrimination power of the learned detectors and reduce class confusion. We evaluate the proposed method on two challenging few-shot object detection benchmarks, and empirical results show that the proposed method obtains competitive performance compared to the related state-of-the-art.


Self-Augmented In-Context Learning for Unsupervised Word Translation

Abstract: Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditional' mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.


Benchmarking federated strategies in Peer-to-Peer Federated learning for  biomedical data

Abstract: The increasing requirements for data protection and privacy has attracted a huge research interest on distributed artificial intelligence and specifically on federated learning, an emerging machine learning approach that allows the construction of a model between several participants who hold their own private data. In the initial proposal of federated learning the architecture was centralised and the aggregation was done with federated averaging, meaning that a central server will orchestrate the federation using the most straightforward averaging strategy. This research is focused on testing different federated strategies in a peer-to-peer environment. The authors propose various aggregation strategies for federated learning, including weighted averaging aggregation, using different factors and strategies based on participant contribution. The strategies are tested with varying data sizes to identify the most robust ones. This research tests the strategies with several biomedical datasets and the results of the experiments show that the accuracy-based weighted average outperforms the classical federated averaging method.


Zero-Shot Reasoning: Personalized Content Generation Without the Cold  Start Problem

Abstract: Procedural content generation uses algorithmic techniques to create large amounts of new content for games at much lower production costs. In newer approaches, procedural content generation utilizes machine learning. However, these methods usually require expensive collection of large amounts of data, as well as the development and training of fairly complex learning models, which can be both extremely time-consuming and expensive. The core of our research is to explore whether we can lower the barrier to the use of personalized procedural content generation through a more practical and generalizable approach with large language models. Matching game content with player preferences benefits both players, who enjoy the game more, and developers, who increasingly depend on players enjoying the game before being able to monetize it. Therefore, this paper presents a novel approach to achieving personalization by using large language models to propose levels based on the gameplay data continuously collected from individual players. We compared the levels generated using our approach with levels generated with more traditional procedural generation techniques. Our easily reproducible method has proven viable in a production setting and outperformed levels generated by traditional methods in the probability that a player will not quit the game mid-level.


A privacy-preserving, distributed and cooperative FCM-based learning  approach for Cancer Research

Abstract: Distributed Artificial Intelligence is attracting interest day by day. In this paper, the authors introduce an innovative methodology for distributed learning of Particle Swarm Optimization-based Fuzzy Cognitive Maps in a privacy-preserving way. The authors design a training scheme for collaborative FCM learning that offers data privacy compliant with the current regulation. This method is applied to a cancer detection problem, proving that the performance of the model is improved by the Federated Learning process, and obtaining similar results to the ones that can be found in the literature.


A chaotic maps-based privacy-preserving distributed deep learning for  incomplete and Non-IID datasets

Abstract: Federated Learning is a machine learning approach that enables the training of a deep learning model among several participants with sensitive data that wish to share their own knowledge without compromising the privacy of their data. In this research, the authors employ a secured Federated Learning method with an additional layer of privacy and proposes a method for addressing the non-IID challenge. Moreover, differential privacy is compared with chaotic-based encryption as layer of privacy. The experimental approach assesses the performance of the federated deep learning model with differential privacy using both IID and non-IID data. In each experiment, the Federated Learning process improves the average performance metrics of the deep neural network, even in the case of non-IID data.


Strategic Vote Timing in Online Elections With Public Tallies

Abstract: We study the effect of public tallies on online elections, in a setting where voting is costly and voters are allowed to strategically time their votes. The strategic importance of choosing \emph{when} to vote arises when votes are public, such as in online event scheduling polls (e.\,g., Doodle), or in blockchain governance mechanisms. In particular, there is a tension between voting early to influence future votes and waiting to observe interim results and avoid voting costs if the outcome has already been decided. Our study draws on empirical findings showing that ``temporal'' bandwagon effects occur when interim results are revealed to the electorate: late voters are more likely to vote for leading candidates. To capture this phenomenon, we analyze a novel model where the electorate consists of informed voters who have a preferred candidate, and uninformed swing voters who can be swayed according to the interim outcome at the time of voting. In our main results, we prove the existence of equilibria where both early and late voting occur with a positive probability, and we characterize conditions that lead to the appearance of ``last minute'' voting behavior, where all informed voters vote late.


Mixture of Experts for Network Optimization: A Large Language  Model-enabled Approach

Abstract: Optimizing various wireless user tasks poses a significant challenge for networking systems because of the expanding range of user requirements. Despite advancements in Deep Reinforcement Learning (DRL), the need for customized optimization tasks for individual users complicates developing and applying numerous DRL models, leading to substantial computation resource and energy consumption and can lead to inconsistent outcomes. To address this issue, we propose a novel approach utilizing a Mixture of Experts (MoE) framework, augmented with Large Language Models (LLMs), to analyze user objectives and constraints effectively, select specialized DRL experts, and weigh each decision from the participating experts. Specifically, we develop a gate network to oversee the expert models, allowing a collective of experts to tackle a wide array of new tasks. Furthermore, we innovatively substitute the traditional gate network with an LLM, leveraging its advanced reasoning capabilities to manage expert model selection for joint decisions. Our proposed method reduces the need to train new DRL models for each unique optimization problem, decreasing energy consumption and AI model implementation costs. The LLM-enabled MoE approach is validated through a general maze navigation task and a specific network service provider utility maximization task, demonstrating its effectiveness and practical applicability in optimizing complex networking systems.


An Approximation Based Theory of Linear Regression

Abstract: The goal of this paper is to provide a theory linear regression based entirely on approximations. It will be argued that the standard linear regression model based theory whether frequentist or Bayesian has failed and that this failure is due to an 'assumed (revealed?) truth' (John Tukey) attitude to the models. This is reflected in the language of statistical inference which involves a concept of truth, for example efficiency, consistency and hypothesis testing. The motivation behind this paper was to remove the word `true' from the theory and practice of linear regression and to replace it by approximation. The approximations considered are the least squares approximations. An approximation is called valid if it contains no irrelevant covariates. This is operationalized using the concept of a Gaussian P-value which is the probability that pure Gaussian noise is better in term of least squares than the covariate. The precise definition given in the paper is intuitive and requires only four simple equations. Given this a valid approximation is one where all the Gaussian P-values are less than a threshold $p0$ specified by the statistician, in this paper with the default value 0.01. This approximations approach is not only much simpler it is overwhelmingly better than the standard model based approach. This will be demonstrated using six real data sets, four from high dimensional regression and two from vector autoregression. Both the simplicity and the superiority of Gaussian P-values derive from their universal exactness and validity. This is in complete contrast to standard F P-values which are valid only for carefully designed simulations. The paper contains excerpts from an unpublished paper by John Tukey entitled `Issues relevant to an honest account of data-based inference partially in the light of Laurie Davies's paper'.


Efficient Language Adaptive Pre-training: Extending State-of-the-Art  Large Language Models for Polish

Abstract: This study explores the potential of fine-tuning foundational English Large Language Models (LLMs) for generating Polish text. The first step involves Language Adaptive Pre-training (LAPT) on a high-quality dataset of 3.11 GB, consisting of 276 million Polish tokens. The LAPT is followed by additional fine-tuning aimed at solving nine KLEJ challenges. Our trained model Curie-7B-v1 not only generates Polish text with the lowest perplexity of 3.02 among decoder-based Polish models but also closely rivals the performance of the best Polish encoder-decoder models with a less than 2% gap on 8 out of 9 tasks. Curie-7B-v1 used approximately 2-3% of a typical dataset size to learn Polish. The LAPT was completed in less than five days using a consumer GPU, highlighting the method's efficiency. The proficiency of the model in Polish was significantly enhanced, demonstrating the viability of this approach for adding new languages to existing LLMs by training just 1.2% of its parameters. To contribute to the community's collaborative progress, the model has been released as open-source.


Improving the efficiency of GP-GOMEA for higher-arity operators

Abstract: Deploying machine learning models into sensitive domains in our society requires these models to be explainable. Genetic Programming (GP) can offer a way to evolve inherently interpretable expressions. GP-GOMEA is a form of GP that has been found particularly effective at evolving expressions that are accurate yet of limited size and, thus, promote interpretability. Despite this strength, a limitation of GP-GOMEA is template-based. This negatively affects its scalability regarding the arity of operators that can be used, since with increasing operator arity, an increasingly large part of the template tends to go unused. In this paper, we therefore propose two enhancements to GP-GOMEA: (i) semantic subtree inheritance, which performs additional variation steps that consider the semantic context of a subtree, and (ii) greedy child selection, which explicitly considers parts of the template that in standard GP-GOMEA remain unused. We compare different versions of GP-GOMEA regarding search enhancements on a set of continuous and discontinuous regression problems, with varying tree depths and operator sets. Experimental results show that both proposed search enhancements have a generally positive impact on the performance of GP-GOMEA, especially when the set of operators to choose from is large and contains higher-arity operators.


Recommendations for Baselines and Benchmarking Approximate Gaussian  Processes

Abstract: Gaussian processes (GPs) are a mature and widely-used component of the ML toolbox. One of their desirable qualities is automatic hyperparameter selection, which allows for training without user intervention. However, in many realistic settings, approximations are typically needed, which typically do require tuning. We argue that this requirement for tuning complicates evaluation, which has led to a lack of a clear recommendations on which method should be used in which situation. To address this, we make recommendations for comparing GP approximations based on a specification of what a user should expect from a method. In addition, we develop a training procedure for the variational method of Titsias [2009] that leaves no choices to the user, and show that this is a strong baseline that meets our specification. We conclude that benchmarking according to our suggestions gives a clearer view of the current state of the field, and uncovers problems that are still open that future papers should address.


Beyond Kalman Filters: Deep Learning-Based Filters for Improved Object  Tracking

Abstract: Traditional tracking-by-detection systems typically employ Kalman filters (KF) for state estimation. However, the KF requires domain-specific design choices and it is ill-suited to handling non-linear motion patterns. To address these limitations, we propose two innovative data-driven filtering methods. Our first method employs a Bayesian filter with a trainable motion model to predict an object's future location and combines its predictions with observations gained from an object detector to enhance bounding box prediction accuracy. Moreover, it dispenses with most domain-specific design choices characteristic of the KF. The second method, an end-to-end trainable filter, goes a step further by learning to correct detector errors, further minimizing the need for domain expertise. Additionally, we introduce a range of motion model architectures based on Recurrent Neural Networks, Neural Ordinary Differential Equations, and Conditional Neural Processes, that are combined with the proposed filtering methods. Our extensive evaluation across multiple datasets demonstrates that our proposed filters outperform the traditional KF in object tracking, especially in the case of non-linear motion patterns -- the use case our filters are best suited to. We also conduct noise robustness analysis of our filters with convincing positive results. We further propose a new cost function for associating observations with tracks. Our tracker, which incorporates this new association cost with our proposed filters, outperforms the conventional SORT method and other motion-based trackers in multi-object tracking according to multiple metrics on motion-rich DanceTrack and SportsMOT datasets.


Revisiting Recurrent Reinforcement Learning with Memory Monoids

Abstract: In RL, memory models such as RNNs and transformers address Partially Observable Markov Decision Processes (POMDPs) by mapping trajectories to latent Markov states. Neither model scales particularly well to long sequences, especially compared to an emerging class of memory models sometimes called linear recurrent models. We discover that the recurrent update of these models is a monoid, leading us to formally define a novel memory monoid framework. We revisit the traditional approach to batching in recurrent RL, highlighting both theoretical and empirical deficiencies. Leveraging the properties of memory monoids, we propose a new batching method that improves sample efficiency, increases the return, and simplifies the implementation of recurrent loss functions in RL.


Expressivity of parameterized quantum circuits for generative modeling  of continuous multivariate distributions

Abstract: Parameterized quantum circuits have been extensively used as the basis for machine learning models in regression, classification, and generative tasks. For supervised learning their expressivity has been thoroughly investigated and several universality properties have been proven. However, in the case of quantum generative modeling, the situation is less clear, especially when the task is to model distributions over continuous variables. In this work, we focus on expectation value sampling-based models; models where random variables are sampled classically, encoded with a parametrized quantum circuit, and the expectation value of fixed observables is measured and returned as a sample. We prove the universality of such variational quantum algorithms for the generation of multivariate distributions. Additionally, we provide a detailed analysis of these models, including fundamental upper bounds on the dimensionality of the distributions these models can represent. We further present a tight trade-off result connecting the needed number of measurements and qubit numbers in order to have universality for a desired dimension of output distribution within an error tolerance. Finally we also show that the data encoding strategy relates to the so-called polynomial chaos expansion, which is an analog of the Fourier expansion. Our results may help guide the design of future quantum circuits in generative modeling tasks.


Predictors from causal features do not generalize better to new domains

Abstract: We study how well machine learning models trained on causal features generalize across domains. We consider 16 prediction tasks on tabular datasets covering applications in health, employment, education, social benefits, and politics. Each dataset comes with multiple domains, allowing us to test how well a model trained in one domain performs in another. For each prediction task, we select features that have a causal influence on the target of prediction. Our goal is to test the hypothesis that models trained on causal features generalize better across domains. Without exception, we find that predictors using all available features, regardless of causality, have better in-domain and out-of-domain accuracy than predictors using causal features. Moreover, even the absolute drop in accuracy from one domain to the other is no better for causal predictors than for models that use all features. If the goal is to generalize to new domains, practitioners might as well train the best possible model on all available features.