Notes on zeta ratio stabilization

Abstract: This semi-expository note clarifies the extent to which recent ideas in homological stability can resolve the Ratios Conjecture over $\mathbb{F}_q(t)$. For large fixed $q$, a uniform power saving at distance $\ge q^{-\delta}$ from the critical line is possible. This implies cancellation-beyond-GRH in arbitrarily large ranges of moduli relative to the family of $L$-functions. It has applications to the statistics of low-lying zeros.


Structured World Modeling via Semantic Vector Quantization

Abstract: Neural discrete representations are crucial components of modern neural networks. However, their main limitation is that the primary strategies such as VQ-VAE can only provide representations at the patch level. Therefore, one of the main goals of representation learning, acquiring structured, semantic, and compositional abstractions such as the color and shape of an object, remains elusive. In this paper, we present the first approach to semantic neural discrete representation learning. The proposed model, called Semantic Vector-Quantized Variational Autoencoder (SVQ), leverages recent advances in unsupervised object-centric learning to address this limitation. Specifically, we observe that a simple approach quantizing at the object level poses a significant challenge and propose constructing scene representations hierarchically, from low-level discrete concept schemas to object representations. Additionally, we suggest a novel method for structured semantic world modeling by training a prior over these representations, enabling the ability to generate images by sampling the semantic properties of the objects in the scene. In experiments on various 2D and 3D object-centric datasets, we find that our model achieves superior generation performance compared to non-semantic vector quantization methods such as VQ-VAE and previous object-centric generative models. Furthermore, we find that the semantic discrete representations can solve downstream scene understanding tasks that require reasoning about the properties of different objects in the scene.


Conditional Normalizing Flows for Active Learning of Coarse-Grained  Molecular Representations

Abstract: Efficient sampling of the Boltzmann distribution of molecular systems is a long-standing challenge. Recently, instead of generating long molecular dynamics simulations, generative machine learning methods such as normalizing flows have been used to learn the Boltzmann distribution directly, without samples. However, this approach is susceptible to mode collapse and thus often does not explore the full configurational space. In this work, we address this challenge by separating the problem into two levels, the fine-grained and coarse-grained degrees of freedom. A normalizing flow conditioned on the coarse-grained space yields a probabilistic connection between the two levels. To explore the configurational space, we employ coarse-grained simulations with active learning which allows us to update the flow and make all-atom potential energy evaluations only when necessary. Using alanine dipeptide as an example, we show that our methods obtain a speedup to molecular dynamics simulations of approximately 15.9 to 216.2 compared to the speedup of 4.5 of the current state-of-the-art machine learning approach.


Few-Shot Class-Incremental Learning with Prior Knowledge

Abstract: To tackle the issues of catastrophic forgetting and overfitting in few-shot class-incremental learning (FSCIL), previous work has primarily concentrated on preserving the memory of old knowledge during the incremental phase. The role of pre-trained model in shaping the effectiveness of incremental learning is frequently underestimated in these studies. Therefore, to enhance the generalization ability of the pre-trained model, we propose Learning with Prior Knowledge (LwPK) by introducing nearly free prior knowledge from a few unlabeled data of subsequent incremental classes. We cluster unlabeled incremental class samples to produce pseudo-labels, then jointly train these with labeled base class samples, effectively allocating embedding space for both old and new class data. Experimental results indicate that LwPK effectively enhances the model resilience against catastrophic forgetting, with theoretical analysis based on empirical risk minimization and class distance measurement corroborating its operational principles. The source code of LwPK is publicly available at: \url{https://github.com/StevenJ308/LwPK}.


Beyond the Request: Harnessing HTTP Response Headers for Cross-Browser  Web Tracker Classification in an Imbalanced Setting

Abstract: The World Wide Web's connectivity is greatly attributed to the HTTP protocol, with HTTP messages offering informative header fields that appeal to disciplines like web security and privacy, especially concerning web tracking. Despite existing research employing HTTP/S request messages to identify web trackers, HTTP/S response headers are often overlooked. This study endeavors to design effective machine learning classifiers for web tracker detection using HTTP/S response headers. Data from the Chrome, Firefox, and Brave browsers, obtained through the traffic monitoring browser extension T.EX, serves as our data set. Eleven supervised models were trained on Chrome data and tested across all browsers. The results demonstrated high accuracy, F1-score, precision, recall, and minimal log-loss error for Chrome and Firefox, but subpar performance on Brave, potentially due to its distinct data distribution and feature set. The research suggests that these classifiers are viable for detecting web trackers in Chrome and Firefox. However, real-world application testing remains pending, and the distinction between tracker types and broader label sources could be explored in future studies.


Flexible Variational Information Bottleneck: Achieving Diverse  Compression with a Single Training

Abstract: Information Bottleneck (IB) is a widely used framework that enables the extraction of information related to a target random variable from a source random variable. In the objective function, IB controls the trade-off between data compression and predictiveness through the Lagrange multiplier $\beta$. Traditionally, to find the trade-off to be learned, IB requires a search for $\beta$ through multiple training cycles, which is computationally expensive. In this study, we introduce Flexible Variational Information Bottleneck (FVIB), an innovative framework for classification task that can obtain optimal models for all values of $\beta$ with single, computationally efficient training. We theoretically demonstrate that across all values of reasonable $\beta$, FVIB can simultaneously maximize an approximation of the objective function for Variational Information Bottleneck (VIB), the conventional IB method. Then we empirically show that FVIB can learn the VIB objective as effectively as VIB. Furthermore, in terms of calibration performance, FVIB outperforms other IB and calibration methods by enabling continuous optimization of $\beta$. Our codes are available at https://github.com/sotakudo/fvib.


Taming Uncertainty in Sparse-view Generalizable NeRF via Indirect  Diffusion Guidance

Abstract: Neural Radiance Fields (NeRF) have demonstrated effectiveness in synthesizing novel views. However, their reliance on dense inputs and scene-specific optimization has limited their broader applicability. Generalizable NeRFs (Gen-NeRF), while intended to address this, often produce blurring artifacts in unobserved regions with sparse inputs, which are full of uncertainty. In this paper, we aim to diminish the uncertainty in Gen-NeRF for plausible renderings. We assume that NeRF's inability to effectively mitigate this uncertainty stems from its inherent lack of generative capacity. Therefore, we innovatively propose an Indirect Diffusion-guided NeRF framework, termed ID-NeRF, to address this uncertainty from a generative perspective by leveraging a distilled diffusion prior as guidance. Specifically, to avoid model confusion caused by directly regularizing with inconsistent samplings as in previous methods, our approach introduces a strategy to indirectly inject the inherently missing imagination into the learned implicit function through a diffusion-guided latent space. Empirical evaluation across various benchmarks demonstrates the superior performance of our approach in handling uncertainty with sparse inputs.


Enumeration of linear codes with different hulls

Abstract: The hull of a linear code $C$ is the intersection of $C$ with its dual code. We present and analyze the number of linear $q$-ary codes of the same length and dimension but with different dimensions for their hulls. We prove that for given dimension $k$ and length $n\ge 2k$ the number of all $[n,k]_q$ linear codes with hull dimension $l$ decreases as $l$ increases. We also present classification results for binary and ternary linear codes with trivial hulls (LCD and self-orthogonal) for some values of the length $n$ and dimension $k$, comparing the obtained numbers with the number of all linear codes for the given $n$ and $k$.


Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field  Dynamics on the Attention Landscape

Abstract: Large language models based on the Transformer architecture have demonstrated impressive capabilities to learn in context. However, existing theoretical studies on how this phenomenon arises are limited to the dynamics of a single layer of attention trained on linear regression tasks. In this paper, we study the optimization of a Transformer consisting of a fully connected layer followed by a linear attention layer. The MLP acts as a common nonlinear representation or feature map, greatly enhancing the power of in-context learning. We prove in the mean-field and two-timescale limit that the infinite-dimensional loss landscape for the distribution of parameters, while highly nonconvex, becomes quite benign. We also analyze the second-order stability of mean-field dynamics and show that Wasserstein gradient flow almost always avoids saddle points. Furthermore, we establish novel methods for obtaining concrete improvement rates both away from and near critical points. This represents the first saddle point analysis of mean-field dynamics in general and the techniques are of independent interest.


Global Flows of Foliated Gravity-Matter Systems

Abstract: Asymptotic safety is a promising mechanism for obtaining a consistent and predictive quantum theory for gravity. The ADM formalism allows to introduce a (Euclidean) time-direction in this framework. It equips spacetime with a foliation structure by encoding the gravitational degrees of freedom in a lapse function, shift vector, and a metric measuring distances on the spatial slices. We use the Wetterich equation to study the renormalization group flow of the graviton 2-point function extracted from the spatial metric. The flow is driven by the 3- and 4-point vertices generated by the foliated Einstein-Hilbert action supplemented by minimally coupled scalar and vector fields. We derive bounds on the number of matter fields cast by asymptotic safety. Moreover, we show that the phase diagram obtained in the pure gravity case is qualitatively stable within these bounds. An intriguing feature is the presence of an IR-fixed point for the graviton mass which prevents the squared mass taking negative values. This feature persists for any number of matter fields and, in particular, also in situations where there is no suitable interacting fixed point rendering the theory asymptotically safe. Our work complements earlier studies of the subject by taking contributions from the matter fields into account.


Coherence for logicians

Abstract: This paper is addressed to logicians not familiar with category theory. It gives a new proof of coherence for symmetric monoidal closed categories, proven by Kelly and Mac Lane in early 1970s. We find this result of great importance for proof theory and it is formulated here in pure logical terminology free of categorial notions. Coherence is related to the generality conjecture in general proof theory and we hope that our formulation will make it closer to the proof-theoretical community.


Towards the new XAI: A Hypothesis-Driven Approach to Decision Support  Using Evidence

Abstract: Prior research on AI-assisted human decision-making has explored several different explainable AI (XAI) approaches. A recent paper has proposed a paradigm shift calling for hypothesis-driven XAI through a conceptual framework called evaluative AI that gives people evidence that supports or refutes hypotheses without necessarily giving a decision-aid recommendation. In this paper we describe and evaluate an approach for hypothesis-driven XAI based on the Weight of Evidence (WoE) framework, which generates both positive and negative evidence for a given hypothesis. Through human behavioural experiments, we show that our hypothesis-driven approach increases decision accuracy, reduces reliance compared to a recommendation-driven approach and an AI-explanation-only baseline, but with a small increase in under-reliance compared to the recommendation-driven approach. Further, we show that participants used our hypothesis-driven approach in a materially different way to the two baselines.


Can MLLMs Perform Text-to-Image In-Context Learning?

Abstract: The evolution from Large Language Models (LLMs) to Multimodal Large Language Models (MLLMs) has spurred research into extending In-Context Learning (ICL) to its multimodal counterpart. Existing such studies have primarily concentrated on image-to-text ICL. However, the Text-to-Image ICL (T2I-ICL), with its unique characteristics and potential applications, remains underexplored. To address this gap, we formally define the task of T2I-ICL and present CoBSAT, the first T2I-ICL benchmark dataset, encompassing ten tasks. Utilizing our dataset to benchmark six state-of-the-art MLLMs, we uncover considerable difficulties MLLMs encounter in solving T2I-ICL. We identify the primary challenges as the inherent complexity of multimodality and image generation. To overcome these challenges, we explore strategies like fine-tuning and Chain-of-Thought prompting, demonstrating notable improvements. Our code and dataset are available at \url{https://github.com/UW-Madison-Lee-Lab/CoBSAT}.


Bribe & Fork: Cheap Bribing Attacks via Forking Threat

Abstract: In this work, we reexamine the vulnerability of Payment Channel Networks (PCNs) to bribing attacks, where an adversary incentivizes blockchain miners to deliberately ignore a specific transaction to undermine the punishment mechanism of PCNs. While previous studies have posited a prohibitive cost for such attacks, we show that this cost may be dramatically reduced (to approximately \$125), thereby increasing the likelihood of these attacks. To this end, we introduce Bribe & Fork, a modified bribing attack that leverages the threat of a so-called feather fork which we analyze with a novel formal model for the mining game with forking. We empirically analyze historical data of some real-world blockchain implementations to evaluate the scale of this cost reduction. Our findings shed more light on the potential vulnerability of PCNs and highlight the need for robust solutions.


Video Semantic Communication with Major Object Extraction and Contextual  Video Encoding

Abstract: This paper studies an end-to-end video semantic communication system for massive communication. In the considered system, the transmitter must continuously send the video to the receiver to facilitate character reconstruction in immersive applications, such as interactive video conference. However, transmitting the original video information with substantial amounts of data poses a challenge to the limited wireless resources. To address this issue, we reduce the amount of data transmitted by making the transmitter extract and send the semantic information from the video, which refines the major object and the correlation of time and space in the video. Specifically, we first develop a video semantic communication system based on major object extraction (MOE) and contextual video encoding (CVE) to achieve efficient video transmission. Then, we design the MOE and CVE modules with convolutional neural network based motion estimation, contextual extraction and entropy coding. Simulation results show that compared to the traditional coding schemes, the proposed method can reduce the amount of transmitted data by up to 25% while increasing the peak signal-to-noise ratio (PSNR) of the reconstructed video by up to 14%.


FedMoE: Data-Level Personalization with Mixture of Experts for  Model-Heterogeneous Personalized Federated Learning

Abstract: Federated learning (FL) is widely employed for collaborative training on decentralized data but faces challenges like data, system, and model heterogeneity. This prompted the emergency of model-heterogeneous personalized federated learning (MHPFL). However, concerns persist regarding data and model privacy, model performance, communication, and computational costs in current MHPFL methods. To tackle these concerns, we propose a novel model-heterogeneous personalized Federated learning algorithm (FedMoE) with the Mixture of Experts (MoE), renowned for enhancing large language models (LLMs). It assigns a shared homogeneous small feature extractor and a local gating network for each client's local heterogeneous large model. (1) During local training, the local heterogeneous model's feature extractor acts as a local expert for personalized feature (representation) extraction, while the shared homogeneous small feature extractor serves as a global expert for generalized feature extraction. The local gating network produces personalized weights for extracted representations from both experts on each data sample. The three models form a local heterogeneous MoE. The weighted mixed representation fuses global generalized and local personalized features and is processed by the local heterogeneous large model's header with personalized prediction information for output. The MoE and prediction header are updated synchronously. (2) The trained local homogeneous small feature extractors are sent to the server for cross-client information fusion via aggregation. Briefly, FedMoE first enhances local model personalization at a fine-grained data level while supporting model heterogeneity.


Beyond the Answers: Reviewing the Rationality of Multiple Choice  Question Answering for the Evaluation of Large Language Models

Abstract: In the field of natural language processing (NLP), Large Language Models (LLMs) have precipitated a paradigm shift, markedly enhancing performance in natural language generation tasks. Despite these advancements, the comprehensive evaluation of LLMs remains an inevitable challenge for the community. Recently, the utilization of Multiple Choice Question Answering (MCQA) as a benchmark for LLMs has gained considerable traction. This study investigates the rationality of MCQA as an evaluation method for LLMs. If LLMs genuinely understand the semantics of questions, their performance should exhibit consistency across the varied configurations derived from the same questions. Contrary to this expectation, our empirical findings suggest a notable disparity in the consistency of LLM responses, which we define as REsponse VAriability Syndrome (REVAS) of the LLMs, indicating that current MCQA-based benchmarks may not adequately capture the true capabilities of LLMs, which underscores the need for more robust evaluation mechanisms in assessing the performance of LLMs.


Monotone, Bi-Lipschitz, and Polyak-\L{}ojasiewicz Networks

Abstract: This paper presents a new \emph{bi-Lipschitz} invertible neural network, the BiLipNet, which has the ability to control both its \emph{Lipschitzness} (output sensitivity to input perturbations) and \emph{inverse Lipschitzness} (input distinguishability from different outputs). The main contribution is a novel invertible residual layer with certified strong monotonicity and Lipschitzness, which we compose with orthogonal layers to build bi-Lipschitz networks. The certification is based on incremental quadratic constraints, which achieves much tighter bounds compared to spectral normalization. Moreover, we formulate the model inverse calculation as a three-operator splitting problem, for which fast algorithms are known. Based on the proposed bi-Lipschitz network, we introduce a new scalar-output network, the PLNet, which satisfies the Polyak-\L{}ojasiewicz condition. It can be applied to learn non-convex surrogate losses with favourable properties, e.g., a unique and efficiently-computable global minimum.


Simulator-Free Visual Domain Randomization via Video Games

Abstract: Domain randomization is an effective computer vision technique for improving transferability of vision models across visually distinct domains exhibiting similar content. Existing approaches, however, rely extensively on tweaking complex and specialized simulation engines that are difficult to construct, subsequently affecting their feasibility and scalability. This paper introduces BehAVE, a video understanding framework that uniquely leverages the plethora of existing commercial video games for domain randomization, without requiring access to their simulation engines. Under BehAVE (1) the inherent rich visual diversity of video games acts as the source of randomization and (2) player behavior -- represented semantically via textual descriptions of actions -- guides the *alignment* of videos with similar content. We test BehAVE on 25 games of the first-person shooter (FPS) genre across various video and text foundation models and we report its robustness for domain randomization. BehAVE successfully aligns player behavioral patterns and is able to zero-shot transfer them to multiple unseen FPS games when trained on just one FPS game. In a more challenging setting, BehAVE manages to improve the zero-shot transferability of foundation models to unseen FPS games (up to 22%) even when trained on a game of a different genre (Minecraft). Code and dataset can be found at https://github.com/nrasajski/BehAVE.


CC-VPSTO: Chance-Constrained Via-Point-based Stochastic Trajectory  Optimisation for Safe and Efficient Online Robot Motion Planning

Abstract: Safety in the face of uncertainty is a key challenge in robotics. In this work, we propose a real-time capable framework to generate safe and task-efficient robot trajectories for stochastic control problems. For that, we first formulate the problem as a chance-constrained optimisation problem, in which the probability of the controlled system to violate a safety constraint is constrained to be below a user-defined threshold. To solve the chance-constrained optimisation problem, we propose a Monte--Carlo approximation relying on samples of the uncertainty to estimate the probability of violating a safety constraint given a controller. We use this approximation in the motion planner VP-STO to solve the sampled-based problem. Consequently, we refer to our adapted approach as CC-VPSTO, which stands for Chance-Constrained VP-STO. We address the crucial issue concerning the Monte--Carlo approximation: given a predetermined number of uncertainty samples, we propose several ways to define the sample-based problem such that it is a reliable over-approximation of the original problem, i.e. any solution to the sample-based problem adheres to the original chance-constrained problem with high confidence. The strengths of our approach lie in i) its generality, as it does not require any specific assumptions on the underlying uncertainty distribution, the dynamics of the system, the cost function, and for some of the proposed sample-based approximations, on the form of inequality constraints; and ii) its applicability to MPC-settings. We demonstrate the validity and efficiency of our approach on both simulation and real-world robot experiments. For additional material, please visit https://sites.google.com/oxfordrobotics.institute/cc-vpsto.


Cheating Suffix: Targeted Attack to Text-To-Image Diffusion Models with  Multi-Modal Priors

Abstract: Diffusion models have been widely deployed in various image generation tasks, demonstrating an extraordinary connection between image and text modalities. However, they face challenges of being maliciously exploited to generate harmful or sensitive images by appending a specific suffix to the original prompt. Existing works mainly focus on using single-modal information to conduct attacks, which fails to utilize multi-modal features and results in less than satisfactory performance. Integrating multi-modal priors (MMP), i.e. both text and image features, we propose a targeted attack method named MMP-Attack in this work. Specifically, the goal of MMP-Attack is to add a target object into the image content while simultaneously removing the original object. The MMP-Attack shows a notable advantage over existing works with superior universality and transferability, which can effectively attack commercial text-to-image (T2I) models such as DALL-E 3. To the best of our knowledge, this marks the first successful attempt of transfer-based attack to commercial T2I models. Our code is publicly available at \url{https://github.com/ydc123/MMP-Attack}.


Frenetix Motion Planner: High-Performance and Modular Trajectory  Planning Algorithm for Complex Autonomous Driving Scenarios

Abstract: Our work aims to present a high-performance and modular sampling-based trajectory planning algorithm for autonomous vehicles. This algorithm is tailored to address the complex challenges in solution space construction and optimization problem formulation within the path planning domain. Our method employs a multi-objective optimization strategy for efficient navigation in static and highly dynamic environments, focusing on optimizing trajectory comfort, safety, and path precision. This algorithm was then used to analyze the algorithm performance and success rate in 1750 virtual complex urban and highway scenarios. Our results demonstrate fast calculation times (8ms for 800 trajectories), a high success rate in complex scenarios (88%), and easy adaptability with different modules presented. The most noticeable difference exhibited was the fast trajectory sampling, feasibility check, and cost evaluation step across various trajectory counts. While our study presents promising results, it's important to note that our assessments have been conducted exclusively in simulated environments, and real-world testing is required to fully validate our findings. The code and the additional modules used in this research are publicly available as open-source software and can be accessed at the following link: https://github.com/TUM-AVS/Frenetix-Motion-Planner.


Guidance Graph Optimization for Lifelong Multi-Agent Path Finding

Abstract: We study how to use guidance to improve the throughput of lifelong Multi-Agent Path Finding (MAPF). Previous studies have demonstrated that while incorporating guidance, such as highways, can accelerate MAPF algorithms, this often results in a trade-off with solution quality. In addition, how to generate good guidance automatically remains largely unexplored, with current methods falling short of surpassing manually designed ones. In this work, we introduce the directed guidance graph as a versatile representation of guidance for lifelong MAPF, framing Guidance Graph Optimization (GGO) as the task of optimizing its edge weights. We present two GGO algorithms to automatically generate guidance for arbitrary lifelong MAPF algorithms and maps. The first method directly solves GGO by employing CMA-ES, a black-box optimization algorithm. The second method, PIU, optimizes an update model capable of generating guidance, demonstrating the ability to transfer optimized guidance graphs to larger maps with similar layouts. Empirically, we show that (1) our guidance graphs improve the throughput of three representative lifelong MAPF algorithms in four benchmark maps, and (2) our update model can generate guidance graphs for as large as $93 \times 91$ maps and as many as 3000 agents.


Sequence Shortening for Context-Aware Machine Translation

Abstract: Context-aware Machine Translation aims to improve translations of sentences by incorporating surrounding sentences as context. Towards this task, two main architectures have been applied, namely single-encoder (based on concatenation) and multi-encoder models. In this study, we show that a special case of multi-encoder architecture, where the latent representation of the source sentence is cached and reused as the context in the next step, achieves higher accuracy on the contrastive datasets (where the models have to rank the correct translation among the provided sentences) and comparable BLEU and COMET scores as the single- and multi-encoder approaches. Furthermore, we investigate the application of Sequence Shortening to the cached representations. We test three pooling-based shortening techniques and introduce two novel methods - Latent Grouping and Latent Selecting, where the network learns to group tokens or selects the tokens to be cached as context. Our experiments show that the two methods achieve competitive BLEU and COMET scores and accuracies on the contrastive datasets to the other tested methods while potentially allowing for higher interpretability and reducing the growth of memory requirements with increased context size.


SMLP: Symbolic Machine Learning Prover

Abstract: Symbolic Machine Learning Prover (SMLP) is a tool and a library for system exploration based on data samples obtained by simulating or executing the system on a number of input vectors. SMLP aims at exploring the system based on this data by taking a grey-box approach: SMLP combines statistical methods of data exploration with building and exploring machine learning models in close feedback loop with the system's response, and exploring these models by combining probabilistic and formal methods. SMLP has been applied in industrial setting at Intel for analyzing and optimizing hardware designs at the analog level. SMLP is a general purpose tool and can be applied to systems that can be sampled and modeled by machine learning models.


XAI for Skin Cancer Detection with Prototypes and Non-Expert Supervision

Abstract: Skin cancer detection through dermoscopy image analysis is a critical task. However, existing models used for this purpose often lack interpretability and reliability, raising the concern of physicians due to their black-box nature. In this paper, we propose a novel approach for the diagnosis of melanoma using an interpretable prototypical-part model. We introduce a guided supervision based on non-expert feedback through the incorporation of: 1) binary masks, obtained automatically using a segmentation network; and 2) user-refined prototypes. These two distinct information pathways aim to ensure that the learned prototypes correspond to relevant areas within the skin lesion, excluding confounding factors beyond its boundaries. Experimental results demonstrate that, even without expert supervision, our approach achieves superior performance and generalization compared to non-interpretable models.


Adjoint Natural Deduction (Extended Version)

Abstract: Adjoint logic is a general approach to combining multiple logics with different structural properties, including linear, affine, strict, and (ordinary) intuitionistic logics, where each proposition has an intrinsic mode of truth. It has been defined in the form of a sequent calculus because the central concept of independence is most clearly understood in this form, and because it permits a proof of cut elimination following standard techniques. In this paper we present a natural deduction formulation of adjoint logic and show how it is related to the sequent calculus. As a consequence, every provable proposition has a verification (sometimes called a long normal form). We also give a computational interpretation of adjoint logic in the form of a functional language and prove properties of computations that derive from the structure of modes, including freedom from garbage (for modes without weakening and contraction), strictness (for modes disallowing weakening), and erasure (based on a preorder between modes). Finally, we present a surprisingly subtle algorithm for type checking.


On Measuring Context Utilization in Document-Level MT Systems

Abstract: Document-level translation models are usually evaluated using general metrics such as BLEU, which are not informative about the benefits of context. Current work on context-aware evaluation, such as contrastive methods, only measure translation accuracy on words that need context for disambiguation. Such measures cannot reveal whether the translation model uses the correct supporting context. We propose to complement accuracy-based evaluation with measures of context utilization. We find that perturbation-based analysis (comparing models' performance when provided with correct versus random context) is an effective measure of overall context utilization. For a finer-grained phenomenon-specific evaluation, we propose to measure how much the supporting context contributes to handling context-dependent discourse phenomena. We show that automatically-annotated supporting context gives similar conclusions to human-annotated context and can be used as alternative for cases where human annotations are not available. Finally, we highlight the importance of using discourse-rich datasets when assessing context utilization.


Climbing the Ladder of Interpretability with Counterfactual Concept  Bottleneck Models

Abstract: Current deep learning models are not designed to simultaneously address three fundamental questions: predict class labels to solve a given classification task (the "What?"), explain task predictions (the "Why?"), and imagine alternative scenarios that could result in different predictions (the "What if?"). The inability to answer these questions represents a crucial gap in deploying reliable AI agents, calibrating human trust, and deepening human-machine interaction. To bridge this gap, we introduce CounterFactual Concept Bottleneck Models (CF-CBMs), a class of models designed to efficiently address the above queries all at once without the need to run post-hoc searches. Our results show that CF-CBMs produce: accurate predictions (the "What?"), simple explanations for task predictions (the "Why?"), and interpretable counterfactuals (the "What if?"). CF-CBMs can also sample or estimate the most probable counterfactual to: (i) explain the effect of concept interventions on tasks, (ii) show users how to get a desired class label, and (iii) propose concept interventions via "task-driven" interventions.


A Probabilistic Model to explain Self-Supervised Representation Learning

Abstract: Self-supervised learning (SSL) learns representations by leveraging an auxiliary unsupervised task, such as classifying semantically related samples, e.g. different data augmentations or modalities. Of the many approaches to SSL, contrastive methods, e.g. SimCLR, CLIP and VicREG, have gained attention for learning representations that achieve downstream performance close to that of supervised learning. However, a theoretical understanding of the mechanism behind these methods eludes. We propose a generative latent variable model for the data and show that several families of discriminative self-supervised algorithms, including contrastive methods, approximately induce its latent structure over representations, providing a unifying theoretical framework. We also justify links to mutual information and the use of a projection head. Fitting our model generatively, as SimVE, improves performance over previous VAE methods on common benchmarks (e.g. FashionMNIST, CIFAR10, CelebA), narrows the gap to discriminative methods on _content_ classification and, as our analysis predicts, outperforms them where _style_ information is required, taking a step toward task-agnostic representations.


A comparison study of supervised learning techniques for the  approximation of high dimensional functions and feedback control

Abstract: Approximation of high dimensional functions is in the focus of machine learning and data-based scientific computing. In many applications, empirical risk minimisation techniques over nonlinear model classes are employed. Neural networks, kernel methods and tensor decomposition techniques are among the most popular model classes. We provide a numerical study comparing the performance of these methods on various high-dimensional functions with focus on optimal control problems, where the collection of the dataset is based on the application of the State-Dependent Riccati Equation.


A Reinforcement Learning-Boosted Motion Planning Framework:  Comprehensive Generalization Performance in Autonomous Driving

Abstract: This study introduces a novel approach to autonomous motion planning, informing an analytical algorithm with a reinforcement learning (RL) agent within a Frenet coordinate system. The combination directly addresses the challenges of adaptability and safety in autonomous driving. Motion planning algorithms are essential for navigating dynamic and complex scenarios. Traditional methods, however, lack the flexibility required for unpredictable environments, whereas machine learning techniques, particularly reinforcement learning (RL), offer adaptability but suffer from instability and a lack of explainability. Our unique solution synergizes the predictability and stability of traditional motion planning algorithms with the dynamic adaptability of RL, resulting in a system that efficiently manages complex situations and adapts to changing environmental conditions. Evaluation of our integrated approach shows a significant reduction in collisions, improved risk management, and improved goal success rates across multiple scenarios. The code used in this research is publicly available as open-source software and can be accessed at the following link: https://github.com/TUM-AVS/Frenetix-RL.


Synthetic Data for the Mitigation of Demographic Biases in Face  Recognition

Abstract: This study investigates the possibility of mitigating the demographic biases that affect face recognition technologies through the use of synthetic data. Demographic biases have the potential to impact individuals from specific demographic groups, and can be identified by observing disparate performance of face recognition systems across demographic groups. They primarily arise from the unequal representations of demographic groups in the training data. In recent times, synthetic data have emerged as a solution to some problems that affect face recognition systems. In particular, during the generation process it is possible to specify the desired demographic and facial attributes of images, in order to control the demographic distribution of the synthesized dataset, and fairly represent the different demographic groups. We propose to fine-tune with synthetic data existing face recognition systems that present some demographic biases. We use synthetic datasets generated with GANDiffFace, a novel framework able to synthesize datasets for face recognition with controllable demographic distribution and realistic intra-class variations. We consider multiple datasets representing different demographic groups for training and evaluation. Also, we fine-tune different face recognition systems, and evaluate their demographic fairness with different metrics. Our results support the proposed approach and the use of synthetic data to mitigate demographic biases in face recognition.


Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian  Processes

Abstract: While the great capability of Transformers significantly boosts prediction accuracy, it could also yield overconfident predictions and require calibrated uncertainty estimation, which can be commonly tackled by Gaussian processes (GPs). Existing works apply GPs with symmetric kernels under variational inference to the attention kernel; however, omitting the fact that attention kernels are in essence asymmetric. Moreover, the complexity of deriving the GP posteriors remains high for large-scale data. In this work, we propose Kernel-Eigen Pair Sparse Variational Gaussian Processes (KEP-SVGP) for building uncertainty-aware self-attention where the asymmetry of attention kernels is tackled by Kernel SVD (KSVD) and a reduced complexity is acquired. Through KEP-SVGP, i) the SVGP pair induced by the two sets of singular vectors from KSVD w.r.t. the attention kernel fully characterizes the asymmetry; ii) using only a small set of adjoint eigenfunctions from KSVD, the derivation of SVGP posteriors can be based on the inversion of a diagonal matrix containing singular values, contributing to a reduction in time complexity; iii) an evidence lower bound is derived so that variational parameters can be optimized towards this objective. Experiments verify our excellent performances and efficiency on in-distribution, distribution-shift and out-of-distribution benchmarks.


Integrating Large Language Models in Causal Discovery: A Statistical  Causal Approach

Abstract: In practical statistical causal discovery (SCD), embedding domain expert knowledge as constraints into the algorithm is widely accepted as significant for creating consistent meaningful causal models, despite the recognized challenges in systematic acquisition of the background knowledge. To overcome these challenges, this paper proposes a novel methodology for causal inference, in which SCD methods and knowledge based causal inference (KBCI) with a large language model (LLM) are synthesized through "statistical causal prompting (SCP)" for LLMs and prior knowledge augmentation for SCD. Experiments have revealed that GPT-4 can cause the output of the LLM-KBCI and the SCD result with prior knowledge from LLM-KBCI to approach the ground truth, and that the SCD result can be further improved, if GPT-4 undergoes SCP. Furthermore, it has been clarified that an LLM can improve SCD with its background knowledge, even if the LLM does not contain information on the dataset. The proposed approach can thus address challenges such as dataset biases and limitations, illustrating the potential of LLMs to improve data-driven causal inference across diverse scientific domains.


A Comparative Analysis of Conversational Large Language Models in  Knowledge-Based Text Generation

Abstract: Generating natural language text from graph-structured data is essential for conversational information seeking. Semantic triples derived from knowledge graphs can serve as a valuable source for grounding responses from conversational agents by providing a factual basis for the information they communicate. This is especially relevant in the context of large language models, which offer great potential for conversational interaction but are prone to hallucinating, omitting, or producing conflicting information. In this study, we conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples. We compare four large language models of varying sizes with different prompting techniques. Through a series of benchmark experiments on the WebNLG dataset, we analyze the models' performance and identify the most common issues in the generated predictions. Our findings show that the capabilities of large language models in triple verbalization can be significantly improved through few-shot prompting, post-processing, and efficient fine-tuning techniques, particularly for smaller models that exhibit lower zero-shot performance.


Moving Aggregate Modified Autoregressive Copula-Based Time Series Models  (MAGMAR-Copulas) Without Markov Restriction

Abstract: Copula-based time series models implicitly assume a finite Markov order. In reality a time series may not follow the Markov property. We modify the copula-based time series models by introducing a moving aggregate (MAG) part into the model updating equation. The functional form of the MAG-part is given as the inverse of a conditional copula. The resulting MAG-modified Autoregressive Copula-Based Time Series model (MAGMAR-Copula) is discussed in detail and distributional properties are derived in a D-vine framework. The model nests the classical ARMA model as well as the copula-based time series model. The modeling performance is compared with the model from \cite{mcneil2022time} modeling US inflation. Our model is competitive in terms of information criteria. It is a generalization of ARMA and also copula-based time series models and is in spirit similar to other moving average time series models such as ARMA and GARCH.


Evaluating UAV Path Planning Algorithms for Realistic Maritime Search  and Rescue Missions

Abstract: Unmanned Aerial Vehicles (UAVs) are emerging as very important tools in search and rescue (SAR) missions at sea, enabling swift and efficient deployment for locating individuals or vessels in distress. The successful execution of these critical missions heavily relies on effective path planning algorithms that navigate UAVs through complex maritime environments while considering dynamic factors such as water currents and wind flow. Furthermore, they need to account for the uncertainty in search target locations. However, existing path planning methods often fail to address the inherent uncertainty associated with the precise location of search targets and the uncertainty of oceanic forces. In this paper, we develop a framework to develop and investigate trajectory planning algorithms for maritime SAR scenarios employing UAVs. We adopt it to compare multiple planning strategies, some of them used in practical applications by the United States Coast Guard. Furthermore, we propose a novel planner that aims at bridging the gap between computation heavy, precise algorithms and lightweight strategies applicable to real-world scenarios.


Code-Switched Language Identification is Harder Than You Think

Abstract: Code switching (CS) is a very common phenomenon in written and spoken communication but one that is handled poorly by many natural language processing applications. Looking to the application of building CS corpora, we explore CS language identification (LID) for corpus building. We make the task more realistic by scaling it to more languages and considering models with simpler architectures for faster inference. We also reformulate the task as a sentence-level multi-label tagging problem to make it more tractable. Having defined the task, we investigate three reasonable models for this task and define metrics which better reflect desired performance. We present empirical evidence that no current approach is adequate and finally provide recommendations for future work in this area.


Di-NeRF: Distributed NeRF for Collaborative Learning with Unknown  Relative Poses

Abstract: Collaborative mapping of unknown environments can be done faster and more robustly than a single robot. However, a collaborative approach requires a distributed paradigm to be scalable and deal with communication issues. This work presents a fully distributed algorithm enabling a group of robots to collectively optimize the parameters of a Neural Radiance Field (NeRF). The algorithm involves the communication of each robot's trained NeRF parameters over a mesh network, where each robot trains its NeRF and has access to its own visual data only. Additionally, the relative poses of all robots are jointly optimized alongside the model parameters, enabling mapping with unknown relative camera poses. We show that multi-robot systems can benefit from differentiable and robust 3D reconstruction optimized from multiple NeRFs. Experiments on real-world and synthetic data demonstrate the efficiency of the proposed algorithm. See the website of the project for videos of the experiments and supplementary material(https://sites.google.com/view/di-nerf/home).


Connecting the Dots: Is Mode-Connectedness the Key to Feasible  Sample-Based Inference in Bayesian Neural Networks?

Abstract: A major challenge in sample-based inference (SBI) for Bayesian neural networks is the size and structure of the networks' parameter space. Our work shows that successful SBI is possible by embracing the characteristic relationship between weight and function space, uncovering a systematic link between overparameterization and the difficulty of the sampling problem. Through extensive experiments, we establish practical guidelines for sampling and convergence diagnosis. As a result, we present a Bayesian deep ensemble approach as an effective solution with competitive performance and uncertainty quantification.


Multilingual Gradient Word-Order Typology from Universal Dependencies

Abstract: While information from the field of linguistic typology has the potential to improve performance on NLP tasks, reliable typological data is a prerequisite. Existing typological databases, including WALS and Grambank, suffer from inconsistencies primarily caused by their categorical format. Furthermore, typological categorisations by definition differ significantly from the continuous nature of phenomena, as found in natural language corpora. In this paper, we introduce a new seed dataset made up of continuous-valued data, rather than categorical data, that can better reflect the variability of language. While this initial dataset focuses on word-order typology, we also present the methodology used to create the dataset, which can be easily adapted to generate data for a broader set of features and languages.


Distractor Generation for Multiple-Choice Questions: A Survey of  Methods, Datasets, and Evaluation

Abstract: Distractors are important in learning evaluation. This paper surveys distractor generation tasks using English multiple-choice question datasets for textual and multimodal contexts. In particular, this paper presents a thorough literature review of the recent studies on distractor generation tasks, discusses multiple choice components and their characteristics, analyzes the related datasets, and summarizes the evaluation metrics of distractor generation. Our investigation reveals that more than half of datasets are human-generated from educational sources in specific domains such as Science and English, which are largely text-based, with a lack of open domain and multimodal datasets.


A Hybrid Strategy for Chat Transcript Summarization

Abstract: Text summarization is the process of condensing a piece of text to fewer sentences, while still preserving its content. Chat transcript, in this context, is a textual copy of a digital or online conversation between a customer (caller) and agent(s). This paper presents an indigenously (locally) developed hybrid method that first combines extractive and abstractive summarization techniques in compressing ill-punctuated or un-punctuated chat transcripts to produce more readable punctuated summaries and then optimizes the overall quality of summarization through reinforcement learning. Extensive testing, evaluations, comparisons, and validation have demonstrated the efficacy of this approach for large-scale deployment of chat transcript summarization, in the absence of manually generated reference (annotated) summaries.


Mapping the Multiverse of Latent Representations

Abstract: Echoing recent calls to counter reliability and robustness concerns in machine learning via multiverse analysis, we present PRESTO, a principled framework for mapping the multiverse of machine-learning models that rely on latent representations. Although such models enjoy widespread adoption, the variability in their embeddings remains poorly understood, resulting in unnecessary complexity and untrustworthy representations. Our framework uses persistent homology to characterize the latent spaces arising from different combinations of diverse machine-learning methods, (hyper)parameter configurations, and datasets, allowing us to measure their pairwise (dis)similarity and statistically reason about their distributions. As we demonstrate both theoretically and empirically, our pipeline preserves desirable properties of collections of latent representations, and it can be leveraged to perform sensitivity analysis, detect anomalous embeddings, or efficiently and effectively navigate hyperparameter search spaces.


The closed-branch decoder for quantum LDPC codes

Abstract: Quantum error correction is the building block for constructing fault-tolerant quantum processors that can operate reliably even if its constituting elements are corrupted by decoherence. In this context, real-time decoding is a necessity for implementing arbitrary quantum computations on the logical level. In this work, we present a new decoder for Quantum Low Density Parity Check (QLDPC) codes, named the closed-branch decoder, with a worst-case complexity loosely upper bounded by $\mathcal{O}(n\text{max}_{\text{gr}}\text{max}_{\text{br}})$, where $\text{max}_{\text{gr}}$ and $\text{max}_{\text{br}}$ are tunable parameters that pose the accuracy versus speed trade-off of decoding algorithms. For the best precision, the $\text{max}_{\text{gr}}\text{max}_{\text{br}}$ product is exponentially increasing, but we numerically prove that considering small values that are polynomials of the code distance are enough for good error correction performance. The decoder is described to great extent and compared with the Belief Propagation Ordered Statistics Decoder (BPOSD) operating over data qubit, phenomenological and circuit-level noise models for the class of Bivariate Bicycle (BB) codes. The results showcase a promising performance of the decoder, obtaining similar results with much lower complexity than BPOSD when considering the smallest distance codes, but experiencing some logical error probability degradation for the bigger ones. Ultimately, the performance and complexity of the decoder depends on the product $\text{max}_{\text{gr}}\text{max}_{\text{br}}$, which can be considered taking into account benefiting one of the two aspects at the expense of the other.


Continuously Distributing Entanglement in Quantum Networks with Regular  Topologies

Abstract: Small interconnected quantum processors can collaborate to tackle quantum computational problems that typically demand more capable devices. These linked processors, referred to as quantum nodes, can use shared entangled states to execute nonlocal operations. As a consequence, understanding how to distribute entangled states among nodes is essential for developing hardware and software. We analyze a protocol where entanglement is continuously distributed among nodes that are physically arranged in a regular pattern: a chain, a honeycomb lattice, a square grid, and a triangular lattice. These regular patterns allow for the modular expansion of networks for large-scale distributed quantum computing. Within the entanglement distribution protocol, nodes can fix the probability of attempting entanglement swaps to trade off multiple entangled states shared with neighboring nodes for fewer entangled states shared with non-neighboring nodes. We evaluate the protocol's performance using the virtual neighborhood size -- a metric indicating the number of other nodes with which a given node shares entangled states. Employing numerical methods, we find that nodes must perform more swaps to maximize the virtual neighborhood size when coherence times are short. In a chain network, the virtual neighborhood size's dependence on swap attempt probability differs for each node based on its distance from the end of the chain. Conversely, all nodes in the square grid exhibit a qualitatively similar dependence of the virtual neighborhood size on the swap probability.


Big data applications on small quantum computers

Abstract: Current quantum hardware prohibits any direct use of large classical datasets. Coresets allow for a succinct description of these large datasets and their solution in a computational task is competitive with the solution on the original dataset. The method of combining coresets with small quantum computers to solve a given task that requires a large number of data points was first introduced by Harrow [arXiv:2004.00026]. In this paper, we apply the coreset method in three different well-studied classical machine learning problems, namely Divisive Clustering, 3-means Clustering, and Gaussian Mixture Model Clustering. We provide a Hamiltonian formulation of the aforementioned problems for which the number of qubits scales linearly with the size of the coreset. Then, we evaluate how the variational quantum eigensolver (VQE) performs on these problems and demonstrate the practical efficiency of coresets when used along with a small quantum computer. We perform noiseless simulations on instances of sizes up to 25 qubits on CUDA Quantum and show that our approach provides comparable performance to classical solvers.


Low-Resource Cross-Domain Singing Voice Synthesis via Reduced  Self-Supervised Speech Representations

Abstract: In this paper, we propose a singing voice synthesis model, Karaoker-SSL, that is trained only on text and speech data as a typical multi-speaker acoustic model. It is a low-resource pipeline that does not utilize any singing data end-to-end, since its vocoder is also trained on speech data. Karaoker-SSL is conditioned by self-supervised speech representations in an unsupervised manner. We preprocess these representations by selecting only a subset of their task-correlated dimensions. The conditioning module is indirectly guided to capture style information during training by multi-tasking. This is achieved with a Conformer-based module, which predicts the pitch from the acoustic model's output. Thus, Karaoker-SSL allows singing voice synthesis without reliance on hand-crafted and domain-specific features. There are also no requirements for text alignments or lyrics timestamps. To refine the voice quality, we employ a U-Net discriminator that is conditioned on the target speaker and follows a Diffusion GAN training scheme.


Adaptive Optimization for Prediction with Missing Data

Abstract: When training predictive models on data with missing entries, the most widely used and versatile approach is a pipeline technique where we first impute missing entries and then compute predictions. In this paper, we view prediction with missing data as a two-stage adaptive optimization problem and propose a new class of models, adaptive linear regression models, where the regression coefficients adapt to the set of observed features. We show that some adaptive linear regression models are equivalent to learning an imputation rule and a downstream linear regression model simultaneously instead of sequentially. We leverage this joint-impute-then-regress interpretation to generalize our framework to non-linear models. In settings where data is strongly not missing at random, our methods achieve a 2-10% improvement in out-of-sample accuracy.


Quantum advantage in zero-error function computation with side  information

Abstract: We consider the problem of zero-error function computation with side information. Alice has a source $X$ and Bob has correlated source $Y$ and they can communicate via either classical or a quantum channel. Bob wants to calculate $f(X,Y)$ with zero error. We aim to characterize the minimum amount of information that Alice needs to send to Bob for this to happen with zero-error. In the classical setting, this quantity depends on the asymptotic growth of $\chi(G^{(m)})$, the chromatic number of an appropriately defined $m$-instance "confusion graph". In this work we present structural characterizations of $G^{(m)}$ and demonstrate two function computation scenarios that have the same single-instance confusion graph. However, in one case there a strict advantage in using quantum transmission as against classical transmission, whereas there is no such advantage in the other case.


Boximator: Generating Rich and Controllable Motions for Video Synthesis

Abstract: Generating rich and controllable motion is a pivotal challenge in video synthesis. We propose Boximator, a new approach for fine-grained motion control. Boximator introduces two constraint types: hard box and soft box. Users select objects in the conditional frame using hard boxes and then use either type of boxes to roughly or rigorously define the object's position, shape, or motion path in future frames. Boximator functions as a plug-in for existing video diffusion models. Its training process preserves the base model's knowledge by freezing the original weights and training only the control module. To address training challenges, we introduce a novel self-tracking technique that greatly simplifies the learning of box-object correlations. Empirically, Boximator achieves state-of-the-art video quality (FVD) scores, improving on two base models, and further enhanced after incorporating box constraints. Its robust motion controllability is validated by drastic increases in the bounding box alignment metric. Human evaluation also shows that users favor Boximator generation results over the base model.


Deep Active Learning for Data Mining from Conflict Text Corpora

Abstract: High-resolution event data on armed conflict and related processes have revolutionized the study of political contention with datasets like UCDP GED, ACLED etc. However, most of these datasets limit themselves to collecting spatio-temporal (high-resolution) and intensity data. Information on dynamics, such as targets, tactics, purposes etc. are rarely collected owing to the extreme workload of collecting data. However, most datasets rely on a rich corpus of textual data allowing further mining of further information connected to each event. This paper proposes one such approach that is inexpensive and high performance, leveraging active learning - an iterative process of improving a machine learning model based on sequential (guided) human input. Active learning is employed to then step-wise train (fine-tuning) of a large, encoder-only language model adapted for extracting sub-classes of events relating to conflict dynamics. The approach shows performance similar to human (gold-standard) coding while reducing the amount of required human annotation by as much as 99%.


Efficient and Interaction-Aware Trajectory Planning for Autonomous  Vehicles with Particle Swarm Optimization

Abstract: This paper introduces a novel numerical approach to achieving smooth lane-change trajectories in autonomous driving scenarios. Our trajectory generation approach leverages particle swarm optimization (PSO) techniques, incorporating Neural Network (NN) predictions for trajectory refinement. The generation of smooth and dynamically feasible trajectories for the lane change maneuver is facilitated by combining polynomial curve fitting with particle propagation, which can account for vehicle dynamics. The proposed planning algorithm is capable of determining feasible trajectories with real-time computation capability. We conduct comparative analyses with two baseline methods for lane changing, involving analytic solutions and heuristic techniques in numerical simulations. The simulation results validate the efficacy and effectiveness of our proposed approach.


Spiking Music: Audio Compression with Event Based Auto-encoders

Abstract: Neurons in the brain communicate information via punctual events called spikes. The timing of spikes is thought to carry rich information, but it is not clear how to leverage this in digital systems. We demonstrate that event-based encoding is efficient for audio compression. To build this event-based representation we use a deep binary auto-encoder, and under high sparsity pressure, the model enters a regime where the binary event matrix is stored more efficiently with sparse matrix storage algorithms. We test this on the large MAESTRO dataset of piano recordings against vector quantized auto-encoders. Not only does our "Spiking Music compression" algorithm achieve a competitive compression/reconstruction trade-off, but selectivity and synchrony between encoded events and piano key strikes emerge without supervision in the sparse regime.


Free Quantum Codes from Twisted Unitary $t$-groups

Abstract: We introduce twisted unitary $t$-groups, a generalization of unitary $t$-groups under a twisting by an irreducible representation. We then apply representation theoretic methods to the Knill-Laflamme error correction conditions to show that twisted unitary $t$-groups automatically correspond to quantum codes with distance $d=t+1$. By construction these codes have many transversal gates, which are naturally fault tolerant.


Stochastic Two Points Method for Deep Model Zeroth-order Optimization

Abstract: Large foundation models, such as large language models, have performed exceptionally well in various application scenarios. Building or fully fine-tuning such large models is usually prohibitive due to either hardware budget or lack of access to backpropagation. The zeroth-order methods offer a promising direction for tackling this challenge, where only forward passes are needed to update the model. This paper introduces an efficient Stochastic Two-Point (S2P) approach within the gradient-free regime. We present the theoretical convergence properties of S2P under the general and relaxed smoothness assumptions. The theoretical properties also shed light on a faster and more stable S2P variant, Accelerated S2P (AS2P), through exploiting our new convergence properties that better represent the dynamics of deep models in training. Our comprehensive empirical results show that AS2P is highly effective in optimizing objectives for large deep models, including language models, and outperforms standard methods across various model types and scales, with 2 $\times$ speed-up in training over most conducted tasks.


Position Paper: Generalized grammar rules and structure-based  generalization beyond classical equivariance for lexical tasks and  transduction

Abstract: Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the concept of Generalized Grammar Rules (GGRs), a class of symmetry-based compositional constraints for transduction tasks, which we view as a transduction analogue of equivariance constraints in physics-inspired tasks. Besides formalizing generalized notions of symmetry for language transduction, our framework is general enough to contain many existing works as special cases. We present ideas on how GGRs might be implemented, and in the process draw connections to reinforcement learning and other areas of research.


Hyperparameter tuning via trajectory predictions: Stochastic prox-linear  methods in matrix sensing

Abstract: Motivated by the desire to understand stochastic algorithms for nonconvex optimization that are robust to their hyperparameter choices, we analyze a mini-batched prox-linear iterative algorithm for the problem of recovering an unknown rank-1 matrix from rank-1 Gaussian measurements corrupted by noise. We derive a deterministic recursion that predicts the error of this method and show, using a non-asymptotic framework, that this prediction is accurate for any batch-size and a large range of step-sizes. In particular, our analysis reveals that this method, though stochastic, converges linearly from a local initialization with a fixed step-size to a statistical error floor. Our analysis also exposes how the batch-size, step-size, and noise level affect the (linear) convergence rate and the eventual statistical estimation error, and we demonstrate how to use our deterministic predictions to perform hyperparameter tuning (e.g. step-size and batch-size selection) without ever running the method. On a technical level, our analysis is enabled in part by showing that the fluctuations of the empirical iterates around our deterministic predictions scale with the error of the previous iterate.


Estimating Max-Stable Random Vectors with Discrete Spectral Measure  using Model-Based Clustering

Abstract: This study introduces a novel estimation method for the entries and structure of a matrix $A$ in the linear factor model $\textbf{X} = A\textbf{Z} + \textbf{E}$. This is applied to an observable vector $\textbf{X} \in \mathbb{R}^d$ with $\textbf{Z} \in \mathbb{R}^K$, a vector composed of independently regularly varying random variables, and light-tailed independent noise $\textbf{E} \in \mathbb{R}^d$. This leads to max-linear models treated in classical multivariate extreme value theory. The spectral measure of the limit distribution is subsequently discrete and completely characterized by the matrix $A$. Every max-stable random vector with discrete spectral measure can be written as a max-linear model. Each row of the matrix $A$ is both scaled and sparse. Additionally, the value of $K$ is not known a priori. The problem of identifying the matrix $A$ from its matrix of pairwise extremal correlation is addressed. In the presence of pure variables, which are elements of $\textbf{X}$ linked, through $A$, to a single latent factor, the matrix $A$ can be reconstructed from the extremal correlation matrix. Our proofs of identifiability are constructive and pave the way for our innovative estimation for determining the number of factors $K$ and the matrix $A$ from $n$ weakly dependent observations on $\textbf{X}$. We apply the suggested method to weekly maxima rainfall and wildfires to demonstrate its applicability.


NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties

Abstract: In the pursuit to understand the intricacies of human brain's visual processing, reconstructing dynamic visual experiences from brain activities emerges as a challenging yet fascinating endeavor. While recent advancements have achieved success in reconstructing static images from non-invasive brain recordings, the domain of translating continuous brain activities into video format remains underexplored. In this work, we introduce NeuroCine, a novel dual-phase framework to targeting the inherent challenges of decoding fMRI data, such as noises, spatial redundancy and temporal lags. This framework proposes spatial masking and temporal interpolation-based augmentation for contrastive learning fMRI representations and a diffusion model enhanced by dependent prior noise for video generation. Tested on a publicly available fMRI dataset, our method shows promising results, outperforming the previous state-of-the-art models by a notable margin of ${20.97\%}$, ${31.00\%}$ and ${12.30\%}$ respectively on decoding the brain activities of three subjects in the fMRI dataset, as measured by SSIM. Additionally, our attention analysis suggests that the model aligns with existing brain structures and functions, indicating its biological plausibility and interpretability.


A GP-based Robust Motion Planning Framework for Agile Autonomous Robot  Navigation and Recovery in Unknown Environments

Abstract: For autonomous mobile robots, uncertainties in the environment and system model can lead to failure in the motion planning pipeline, resulting in potential collisions. In order to achieve a high level of robust autonomy, these robots should be able to proactively predict and recover from such failures. To this end, we propose a Gaussian Process (GP) based model for proactively detecting the risk of future motion planning failure. When this risk exceeds a certain threshold, a recovery behavior is triggered that leverages the same GP model to find a safe state from which the robot may continue towards the goal. The proposed approach is trained in simulation only and can generalize to real world environments on different robotic platforms. Simulations and physical experiments demonstrate that our framework is capable of both predicting planner failures and recovering the robot to states where planner success is likely, all while producing agile motion.


L2G2G: a Scalable Local-to-Global Network Embedding with Graph  Autoencoders

Abstract: For analysing real-world networks, graph representation learning is a popular tool. These methods, such as a graph autoencoder (GAE), typically rely on low-dimensional representations, also called embeddings, which are obtained through minimising a loss function; these embeddings are used with a decoder for downstream tasks such as node classification and edge prediction. While GAEs tend to be fairly accurate, they suffer from scalability issues. For improved speed, a Local2Global approach, which combines graph patch embeddings based on eigenvector synchronisation, was shown to be fast and achieve good accuracy. Here we propose L2G2G, a Local2Global method which improves GAE accuracy without sacrificing scalability. This improvement is achieved by dynamically synchronising the latent node representations, while training the GAEs. It also benefits from the decoder computing an only local patch loss. Hence, aligning the local embeddings in each epoch utilises more information from the graph than a single post-training alignment does, while maintaining scalability. We illustrate on synthetic benchmarks, as well as real-world examples, that L2G2G achieves higher accuracy than the standard Local2Global approach and scales efficiently on the larger data sets. We find that for large and dense networks, it even outperforms the slow, but assumed more accurate, GAEs.


MAGDi: Structured Distillation of Multi-Agent Interaction Graphs  Improves Reasoning in Smaller Language Models

Abstract: Multi-agent interactions between Large Language Model (LLM) agents have shown major improvements on diverse reasoning tasks. However, these involve long generations from multiple models across several rounds, making them expensive. Moreover, these multi-agent approaches fail to provide a final, single model for efficient inference. To address this, we introduce MAGDi, a new method for structured distillation of the reasoning interactions between multiple LLMs into smaller LMs. MAGDi teaches smaller models by representing multi-agent interactions as graphs, augmenting a base student model with a graph encoder, and distilling knowledge using three objective functions: next-token prediction, a contrastive loss between correct and incorrect reasoning, and a graph-based objective to model the interaction structure. Experiments on seven widely-used commonsense and math reasoning benchmarks show that MAGDi improves the reasoning capabilities of smaller models, outperforming several methods that distill from a single teacher and multiple teachers. Moreover, MAGDi also demonstrates an order of magnitude higher efficiency over its teachers. We conduct extensive analyses to show that MAGDi (1) enhances the generalizability to out-of-domain tasks, (2) scales positively with the size and strength of the base student model, and (3) obtains larger improvements (via our multi-teacher training) when applying self-consistency - an inference technique that relies on model diversity.


Securing Tomorrow: The Intersection of AI, Data, and Analytics in Fraud Prevention

Abstract: Aim: This research investigates the interconnections among Data Analytics, Artificial Intelligence, and other cutting-edge technologies to enhance comprehension of fraud prevention. The advantages of integrating machine learning and data analytics into artificial intelligence systems for industry-wide fraud detection and prevention are examined in this study. Study Design: My approach involved conducting an extensive examination of existing literature and analysing numerous case studies to gather information on the role of artificial intelligence, data, and analytics in fraud prevention. Place and Duration of the Study: A broad spectrum of academic, corporate, and governmental sources is utilised to supply the research study with its international scope. This study examines publications and developments from 2019 to 2023. Methodology: The research procedure incorporated an exhaustive literature review. This assessment was composed of academic journals, conference proceedings, and official publications. A qualitative analysis was conducted to assess the data, identify commonalities, and evaluate the strengths and weaknesses of AI fraud protection solutions. A more comprehensive examination of practical implementations was facilitated by case studies, which enhanced comprehension of fraud prevention strategies propelled by AI. Results: The research revealed important findings concerning the various ways in which analytics, data, and artificial intelligence can be implemented to prevent fraudulent activities. An examination of comparisons between generative AI for social engineering, credit card analytics, and cyber-physical security for Internet of Things (IoT) networks illuminated the merits and demerits of different Artificial Intelligence (AI) approaches. Conclusion: According to the findings of the study, AI, data, and analytics may alter system defences against fraud. The above-mentioned results underscore the significance of flexible fraud prevention strategies. Constant collaboration, innovative technology, and ongoing investigation are required to remain ahead of evolving fraud techniques. The paper concludes by emphasising the significance of future challenges and orientations.


Review of Twenty-First-Century Access Services: On the Front Line of Academic Librarianship, 2nd Edition

Abstract: Review of Twenty-First-Century Access Services: On the Front Line of Academic Librarianship, 2nd edition, edited by Michael J. Krasulski and Trevor A. Dawes. Association of College and Research Libraries. 320 pp. ISBN: 978-0-8389-3931-4 $76


Advancing Spectral Clustering for Categorical and Mixed-Type Data: Insights and Applications

Abstract: This study focuses on adapting spectral clustering, a numeric data-clustering technique, for categorical and mixed-type data. The method enhances spectral clustering for categorical and mixed-type data with novel kernel functions, showing improved accuracy in real-world applications. Despite achieving better clustering for datasets with mixed variables, challenges remain in identifying suitable kernel functions for categorical relationships.


Strategic Management Practices as Fundamental to the Achievement of Organizational Performances

Abstract: Given the increased emphasis on evaluating an organization's effectiveness in relation to strategic management practices, it is now widely recognized that delivering high-quality products and services, utilizing cutting-edge technology, establishing effective plans, and building an efficient resource management system is crucial for gaining a competitive advantage. Therefore, this research investigates the impact of strategic management practices on organizational performance, focusing specifically on the perspectives of personnel in the regional office of UBA in Lagos, Nigeria. To accomplish the study's goals, the researchers utilized the Taro Yamane sampling technique to ascertain an ideal sample size of 171 participants, obtained from a population of 300 randomly selected. Data analysis was performed using the SPSS program, correlation analysis, and scatter plot charts with a predetermined significance level of 0.05. The study's results demonstrate a statistically significant relationship between strategic management practices (strategy formulation, implementation, and evaluation) and organizational performance (profitability, productivity, and operational performance). Hence, the research recommends that management embrace strategic management methodologies to enhance organizational performance by establishing clear objectives, improving decision-making processes, optimizing resource allocation, adapting to environmental shifts, fostering creativity, ensuring organizational coherence, and implementing performance measurement.


Digital Death and Spectacular Death

Abstract: Throughout human history, individuals, communities and societies have always had to confront and tackle the problem of death. Consequently, death remains a topic of social scientific relevance, highlighting the need for its study and for theorising around it. This article analyses the development of the social scientific study of death and dying, taking inspiration from Philippe Ariès’s historical stages to discuss the recent developments in the field, namely the study of digital death. The article begins with a discussion of the visibility of death in modern society in the context of spectacular death. The analysis emphasises its four dimensions: mediatisation, commercialisation, re-ritualisation and the revolution in end-of-life care. The article moves on to discuss the emergence of digital death as the current stage and reflects on its similarities to spectacular death and its transformation of public imaginaries around death in contemporary society. The article concludes with a reflection on future developments in the field, specifically the emergence and study of artificial intelligence (AI) in digitalised death culture.


Multi-object behavior recognition based on object detection for dense crowds

Abstract: In recent years, research on multi-object behavior recognition methods based on pose estimation has made progress. However, in dense scenes, multi-object behavior recognition faces problems such as heavy occlusion, many small-scale objects, and an unbalanced distribution of object classes. Meanwhile, multi-object behavior recognition methods based on pose estimation are difficult to adapt to dense scenes. Therefore, we use the idea of object detection to complete the multi-object behavior recognition in dense scenes. However, compared with general object detection, the fine-grained classification and localization constrain each other due to the similarity of object classes, resulting in poor recognition performance. To solve the above problems, we propose a shallow convolutional neural networks (CNNs) module to extract task features separately in localization and classification task parallel branch structure, so that the two vision tasks avoid mutual constraints. Additionally, we propose a feature fusion mechanism to improve the detection performance of small objects by obtaining multi-scale high-level semantic information. To extract balanced and robust features, we exploit the number of objects in an image to generate dynamic class weights and difficulty weights. Experimental results on dense crowds in student classrooms demonstrated that the proposed method achieves superior performance compared to other state-of-the-art methods, with mAP50 of 88.72% and APs of 35.41%.


Linkage between Students’ Study Habits and their grades analyzed through Bayesian statistics

Abstract: Abstract It is well documented that particular study habits and strategies, among other factors, can influence students' grades and contribute to college success, independent learning, and retention. To further explore such connections, we studied and analyzed distinctions between two groups, those reporting use of "good" study habits vs. "not so good" study habits. Results establish a link between the two groups and Exam 1 grades, coded as Pass/Fail. Logistic regression, Bayesian statistics, and Matlab programming language were utilized for analysis, indicating that study habits depicted in the group reporting use of study habits research indicates as more powerful are indeed more likely to obtain a passing grade compared with those who did not report employing the better study strategies. because Bayesian statistics are not commonly employed to analyze these effects, a brief is provided on Bayesian statistics to facilitate researchers in this field who might also wish to use Bayesian statistics in their work.


Research Experiences for Teachers Summer Program in Biologically-inspired Computing Systems

Abstract: Abstract Funded by NSF Division of Computer and Network Systems, this grant establishes a new Research Experiences for Teachers (RET) Site at the University of South Alabama (USA). In 2021 summer, eight middle school and high school teachers from two local public-school districts spent six weeks engaged with research activities on biologically-inspired computing systems. They worked on discovery-based research projects and obtain transdisciplinary research experience on biologically-inspired computing systems spanning application (cancer detection), algorithm (Spiking Neural Networks), architecture and circuit (synaptic memory design), and device (memristor). The USA faculty mentors, curriculum development specialist from school districts, Instructional Coach from Science/Mathematics faculty at USA coached participants as they designed standards-compliant curriculum modules and conduct professional development activities. The implementation details of the summer program and the evaluations results are presented in this paper.


Can the COVID-19 pandemic boost collaborative online international learning (COIL) in engineering education? – A review for potential implementations

Abstract: Abstract Several adverse consequences resulted from the COVID-19 pandemic. These have affected academia at many levels, including mandatory requirements for educators and students worldwide to switch to remote online learning or blended learning. This need exacerbated new technology and software learning to maintain proper communication and student engagement during online lectures. For instructors, these demands included learning how to use software to produce high-quality lessons, either in a pre-recorded format for asynchronous learning or through live online lectures for synchronous learning. For students, it meant rapid adaptability to learn video teleconferencing communication tools and a demand to keep up with multiple courses and learning management systems (LMS). Nonetheless, this accelerated learning of online communication tools bridges a technology gap that previously limited online international education. Students and instructors worldwide are now familiar with online connectivity tools; thus, it is essential to review potential opportunities to increase collaborative learning and education, particularly at an international level. In the past, Collaborative Online International Learning (COIL) programs provided stewardship to global learning experiences at many universities worldwide. Broadly, these should cover four significant aspects: (a) collaborations with students in other countries with other backgrounds and cultures, (b) engagement through online interactions for assignment completion or lectures, (c) development and assessment of students' global perspectives and competencies, and a (d) reflective aspect of learning and education. This study reviews previous COIL implementations in the literature, emphasizing chemical engineering and other STEM courses. To our knowledge, COIL implementations are widespread in social sciences; however, few reports highlight these practices in engineering courses. We review perspectives, methodologies, challenges, and resources needed for positive and long-lasting COIL implementations here. In addition, we offer a plan to implement a short-term COIL-based project in a chemical engineering course with faculty participants from the US and El Salvador. Any engineering educators planning to implement COIL activities can use these suggestions. Ultimately, this low-cost pedagogy could significantly aid in promoting cultural, technical, and global competence in engineering students in the next post-pandemic years.


Moving Towards Data-Driven Departmental DEI

Abstract: Abstract Faculty and staff can and do influence the climate of a department and achievement of students. Research shows the positive effects of choosing to implement evidence-based teaching practices like active learning and inclusive teaching, and having a growth mindset in relation to the abilities of students. However, research also shows that the local climate in a department could cause students of color to be driven from STEM, or that a chilly climate could have a disproportionate impact on female students. And while the focus of Diversity, Equity, and Inclusion (DEI) efforts tends to be on women and under-represented minorities (URMs, defined as non-white, non-Asian), populations with representation at or above the demographics of the general population face their own challenges. In this paper, we describe recent efforts in the Mechanical Engineering and Applied Mechanics (MEAM) Department at the University of Pennsylvania to address these issues. Most of our initial efforts in this area have focused on the undergraduate population as well as their intersection with faculty and staff. We have started exploring the departmental structures and practices and have some initial demographic data on students and faculty. We are interested in exploring how retention, graduation, and achievement in general overlap and intersect with gender, race, and socio-economic status. We have also recently implemented a DEI Scholars program that further engages undergraduate and graduate students in this process. This initial work establishes baseline numbers and describes the first cohort we will track from acceptance through graduation. It is our aim that sharing these early efforts may encourage others to take on similar endeavors, and will also provide a reference point for future work of ours in this area.


Enhancing Engagement and Qualitative Output of Technical Projects Through Competing Team Assignments

Abstract: Abstract This paper discusses the benefits of engaging engineering student teams through competition in technical development projects. The University of XXX is a minority serving institution situated in the developing region of the XXX state, where recently, there has been a great expansion of advanced manufacturing industries, including innovative aerospace companies. These industries look for engineering and technology students who are creative thinkers and capable of addressing complex engineering problems. The best of student output often comes out with a friendly competition. Hence, two student teams were assigned with the same sponsored project to come out with innovative solutions. The project was sponsored by an aerospace-industry with a budget of $3000 per team for these two competing teams to come out with successful proof of concepts. The project consists of designing and building a semi-automatic weather balloon launching system with minimum human intervention. The competing teams did extensive research on the existing technology and constraints in arriving at feasible solutions. One of the design goals is to develop this automated system with flexibility in site selection and launch schedule. The intended design must include inflating balloon with helium that has a lifting capacity of around two pounds. The payload is a radiosonde- an electronic device that captures and transmits to the control base weather-related details such as humidity, temperature, pressure and wind speed at different altitudes of the atmosphere. This weather data is used to determine flight conditions of rockets that are launched. Hence, this data plays a paramount importance in launching of rockets with valuable payloads. Typically, it takes two persons several man-hours to assemble and launch these weather balloons. Automating this process will save these man-power and can be utilized for other creative and productive tasks in the company. The student teams have to conduct several scientific and technological experiments to understand design parameters for the project. Major activities that are to be automated include, inflating with helium, sealing, and release of balloon. Intended sequence is as follows: attachment of balloon, fill helium to set volume to lift two pounds of payload, power radiosonde and verify signal, attach radiosonde along with de-realer, seal balloon and launch. All these complex tasks have to be automated along with integration. The student teams followed the principles of product design in arriving at a solution to this complex automation problem. The outcomes of the projects were evaluated and assessed by a rubric-instrument developed to meet the ABET-TAC (Accreditation Board of Engineering and Technology- Technology Accreditation Commission) accreditation requirement. Student survey instrument was used to capture collaboration and engagement of students for the successful completion of their projects. Team dynamics and influence of competition in successfully completing a design project are thoroughly discussed in the paper. Also, discussed in detail in the paper are the pedagogical aspects of the Engineering Technology curriculum and seamless integration of industry sponsored projects that have the potential to enhance the marketability of its graduates.


Software Engineering and Security: Lessons Learned Creating a New Course in Security from a Software Engineering Perspective

Abstract: Abstract Security is a rising concern for organizations hiring undergraduates out of college in computing disciplines. This is reflected in the emerging prominence of cybersecurity related courses, certificates, and degree programs, and reflected in the most recent curricular standards guidelines. Perspectives on security recognize it as both a system discipline, meaning the inclusion of hardware, software, and networking components, and a cross-cutting skill across the major phases on the software engineering lifecycle (requirements, design, verification & validation, construction, and evolution and maintenance). Our conjecture is that there are many open and available resources for the first perspective (system) but few for the second perspective (software engineering). In this paper we share experiences creating a new junior-level security course in secure software engineering as a required course in an undergraduate accredited software engineering degree program. Specifically, and aligned with the latter perspective, we share the challenges we faced when seeking curricular resources, including open courseware repositories and textbooks, to jump start the development. We reflect on these experiences by providing a map of curricular resources to cross-cutting software engineering lifecycle phases, examining popular open (and usually federally funded) courseware repositories including the SEED project from Syracuse University and the Cybersecurity Labs and Resource Knowledge-base (CLARK) initiative, as well as ad hoc resources.


Programming learners struggle as much in Python as in C++ or Java

Abstract: Abstract Many computer science instructors believe switching college-level introductory programming courses to Python, versus languages like C++, Java, or C, smooths the learning of programming, due in part to Python's syntax, dynamic types, and interpreted execution. Studies providing quantitative support for that belief are rare, in part due to the challenge of creating comparative studies without extensive confounding factors. However, today one of the top textbooks in C++, Java, and Python all happen to come from the same publisher and have similar style and content, have similar off-the-shelf programming assignments that instructors can use, incorporate a cloud-based development environment and auto-grader that records every run, and each is used by tens of thousands of students at hundreds of universities. As such, a unique opportunity exists to compare student performance data across those three languages. We obtained such data from the publisher for 56 courses that met specific criteria (18 Python, 16 C++, 22 Java) totaling 6,804 students. As a potential indicator of whether Python students had a smoother learning experience, we compared median time and number of attempts on nearly-identical off-the-shelf programming assignments across the three languages. Python students exhibited no reduction in time or attempts. Manual investigation into the students' programs suggest that problem understanding and programming logic, and not language features, may be the main hurdles to learning programming.


Leveraging the power of Python, Octave and Matlab for Machine Learning

Abstract: Abstract The objective of this paper is to bring awareness, instigate interest, and promote the need of using AI and machine learning algorithms for information, engineering technology and premedical students. This paper will also attempt to review some of the widely used methods for supervised and unsupervised machine learning and the key issues that students may come across, suggest some discrete resolutions so as to provide optimal results on the accuracy and validity of Train and Test methodology. Artificial intelligence is the intelligence demonstrated by machines while machine learning is the ability of a computer to learn and make decisions the same as a human. Data science is the exploration and quantitative analysis of all available structured and unstructured data to develop understanding, extract knowledge, and formulate actionable results. Machine learning process may be divided into three main steps: data cleansing, feature extraction and optimization, and train/test system modeling. Models are evaluated based on statistics about the errors, or residuals, in the predicted values. Evaluating models is challenging since there is no testing data with labels to determine the correctness. PCA is used to evaluate clustering methods. Scikit-learn is a Python library that implements the various types of machine learning algorithms, such as classification, regression, clustering, decision tree, and more. Using Scikit-learn, implementing machine learning is now simply a matter of supplying the appropriate data to a function so that you can fit and train the model. The paper will explore selected programming tools, theoretical analysis of selected machine learning algorithms, demonstrate the main ML steps with examples and also explore the use of Matlab, Octave for ML besides python ML.