Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations
Patrick Inoue, Florian Röhrbein, Andreas Knoblauch
While deep neural networks (DNNs) have achieved remarkable performance in tasks such as image recognition, they often struggle with generalization, learning from few examples, and continuous adaptation - abilities inherent in biological neural systems. These challenges arise due to DNNs' failure to emulate the efficient, adaptive learning mechanisms of biological networks. To address these issues, we explore the integration of neurobiologically inspired assumptions in neural network learning. This study introduces a biologically inspired learning rule that naturally integrates neurobiological principles, including sparsity, lognormal weight distributions, and adherence to Dale's law, without requiring explicit enforcement. By aligning with these core neurobiological principles, our model enhances robustness against adversarial attacks and demonstrates superior generalization, particularly in few-shot learning scenarios. Notably, integrating these constraints leads to the emergence of biologically plausible neural representations, underscoring the efficacy of incorporating neurobiological assumptions into neural network design. Preliminary results suggest that this approach could extend from feature-specific to task-specific encoding, potentially offering insights into neural resource allocation for complex tasks.
Superpositions between non linear intermittency maps, application in biological neurons networks
Yiannis F. Contoyiannis
In a series of works of ours we have shown that we can represent the critical and tricritical points of the Statistical Physics of critical phenomena as a Dynamical phenomenon expressed by time series produced by the type I intermittency that exhibits a weak chaos. Recently we have also shown that if we couple these two chaotic dynamics, namely critical and tricritical, we can produce a time sequence which is a temporal Spike Train (ST) of biological-type . In the present work we generalize this issue producing superpositions of critical-tricritical intermittencies with different parameter values. Now arise the question whether the coupling occurs between time series that have resulted from the superposition, will preserved or destroyed the ST biological type , as the number of intermittencies in the superposition will increase? In the other side in present work we find that the spikes produced by the chaotic dynamics of the intermittencies, under the action of superpositions and coupling remain biological-type too. Thus we can say that the dynamics of the fluctuations of the values of the time series produced by the coupling of the superpositions of the intermittencies is the same as the dynamics of the fluctuations of the membrane potential of the biological neuron. Given also that we can manipulate the numerical experiment of superposition and coupling as we wish, we will be able, in future, to approach the cause of neurological problems and decline in thinking ability due to loss of spikes in the brain.
The projected isotropic normal distribution with applications in neuroscience
Kanti V. Mardia, Antonio Mauricio F. L. Miranda de Sa'
This paper is motivated by a cutting-edge application in neuroscience: the analysis of electroencephalogram (EEG) signals recorded under flash stimulation. Under commonly used signal-processing assumptions, only the phase angle of the EEG is required for the analysis of such applications. We demonstrate that these assumptions imply that the phase has a projected isotropic normal distribution. We revisit this distribution and derive several new properties, including closed-form expressions for its trigonometric moments. We then examine the distribution of the mean resultant and its square -- a statistic of central importance in phase-based EEG studies. The distribution of the resultant is analytically intricate; to make it practically useful, we develop two approximations based on the well-known resultant distribution for the von Mises distribution. We then study inference problems for this projected isotropic normal distribution. The method is illustrated with an application to EEG data from flash-stimulation experiments.
Partial Symmetry Enforced Attention Decomposition (PSEAD): A Group-Theoretic Framework for Equivariant Transformers in Biological Systems
Daniel Ayomide Olanrewaju
This research introduces the Theory of Partial Symmetry Enforced Attention Decomposition (PSEAD), a new and rigorous group-theoretic framework designed to seamlessly integrate local symmetry awareness into the core architecture of self-attention mechanisms within Transformer models. We formalize the concept of local permutation subgroup actions on windows of biological data, proving that under such actions, the attention mechanism naturally decomposes into a direct sum of orthogonal irreducible components. Critically, these components are intrinsically aligned with the irreducible representations of the acting permutation subgroup, thereby providing a powerful mathematical basis for disentangling symmetric and asymmetric features. We show that PSEAD offers substantial advantages. These include enhanced generalization capabilities to novel biological motifs exhibiting similar partial symmetries, unprecedented interpretability by allowing direct visualization and analysis of attention contributions from different symmetry channels, and significant computational efficiency gains by focusing representational capacity on relevant symmetric subspaces. Beyond static data analysis, we extend PSEAD's applicability to dynamic biological processes within reinforcement learning paradigms, showcasing its potential to accelerate the discovery and optimization of biologically meaningful policies in complex environments like protein folding and drug discovery. This work lays the groundwork for a new generation of biologically informed, symmetry-aware artificial intelligence models.
Learning Model Parameter Dynamics in a Combination Therapy for Bladder Cancer from Sparse Biological Data
Kayode Olumoyin, Lamees El Naqa, Katarzyna Rejniak
In a mathematical model of interacting biological organisms, where external interventions may alter behavior over time, traditional models that assume fixed parameters usually do not capture the evolving dynamics. In oncology, this is further exacerbated by the fact that experimental data are often sparse and sometimes are composed of a few time points of tumor volume. In this paper, we propose to learn time-varying interactions between cells, such as those of bladder cancer tumors and immune cells, and their response to a combination of anticancer treatments in a limited data scenario. We employ the physics-informed neural network (PINN) approach to predict possible subpopulation trajectories at time points where no observed data are available. We demonstrate that our approach is consistent with the biological explanation of subpopulation trajectories. Our method provides a framework for learning evolving interactions among biological organisms when external interventions are applied to their environment.
Production, Characteristics and Biological effects of Protonated Small Water Clusters
Yixin Zhu
The production and characteristics of protonated small water clusters (PSWCs) were reported in this work, where in electrospray ionization (ESI) of pure water, the species obtained were singly charged molecular ions consisting of 2, 3, 4 or 5 water molecules attached to a hydrogen ion, [(H2O)n+H]+, where n = 2, 3, 4 or 5. We proposed a new type of PSWCs structure: 2, 3, 4, 5 water molecules wrapped around a hydrogen ion which is located at the electrical and geometric center, forming a very stable molecular structure. Furthermore, biological tests of the PSWCs on mitochondrial function of intestinal epithelial cells and liver cells in mice showed the better therapeutic effect on inflammatory bowel diseases compared to that of the biologic agent Infliximab.
DEMO: Dose Exploration, Monitoring, and Optimization Using a Biological Mediator for Clinical Outcomes
Cheng-Han Yang, Peter F. Thall, Ruitao Lin
Phase 1-2 designs provide a methodological advance over phase 1 designs for dose finding by using both clinical response and toxicity. A phase 1-2 trial still may fail to select a truly optimal dose. because early response is not a perfect surrogate for long term therapeutic success. To address this problem, a generalized phase 1-2 design first uses a phase 1-2 design's components to identify a set of candidate doses, adaptively randomizes patients among the candidates, and after longer follow up selects a dose to maximize long-term success rate. In this paper, we extend this paradigm by proposing a design that exploits an early treatment-related, real-valued biological outcome, such as pharmacodynamic activity or an immunological effect, that may act as a mediator between dose and clinical outcomes, including tumor response, toxicity, and survival time. We assume multivariate dose-outcome models that include effects appearing in causal pathways from dose to the clinical outcomes. Bayesian model selection is used to identify and eliminate biologically inactive doses. At the end of the trial, a therapeutically optimal dose is chosen from the set of doses that are acceptably safe, clinically effective, and biologically active to maximize restricted mean survival time. Results of a simulation study show that the proposed design may provide substantial improvements over designs that ignore the biological variable.
Making sense of noise: introducing students to stochastic processes in order to better understand biological behaviors
Michael W. Klymkowsky
Biological systems are characterized by the ubiquitous roles of weak, that is, non-covalent molecular interactions, small, often very small, numbers of specific molecules per cell, and Brownian motion. These combine to produce stochastic behaviors at all levels from the molecular and cellular to the behavioral. That said, students are rarely introduced to the ubiquitous role of stochastic processes in biological systems, and how they produce unpredictable behaviors. Here I present the case that they need to be and provide some suggestions as to how it might be approached.
Biological Modelling with Nonlocal Advection Diffusion Equations
Kevin J Painter, Thomas Hillen, Jonathan R Potts
The employment of nonlocal PDE models to describe biological aggregation and other phenomena has gained considerable traction in recent years. For cell populations, these methods grant a means of accommodating essential elements such as cell adhesion, critical to the development and structure of tissues. For animals, they can be used to describe how the nearby presence of conspecifics and/or heterospecifics influence movement behaviour. In this review, we will focus on classes of biological movement models in which the advective (or directed) component to motion is governed by an integral term that accounts for how the surrounding distribution(s) of the population(s) impact on a member's movement. We recount the fundamental motivation for these models: the intrinsic capacity of cell populations to self-organise and spatially sort within tissues; the wide-ranging tendency of animals towards spatial structuring, from the formations of herds and swarms to territorial segregation. We examine the derivation of these models from an individual level, illustrating in the process methods that allow models to be connected to data. We explore a growing analytical literature, including methods of stability and bifurcation analysis, and existence results. We conclude with a short section that lays out some future challenges and connections to the modelling of sociological phenomena including opinion dynamics.
Reusability report: Prostate cancer stratification with diverse biologically-informed neural architectures
Christian Pedersen, Tiberiu Tesileanu, Tinghui Wu
et al.
In Elmarakeby et al., "Biologically informed deep neural network for prostate cancer discovery", a feedforward neural network with biologically informed, sparse connections (P-NET) was presented to model the state of prostate cancer. We verified the reproducibility of the study conducted by Elmarakeby et al., using both their original codebase, and our own re-implementation using more up-to-date libraries. We quantified the contribution of network sparsification by Reactome biological pathways, and confirmed its importance to P-NET's superior performance. Furthermore, we explored alternative neural architectures and approaches to incorporating biological information into the networks. We experimented with three types of graph neural networks on the same training data, and investigated the clinical prediction agreement between different models. Our analyses demonstrated that deep neural networks with distinct architectures make incorrect predictions for individual patient that are persistent across different initializations of a specific neural architecture. This suggests that different neural architectures are sensitive to different aspects of the data, an important yet under-explored challenge for clinical prediction tasks.
Information Fragmentation, Encryption and Information Flow in Complex Biological Networks
Clifford Bohm, Douglas Kirkpatrick, Victoria Cao
et al.
Assessing where and how information is stored in biological networks (such as neuronal and genetic networks) is a central task both in neuroscience and in molecular genetics, but most available tools focus on the network's structure as opposed to its function. Here we introduce a new information-theoretic tool: "information fragmentation analysis" that, given full phenotypic data, allows us to localize information in complex networks, determine how fragmented (across multiple nodes of the network) the information is, and assess the level of encryption of that information. Using information fragmentation matrices, we can also create information flow graphs that illustrate how information propagates through these networks. We illustrate the use of this tool by analyzing how artificial brains that evolved "in silico" solve particular tasks, and show how information fragmentation analysis provides deeper insights into how these brains process information and "think". The measures of information fragmentation and encryption that result from our methods also quantify complexity of information processing in these networks and how this processing complexity differs between primary exposure to sensory data (early in the lifetime) and later routine processing.
Model Order Reduction in Neuroscience
Bülent Karasözen
The human brain contains approximately $10^9$ neurons, each with approximately $10^3$ connections, synapses, with other neurons. Most sensory, cognitive and motor functions of our brains depend on the interaction of a large population of neurons. In recent years, many technologies are developed for recording large numbers of neurons either sequentially or simultaneously. An increase in computational power and algorithmic developments have enabled advanced analyses of neuronal population parallel to the rapid growth of quantity and complexity of the recorded neuronal activity. Recent studies made use of dimensionality and model order reduction techniques to extract coherent features which are not apparent at the level of individual neurons. It has been observed that the neuronal activity evolves on low-dimensional subspaces. The aim of model reduction of large-scale neuronal networks is an accurate and fast prediction of patterns and their propagation in different areas of the brain. Spatiotemporal features of the brain activity are identified on low dimensional subspaces with methods such as dynamic mode decomposition (DMD), proper orthogonal decomposition (POD), discrete empirical interpolation (DEIM) and combined parameter and state reduction. In this paper, we give an overview of the currently used dimensionality reduction and model order reduction techniques in neuroscience. This work will be featured as a chapter in the upcoming Handbook on Model Order Reduction,(P. Benner, S. Grivet-Talocia, A. Quarteroni, G. Rozza, W. H. A. Schilders, L. M. Silveira, eds, to appear on DE GRUYTER)
Preliminary Studies on the Usefulness of Nonlinear Boundary Element Method for Real-Time Simulation of Biological Organs
Kirana Kumara P
There is some literature on the application of linear boundary element method (BEM) for real-time simulation of biological organs. However, literature is scant when it comes to the application of nonlinear BEM, although there is a possibility that the use of nonlinear BEM would result in better simulations. Hence the present paper explores the possibility of using nonlinear BEM for real-time simulation of biological organs. This paper begins with a general discussion about using the nonlinear BEM for real-time simulation of biological organs. Literature on nonlinear BEM is reviewed and the literature that deal with nonlinear formulations and coding are noted down next. In the later sections, some results obtained from nonlinear analyses are compared with the corresponding results from linear analyses. The last section concludes with remarks that indicate that it might be possible to obtain better simulations in the future by using nonlinear BEM.
Toward the biological model of the hippocampus as the successor representation agent
Hyunsu Lee
The hippocampus is an essential brain region for spatial memory and learning. Recently, a theoretical model of the hippocampus based on temporal difference (TD) learning has been published. Inspired by the successor representation (SR) learning algorithms, which decompose value function of TD learning into reward and state transition, they argued that the rate of firing of CA1 place cells in the hippocampus represents the probability of state transition. This theory, called predictive map theory, claims that the hippocampus representing space learns the probability of transition from the current state to the future state. The neural correlates of expecting the future state are the firing rates of the CA1 place cells. This explanation is plausible for the results recorded in behavioral experiments, but it is lacking the neurobiological implications. Modifying the SR learning algorithm added biological implications to the predictive map theory. Similar with the simultaneous needs of information of the current and future state in the SR learning algorithm, the CA1 place cells receive two inputs from CA3 and entorhinal cortex. Mathematical transformation showed that the SR learning algorithm is equivalent to the heterosynaptic plasticity rule. The heterosynaptic plasticity phenomena in CA1 were discussed and compared with the modified SR update rule. This study attempted to interpret the TD algorithm as the neurobiological mechanism occurring in place learning, and to integrate the neuroscience and artificial intelligence approaches in the field.
Biologically-informed neural networks guide mechanistic modeling from sparse experimental data
John H. Lagergren, John T. Nardini, Ruth E. Baker
et al.
Biologically-informed neural networks (BINNs), an extension of physics-informed neural networks [1], are introduced and used to discover the underlying dynamics of biological systems from sparse experimental data. In the present work, BINNs are trained in a supervised learning framework to approximate in vitro cell biology assay experiments while respecting a generalized form of the governing reaction-diffusion partial differential equation (PDE). By allowing the diffusion and reaction terms to be multilayer perceptrons (MLPs), the nonlinear forms of these terms can be learned while simultaneously converging to the solution of the governing PDE. Further, the trained MLPs are used to guide the selection of biologically interpretable mechanistic forms of the PDE terms which provides new insights into the biological and physical mechanisms that govern the dynamics of the observed system. The method is evaluated on sparse real-world data from wound healing assays with varying initial cell densities [2].
Computer Simulation of Biological Processes at the High School
Olena V. Komarova
The article describes the method of using computer-learning tools during the modeling of biological processes in the high school. The author developed web pages for online processing of the results of simulation of the genetic structure of the population, which are affected by factors of change in its genetic equilibrium. It is argued that the modeling of the genetic structure of a population that is not described by the Hardy-Weinberg law allows students to formulate persistent beliefs about the causes and directions of microevolutionary changes in populations and the individual factors of the evolutionary process in general.
Systematic Measures of Biological Networks, Part I: Invariant measures and Entropy
Yao Li, Yingfei Yi
This paper is Part I of a two-part series devoting to the study of systematic measures in a complex biological network modeled by a system of ordinary differential equations. As the mathematical complement to our previous work [31] with collaborators, the series aims at establishing a mathematical foundation for characterizing three important systematic measures: degeneracy, complexity and robustness, in such a biological network and studying connections among them. To do so, we consider in Part I stationary measures of a Fokker-Planck equation generated from small white noise perturbations of a dissipative system of ordinary differential equations. Some estimations of concentration of stationary measures of the Fokker-Planck equation in the vicinity of the global attractor are presented. Relationship between differential entropy of stationary measures and dimension of the global attractor is also given.
Geometric phase shifts in biological oscillators
David S. Tourigny
Many intracellular processes continue to oscillate during the cell cycle. Although it is not well-understood how they are affected by discontinuities in the cellular environment, the general assumption is that oscillations remain robust provided the period of cell divisions is much larger than the period of the oscillator. Here, I will show that under these conditions a cell will in fact have to correct for an additional quantity added to the phase of oscillation upon every repetition of the cell cycle. The resulting phase shift is an analogue of the geometric phase, a curious entity first discovered in quantum mechanics. In this Letter, I will discuss the theory of the geometric phase shift and demonstrate its relevance to biological oscillations.
Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience
Ross W. Gayler
Jackendoff (2002) posed four challenges that linguistic combinatoriality and rules of language present to theories of brain function. The essence of these problems is the question of how to neurally instantiate the rapid construction and transformation of the compositional structures that are typically taken to be the domain of symbolic processing. He contended that typical connectionist approaches fail to meet these challenges and that the dialogue between linguistic theory and cognitive neuroscience will be relatively unproductive until the importance of these problems is widely recognised and the challenges answered by some technical innovation in connectionist modelling. This paper claims that a little-known family of connectionist models (Vector Symbolic Architectures) are able to meet Jackendoff's challenges.
An Unusual Antagonistic Pleiotropy in the Penna Model for Biological Ageing
A. O. Sousa, S. Moss de Oliveira
We combine the Penna Model for biological aging, which is based on the mutation-accumulation theory, with a sort of antagonistic pleiotropy. We show that depending on how the pleiotropy is introduced, it is possible to reproduce both the humans mortality, which increases exponentially with age, and fruitfly mortality, which decelerates at old ages, allowing the appearance of arbitrarily old Methuselah's.
en
cond-mat.stat-mech, q-bio