Publications by Dr Dominique Chu

Supervised learning in deep neural networks is commonly performed using error backpropagation. However, the sequential propagation of errors during the backward pass limits its scalability and applicability to low-powered neuromorphic hardware. Therefore, there is growing interest in finding local alternatives to backpropagation. Recently proposed methods based on forward-mode automatic differentiation suffer from high variance in large deep neural networks, which affects convergence. In this paper, we propose the Forward Direct Feedback Alignment algorithm that combines Activity-Perturbed Forward Gradients with Direct Feedback Alignment and momentum. We provide both theoretical proofs and empirical evidence that our proposed method achieves lower variance than forward gradient techniques. In this way, our approach enables faster convergence and better performance when compared to other local alternatives to backpropagation and opens a new perspective for the development of online learning algorithms compatible with neuromorphic systems.

Spiking Neural Networks (SNNs) have emerged as a promising alternative to traditional Deep Neural Networks for low-power computing. However, the effectiveness of SNNs is not solely determined by their performance but also by their energy consumption, prediction speed, and robustness to noise. The recent method Fast & Deep, along with others, achieves fast and energy-efficient computation by constraining neurons to fire at most once. Known as Time-To-First-Spike (TTFS), this constraint however restricts the capabilities of SNNs in many aspects. In this work, we explore the relationships between performance, energy consumption, speed and stability when using this constraint. More precisely, we highlight the existence of tradeoffs where performance and robustness are gained at the cost of sparsity and prediction latency. To improve these tradeoffs, we propose a relaxed version of Fast & Deep that allows for multiple spikes per neuron. Our experiments show that relaxing the spike constraint provides higher performance while also benefiting from faster convergence, similar sparsity, comparable prediction latency, and better robustness to noise compared to TTFS SNNs. By highlighting the limitations of TTFS and demonstrating the advantages of unconstrained SNNs we provide valuable insight for the development of effective learning strategies for neuromorphic computing.

Using formal methods complemented by large-scale simulations we investigate information theoretical properties of spiking neurons trained using Hebbian and STDP learning rules. It is shown that weight space contains meta-stable states, which are points where the average weight change under the learning rule vanishes. These points may capture the random walker transiently. The dwell time in the vicinity of the meta-stable state is either quasi-infinite or very short and depends on the level of noise in the system. Moreover, important information theoretic quantities, such as the amount of information the neuron transmits are determined by the meta-stable state. While the Hebbian learning rule reliably leads to meta-stable states, the STDP rule tends to be unstable in the sense that for most choices of hyper-parameters the weights are not captured by meta-stable states, except for a restricted set of choices. It emerges that stochastic fluctuations play an important role in determining which meta-stable state the neuron takes. To understand this, we model the trajectory of the neuron through weight space as an inhomogeneous Markovian random walk, where the transition probabilities between states are determined by the statistics of the input signal.

Hebbian theory seeks to explain how the neurons in the brain adapt to stimuli to enable learning. An interesting feature of Hebbian learning is that it is an unsupervised method and, as such, does not require feedback, making it suitable in contexts where systems have to learn autonomously. This paper explores how molecular systems can be designed to show such protointelligent behaviors and proposes the first chemical reaction network (CRN) that can exhibit autonomous Hebbian learning across arbitrarily many input channels. The system emulates a spiking neuron, and we demonstrate that it can learn statistical biases of incoming inputs. The basic CRN is a minimal, thermodynamically plausible set of microreversible chemical equations that can be analyzed with respect to their energy requirements. However, to explore how such chemical systems might be engineered de novo, we also propose an extended version based on enzyme-driven compartmentalized reactions. Finally, we show how a purely DNA system, built upon the paradigm of DNA strand displacement, can realize neuronal dynamics. Our analysis provides a compelling blueprint for exploring autonomous learning in biological settings, bringing us closer to realizing real synthetic biological intelligence.

Organophosphorus (OP) chemical warfare agents (CWAs) represent an ongoing global threat, through either purposeful environmental release or the need to dispose of historic stockpiles. This presents a need for the development of novel decontamination technologies. Due to the toxic nature and legal limitations placed on OP CWAs, the use of appropriate OP simulants that mimic the reactivity but not the toxicity of the agents themselves is vital to decontamination studies. Herein, we show that association constants derived from non-specific hydrogen-bonded complexation events may be used as parameters within models to predict simulant reactivity. We also discuss the limitations that should be placed on such data.

Organophosphorus (OP) chemical warfare agents (CWAs) represent an ongoing threat but the understandable widespread prohibition of their use places limitations on the development of technologies to counter the effects of any OP CWA release. Herein, we describe new, accessible methods for the identification of appropriate molecular simulants to mimic the hydrogen bond accepting capacity of the P[double bond, length as m-dash]O moiety, common to every member of this class of CWAs. Using the predictive methodologies developed herein, we have identified OP CWA hydrogen bond acceptor simulants for soman and sarin. It is hoped that the effective use of these physical property specific simulants will aid future countermeasure developments.

We analyse mathematically the constraints on weights resulting from Hebbian and STDP learning rules applied to a spiking neuron with weight normalisation. In the case of pure Hebbian learning, we find that the normalised weights equal the promotion probabilities of weights up to correction terms that depend on the learning rate and are usually small. A similar relation can be derived for STDP algorithms, where the normalised weight values reflect a difference between the promotion and demotion probabilities of the weight. These relations are practically useful in that they allow checking for convergence of Hebbian and STDP algorithms. Another application is novelty detection. We demonstrate this using the MNIST dataset.

Herein, we report 50 structurally related supramolecular self-associating amphiphilic (SSA) salts and related compounds. These SSAs are shown to act as antimicrobial agents, active against model Gram-positive (Methicillin-Resistant Staphylococcus aureus) and/or Gram-negative (Escherichia coli) bacteria of clinical interest. Through a combination of solution state, gas phase, solid state and in silico measurements we determine 14 different physicochemical parameters for each of these 50 structurally related compounds. These parameter sets are then used to identify molecular structure – physicochemical property – antimicrobial activity relationships for our model Gram-negative and Gram-positive bacteria, while simultaneously providing insight towards the elucidation of SSA mode of antimicrobial action.

The Multi-Spike Tempotron (MST) is a powerful single spiking neuron model that can solve complex supervised classification tasks. While powerful, it is also internally complex, computationally expensive to evaluate, and not suitable for neuromorphic hardware. Here we aim to understand whether it is possible to simplify the MST model, while retaining its ability to learn and to process information. To this end, we introduce a family of Generalised Neuron Models (GNM) which are a special case of the Spike Response Model and much simpler and cheaper to simulate than the MST. We find that over a wide range of parameters the GNM can learn at least as well as the MST. We identify the temporal autocorrelation of the membrane potential as the single most important ingredient of the GNM which enables it to classify multiple spatio-temporal patterns. We also interpret the GNM as a chemical system, thus conceptually bridging computation by neural networks with molecular information processing. We conclude the paper by proposing alternative training approaches for the GNM including error trace learning and error backpropagation.

The genetic code encodes 20 amino acids using 64 nucleotide triplets or codons. 18 of the 20 amino acids are encoded by multiple synonymous codons which are used in organismal genomes in a biased fashion. Codon bias arises because evolutionary selection favours particular nucleotide sequences over others encoding the same amino acid sequence. Despite many existing hypotheses, there is no current consensus on what the evolutionary drivers are. Using ideas from stochastic thermodynamics we derive from first principles a mathematical model describing the statistics of codon usage bias and apply it to extensive genomic data. Our main conclusions include the following findings: (1) Codon usage cannot be explained solely by selection pressures that act on the genome-wide frequency of codons, but also includes pressures that act at the level of individual genes. (2) Codon usage is not only biased in the usage frequency of nucleotide triplets but also in how they are distributed across mRNAs. (3) A new model-based measure of codon usage bias that extends existing measures by taking into account both codon frequency and codon distribution reveals distinct, amino acid specific patterns of selection in distinct branches of the tree of life.

Finite state machines (FSMs) are a theoretically and practically important model of computation. We propose a general, thermodynamically consistent model of FSMs and characterise the resource requirements of these machines. We model FSMs as time-inhomogeneous Markov chains. The computation is driven by instantaneous manipulations of the energy levels of the states. We calculate the entropy production of the machine, its error probability, and the time required to complete one update step. We find that a sequence of generalised bit-setting operations is sufficient to implement any FSM.

Fully mechanized Maxwell's demons, also called information ratchets, are an important conceptual link between computation, information theory, and statistical physics. They exploit low-entropy information reservoirs to extract work from a heat reservoir. Previous models of such demons have either ignored the cost of delivering bits to the demon from the information reservoir or assumed random access or infinite-dimensional information reservoirs to avoid such an issue. In this work we account for this cost when exploiting information reservoirs with physical structure and show that the dimensionality of the reservoir has a significant impact on the performance and phase diagram of the demon. We find that for conventional one-dimensional tapes the scope for work extraction is greatly reduced. An expression for the net-extracted work by demons exploring information reservoirs by means of biased random walks on d-dimensional, \(\mathbb Z^d\), information reservoirs is presented. Furthermore, we derive exact probabilities of recurrence in these systems, generalizing previously known results. We find that the demon is characterized by two critical dimensions. First, to extract work at zero bias the dimensionality of the information reservoir must be larger than d=2, corresponding to the dimensions where a simple random walker is transient. Second, for integer dimensions d>4 the unbiased random walk optimizes work extraction corresponding to the dimensions where a simple random walker is strongly transient.

The properties and fundamental limits of chemical computers have recently attracted significant interest as a model of computation, an unifying principle of cellular organisation and in the context of bio-engineering. As of yet, research in this topic is based on case-studies. There exists no generally accepted criterion to distinguish between chemical processes that compute and those that do not. Here, the concept of entropy driven computer (EDC) is proposed as a general model of chemical computation. It is found that entropy driven computation is subject to a trade-off between accuracy and entropy production, but unlike many biological systems, there are no trade-offs involving time. The latter only arise when it is taken into account that the observation of the state of the EDC is not energy neutral, but comes at a cost. The significance of this conclusion in relation to biological systems is discussed. Three examples of biological computers, including an implementation of a neural network as an EDC are given.

A central result of stochastic thermodynamics is that irreversible state transitions of Markovian systems entail a cost in terms of an infinite entropy production. A corollary of this is that strictly deterministic computation is not possible. Using a thermodynamically consistent model, we show that quasideterministic computation can be achieved at finite, and indeed modest cost with accuracies that are indistinguishable from deterministic behavior for all practical purposes. Concretely, we consider the entropy production of stochastic (Markovian) systems that behave like and and a not gates. Combinations of these gates can implement any logical function. We require that these gates return the correct result with a probability that is very close to 1, and additionally, that they do so within finite time. The central component of the model is a machine that can read and write binary tapes. We find that the error probability of the computation of these gates falls with the power of the system size, whereas the cost only increases linearly with the system size.

Many microbes when grown on a mixture of two carbon sources utilise first and exclusively the preferred sugar, before switching to the less preferred carbon source. This results in two distinct exponential growth phases, often interrupted by a lag-phase of reduced growth termed the lag-phase. While the lag-phase appears to be an evolved feature, it is not clear what drives its evolution, as it comes with a substantial up-front fitness penalty due to lost growth. In this article a minimal mathematical model based on a master-equation approach is proposed. This model can explain many empirically observed phenomena. It suggests that the lag-phase can be understood as a manifestation of the trade-off between switching speed and switching efficiency. Moreover, the model predicts heterogeneity of the population during the lag-phase. Finally, it is shown that the switch from one carbon source to another one is a sensing problem and the lag-phase is a manifestation of known fundamental limitations of biological sensors.

Bi-phasic or diauxic growth is often observed when microbes are grown in a chemically defined medium containing two sugars (for example glucose and lactose). Typically, the two growth stages are separated by an often lengthy phase of arrested growth, the so-called lag-phase. Diauxic growth is usually interpreted as an adaptation to maximise population growth in multi-nutrient environments. However, the lag-phase implies a substantial loss of growth during the switch-over. It therefore remains unexplained why the lag-phase is adaptive. Here we show by means of a stochastic simulation model based on the bacterial PTS system that it is not possible to shorten the lag-phase without incurring a permanent growth-penalty. Mechanistically, this is due to the inherent and well established limitations of biological sensors to operate efficiently at a given resource cost. Hence, there is a trade-off between lost growth during the diauxic switch and the long-term growth potential of the cell. Using simulated evolution we predict that the lag-phase will evolve depending on the distribution of conditions experienced during adaptation. In environments where switching is less frequently required, the lag-phase will evolve to be longer whereas, in frequently changing environments, the lag-phase will evolve to be shorter.

The glucose effect is a well known phenomenon whereby cells, when presented with two different nutrients, show a diauxic growth pattern, i.e. an episode of exponential growth followed by a lag phase of reduced growth followed by a second phase of exponential growth. Diauxic growth is usually thought of as a an adaptation to maximise biomass production in an environment offering two or more carbon sources. While diauxic growth has been studied widely both experimentally and theoretically, the hypothesis that diauxic growth is a strategy to increase overall growth has remained an unconfirmed conjecture. Here, we present a minimal mathematical model of a bacterial nutrient uptake system and metabolism. We subject this model to artificial evolution to test under which conditions diauxic growth evolves. As a result, we find that, indeed, sequential uptake of nutrients emerges if there is competition for nutrients and the metabolism/uptake system is capacity limited. However, we also find that diauxic growth is a secondary effect of this system and that the speed-up of nutrient uptake is a much larger effect. Notably, this speed-up of nutrient uptake coincides with an overall reduction of efficiency. Our two main conclusions are: (i) Cells competing for the same nutrients evolve rapid but inefficient growth dynamics. (ii) In the deterministic models we use here no substantial lag-phase evolves. This suggests that the lag-phase is a consequence of stochastic gene expression.

Codon usage bias (CUB) is the well known phenomenon that the frequency of synonymous codons is unequal. This is presumably the result of adaptive pressures favouring some codons over others. The underlying reason for this pressure is unknown, although a large number of possible driver mechanisms have been proposed; one of them is the decoding time. The standard model to calculate decoding time is the Gromadski- Rodnina model. Yet, recently, there have been a number of studies arguing to the effect that this conventional speed-model is not relevant to understand the dynamics of translation. However, results remain inconclusive so far. This contribution takes a novel approach to address this issue based on comparing mRNA with random synonymous variants to estimate the evolutionary pressures that have acted on the transcriptome. It emerges that over 70%of ORFs have been subject to a strong selection pressure for translation speed and that there is also a strong selection pressure for the avoidance of traffic jams. Finally, it is also shown that both homogeneous and very heterogeneous transcripts are over-represented. These results corroborate the validity of the Gromadski-Rodnina model.

Amorphous computing is a non-standard computing paradigm that relies on massively parallel execution of computer code by a large number of small, spatially distributed, weakly interacting processing units. Over the last decade or so, amorphous computing has attracted a great deal of interest both as an alternative model of computing and as an inspiration to understand developmental biology. A number of algorithms have been developed that can take advantage of the massive parallelism of this computing paradigm to solve specific problems. One of the interesting properties of amorphous computers is that they are robust with respect to the loss of individual processing units, in the sense that a removal of some of them should not impact on the computation as a whole. However, much less understood is to what extent amorphous computers are robust with respect to minor disturbances to the individual processing units, such as random motion or occasional faulty computation short of total component failure. In this article we address this question. As an example problem we choose an algorithm to calculate a straight line between two points. Using this example, we find that amorphous computers are not in general robust with respect to Brownian motion and noise, but we find strategies that restore reliable computation even in their presence. We will argue that these strategies are generally applicable and not specific to the particular AC we consider, or even specific to electronic computers.

Codon usage bias (CUB) is the well-known phenomenon that the frequency of synonymous codons is unequal. This is presumably the result of adaptive pressures favouring some codons over others. The underlying reason for this pressure is unknown, although a large number of possible driver mechanisms have been proposed. According to one hypothesis, the decoding time could be such a driver. A tacit assumption of this hypothesis is that faster codons lead to a higher translation rate which in turn is more resource efficient. While it is generally assumed that there is such a link, there are no rigorous studies to establish under which conditions the link between translation speed and rate actually exists. Using a computational simulation model and explicitly calculated codon decoding times, this contribution maps the entire range of dynamical regimes of translation. These simulations make it possible to understand precisely under which conditions translation speed and rate are linked.

Synonymous codons encode the same amino acid, but differ in other biophysical properties. The evolutionary selection of codons whose properties are optimal for a cell generates the phenomenon of codon bias. Although recent studies have shown strong effects of codon usage changes on protein expression levels and cellular physiology, no translational control mechanism is known that links codon usage to protein expression levels. Here, we demonstrate a novel translational control mechanism that responds to the speed of ribosome movement immediately after the start codon. High initiation rates are only possible if start codons are liberated sufficiently fast, thus accounting for the observation that fast codons are overrepresented in highly expressed proteins. In contrast, slow codons lead to slow liberation of the start codon by initiating ribosomes, thereby interfering with efficient translation initiation. Codon usage thus evolved as a means to optimise translation on individual mRNAs, as well as global optimisation of ribosome availability.

We designed Adaptive Neuromorphic Architecture (ANA) that self-adjusts its inherent parameters (for instance, the resonant frequency) naturally following the stimuli frequency. Such an architecture is required for brain-like engineered systems because some parameters of the stimuli (for instance, the stimuli frequency) are not known in advance. Such adaptivity comes from a circuit element with memory, namely mem-inductor or mem-capacitor (memristor's sisters), which is history-dependent in its behavior. As a hardware model of biological systems, ANA can be used to adaptively reproduce the observed biological phenomena in amoebae.

Translation in baker's yeast involves the coordinated interaction of 200 000 ribosomes, 3 000 000 tRNAs and between 15 000 and 60 000 mRNAs. It is currently unknown whether this specific constellation of components has particular relevance for the requirements of the yeast proteome, or whether this is simply a frozen accident. Our study uses a computational simulation model of the genome-wide translational apparatus of yeast to explore quantitatively which combinations of mRNAs, ribosomes and tRNAs can produce viable proteomes. Surprisingly, we find that if we only consider total translational activity over time without regard to composition of the proteome, then there are many and widely differing combinations that can generate equivalent synthesis yields. In contrast, translational activity required for generating specific proteomes can only be achieved within a much more constrained parameter space. Furthermore, we find that strongly ribosome limited regimes are optimal for cells in that they are resource efficient and simplify the dynamics of the system.

Motivation: Much is now known about the mechanistic details of gene translation. There are also rapid advances in high-throughput technologies to determine quantitative aspects of the system. As a consequence-realistic and system-wide simulation models of translation are now feasible. Such models are also needed as devices to integrate a large volume of highly fragmented data known about translation. Software: In this application note, we present a novel, highly efficient software tool to model translation. The tool represents the main aspects of translation. Features include a representation of exhaustible tRNA pools, ribosome–ribosome interactions and differential initiation rates for different mRNA species. The tool is written in Java, and is hence portable and can be parameterized for any organism. Availability: The model can be obtained from the authors or directly downloaded from the authors' home-page (http://goo.gl/JUWvI).

Protein synthesis translates information from messenger RNAs into functional proteomes. Because of the finite nature of the resources required by the translational machinery, both the overall protein synthesis activity of a cell and activity on individual mRNAs are controlled by the allocation of limiting resources. Upon introduction of heterologous sequences into an organism—for example for the purposes of bioprocessing or synthetic biology—limiting resources may also become overstretched, thus negatively affecting both endogenous and heterologous gene expression. In this study, we present a mean-field model of translation in Saccharomyces cerevisiae for the investigation of two particular translational resources, namely ribosomes and aminoacylated tRNAs. We firstly use comparisons of experiments with heterologous sequences and simulations of the same conditions to calibrate our model, and then analyse the behaviour of the translational system in yeast upon introduction of different types of heterologous sequences. Our main findings are that: competition for ribosomes, rather than tRNAs, limits global translation in this organism; that tRNA aminoacylation levels exert, at most, weak control over translational activity; and that decoding speeds and codon adaptation exert strong control over local (mRNA specific) translation rates.

Gene networks can often be interpreted as computational circuits. This article investigates the computational properties of gene regulatory networks defined in terms of the speed and the accuracy of the output of a gene network. It will be shown that there is no single optimal set of parameters, but instead, there is a trade-off between speed and accuracy. Using the trade-off it will also be shown how systems with various parameters can be ranked with respect to their computational efficiency. Numerical analysis suggests that the trade-off can be improved when the output gene is repressing itself, even though the accuracy or the speed of the auto-regulated system may be worse than the unregulated system.

We analyse the trade-off between the speed with which a gene can propagate information, the noise of its output and its metabolic cost. Our main finding is that for any given level of metabolic cost there is an optimal trade-off between noise and processing speed. Any system with a non-vanishing leak expression rate is suboptimal, i.e. it will exhibit higher noise and/or slower speed than leak-free systems with the same metabolic cost. We also show that there is an optimal Hill coefficient h which minimizes noise and metabolic cost at fixed speeds, and an optimal threshold K which minimizes noise.

Type 1 fimbriae are a known virulence factor in a number of pathogenic enterobacteriaceae, including Salmonella, Shigella and E. coli. Yet, they are also expressed by some commensal strains, notably of E. coli. One hypothesis of the role of fimbriae in commensals is that they evoke a small but tolerable host immune response in order to have the host release sialic acid, which is a valuable nutrient. Genetic evidence suggests that sialic acid down-regulates fimbriation. This has been believed to enable the cells to reduce virulence when the host response is increasing, thus avoiding a full activation of host defenses. In this article we assess the plausibility of this hypothesis using mathematical models. Our models lead us to two main conclusions: A slight activation of host defenses is only possible with a carefully tuned set of parameters, whereas under a wide range of parameters and assumptions, the model predicts the host defenses to be activated to at least half their potential in response to fimbriation. Secondly, the fact that fimbriation is suppressed by sialic acid seems irrelevant for the global qualitative properties

We present a differential equation model and a computer simulation of particle de-agglomertion in suspensions. The models are qualitative in nature, that is they do not assume specific parameters. Instead, this article investigates whether qualitative features of a series of transient volume–size distributions can give an insight into the underlying dynamics of particle de-agglomeration. We find that this is indeed possible and provide a table summarizing how various breakdown processes can be distinguished by considering the qualitative features of the transient volume–size distributions only

We present three models of how transcription factors (TFs) bind to their specific binding sites on the DNA: a model based on statistical physics, a Markov-chain model and a computational simulation. Comparison of these models suggests that the effect of non-specific binding can be significant. We also investigate possible mechanisms for cooperativity. The simulation model suggests that direct interactions between TFs are unlikely to be the main source of cooperativity between specific binding sites, because such interactions tend to lead to the formation of clusters on the DNA with undesirable side-effects.

Escherichia coli expresses type-I fimbriae; these are protrusions from the outer cell wall and have been identified as a virulence factor. They are also expressed by commensal strains of E. coli although (at any one time) only by a small proportion of the population. The orthodox interpretation of this is that fimbriation is regulated so as (i) to trigger a host-based release of nutrients in the form of inflammation signals by slightly activating host defenses and (ii) while avoiding a full scale inflammatory response. This article presents a number of computer simulations of the evolution of fimbriae to scrutinize the evolutionary plausibility of this orthodox view. It turns out that these simulations suggest a revised interpretation of the fimbriae mediated parasitehost interaction. Rather than being a passive victim the host is actively providing a niche that evolutionary favors less virulent parasites. The article closes with a number of testable predictions of this model.

While complex systems have been studied now for more than two decades, there still is no agreement on what complexity actually is. This lack of a definition might be a problem when asking questions about the evolution of complexity. In this article criteria against which candidate measures of complexity can be assessed are discussed. The main conclusion of this article is that because of the absence of a basic consensus on what complexity is, there is no criterion that can be used to decide whether or not a proposed measure actually measures complexity. The main recommendation is to abandon complexity as a formal notion; instead, research into the evolution of complexity should use well-understood proxy notions (as is sometimes done in the literature). For the time being complexity should remain an informal notion. Research into evolutionary trends of these proxy notions might eventually lead to an emergent community consensus on what complexity is.

Novelty detection is a machine learning technique which identifies new or unknown information in data sets. We present three evolutionary algorithms, a simple genetic algorithm, NEAT and FS-NEAT, for the the task of optimising the structure of an illustrative dynamic predictive coding neural network to improve its performance over stimuli from a number of artificially generated visual environments. We find that NEAT performs more reliably than the other two algorithms in this task and evolves the network with the highest fitness. However, both NEAT and FS-NEAT fail to evolve a network with a significantly higher fitness than the best network evolved by the simple genetic algorithm. The best network evolved demonstrates a more consistent performance over a broader range of inputs than the original network. We also examine the robustness of this network to noise and find that it handles low levels reasonably well, but is outperformed by the illustrative network when the level of noise is increased.

A group-selection model for the evolutionary origin of phase-variation in E. coli is proposed. Populations of commensal strains of E. coli populating mammalian hosts modulate its immune defenses through population-level control of the expression of fimbriae. At any time only a proportion of the population expresses these cell-surface adhesins. Collectively they elicit a host-based nutrient release if the fimbriae expression is low. Too high levels of fimbriation would provoke an inflammatory response and thus intolerable conditions for the cells. The optimal level of fimbriation is a group property and its evolution is difficult to explain by naive individual selection scenarios. This article presents a computational model to simulate the evolution of fimbriae. The two main conclusions of this contribution are: (i) the evolution of this group property requires the population to be partitioned into weakly interacting sub-populations. (ii) Given certain scenarios evolution consistently under-performs, in the sense that it does not find the optimal level of fimbriation.

This article reports on experimental evidence that an Escherichia coli nanR mutant shows inhibited growth in N-acetylneuraminic acid. This effect is prevented when inocula are grown in an excess of glucose, but not in an excess of glycerol. The nanATEK operon is controlled by catabolite repression, suggesting that diminished expression of the nanATEK operon in the presence of glucose explains the inocula effects. Neither double nanR-nagC nor nanR dam mutants show growth inhibition in the presence of N-acetylneuraminic acid. A theoretical model of N-acetylneuraminic acid metabolism (i.e., in particular of the nanATEK and nagBACD operons) is presented; the model suggests an interpretation of this effect as being due to transient high accumulations of GlcNAc-6P in the cell. This accumulation would lead to suppression of central metabolic functions of the cell, thus causing inhibited growth. Based on the theoretical model and experimental data, it is hypothesised that the nanATEK operon is induced in a two-step mechanism. The first step is likely to be repressor displacement by N-acetylneuraminic acid. The second stage is hypothesised to involve Dam methylation to achieve full induction.

Robert Rosen's central theorem states that organisms are fundamentally different to machines, mainly because they are ''closed with respect to efficient causation''. The proof for this theorem rests on two crucial assumptions. The first is that for a certain class of systems (''mechanisms'') analytic modeling is the inverse of synthetic modeling. the second is that aspects of machines can be modeled using relational models and that these relational models are themsleves refined by at least one analytic model. We show that both assumptions are unjustified. We conclude that these results cast serious doubts on the validity of Rosen's proof.

In a recent article in 'Artificial Life', Chu and Ho suggested that Rosen's central result about the simulability of living systems might be flawed. This argument was later declared ''null and void'' by Louie. In this article the validity of Louie's objections are examined.

The 'fim' system in 'E. coli' controls the expression of type-1 fimbriae. These are hair-like structures that can be used to attach to host cells. Fimbriation is controlled by a mechanism called ''orientational control''. We present two families of models for orientational control to understand the details of how it works. We find that the main benefits of orientational control are that (i) it allows rapid adjustment of fimbriation levels in response to a change of environmental conditions while (ii) keeping the overall frequencies with which a cell switches between the fimbriate state and the afimbriate state low. The main reason for the efficiency of orientational control in regulation of fimbriation levels is that it keeps the system far from its steady state.

Current opinion considers two main hypotheses for the evolutionary origin of uptake signal sequences in bacteria: one model regards the uptake signal sequence (USS) as the result of biased gene conversion, whereas the second model views the USS as a molecular tag that evolved as an adaptation. In this article, we present various computational models that implement specific versions of those hypotheses. Those models show that the two hypothesis are not necessarily as opposed to each other as may appear at first glance.

One of Robert Rosen's main contributions to the scientific community is summarized in his book 'Life itself'. There Rosen presents a theoretical framework to define living systems; given this definition, he goes on to show that living systems are not realisable in computational universes. Despite being well known and often cited, Rosen's central proof has so far not been evaluated by the scientific community. In this article we review the essence of Rosen's ideas leading up to his rejection of the possibility of real artificial life in silico. We also evaluate his arguments and point out that some of Rosen's central notions are ill- defined. The conclusion of this article is that Rosen's central proof is wrong.

The DNA of some naturally competent species of bacteria contains a large number of evenly distributed copies of a short (about 10bp long) sequence. This highly overrepresented sequence is believed to be an uptake signal sequence (USS) that helps bacteria to take up DNA selectively from (dead) members of their own species. For some time it has been assumed that the USS evolved in order to enable bacteria to distinguish between conspecific and non-conspecific DNA fragments (the preference-first hypothesis). Recently, Redfield suggested that this hypothesis is not in fact realistic, as it would require biologically implausible group selection. In this article we present a model designed to demonstrate the emergence of similar USSs in a population of simulated evolving agents. We use this model to examine the conditions under which a USS will emerge in a preference-first scenario.

The term "precision medicine" is used to denote existing practices as well as how medical research and practice are imagined to become in the future. One important element of the imaginaries of precision medicine is the development of systems biology and computational models with the promise of numerical precision and conceptual rigour. If precision medicine were to become an exact science that relies on computational models, it might increase precision in diagnosis and treatment, specifying the right drug to the right patient at the right time. It should be noted, though, that computational models require explicit specification of the properties and boundaries of the system to be modelled, whereas cells, tissues and patients are predominantly open systems in their natural state. Accordingly, such models risk being precisely wrong instead of approximately right. Right and wrong, however, are value judgements that depend upon the aims and scope of the scientific and medical enterprise. In order for medicine to become an exact science, cells, tissues and patients would have to be reconceived and/or reconfigured as relatively closed systems with relatively deterministic behaviour. The realization of precision medicine as an exact science may thus be accompanied by a transition from a world of complex natural life to a world of reduced life or a simple delay of death; a transition to be likened with the crossing of the Styx.

Mathematical and computational modelling is now firmly established as an important tool in life sciences. However, it is not yet established as a source of biological knowledge on an equal footing with experimentation. Here we argue that there are now substantial opportunities for a new theoretical biology that not only benefits from experimental research but also influences this research. We argue that such a new theoretical biology would need to find the appropriate level of abstraction and combine aspects of Artificial Life and theoretical biophysics. We also provide an outline of a research agenda that could lead to experimentally testable insights into biological systems.

In this contribution we seek to evolve viable parameter values for a small-scale biological network motif concerned with bacterial nutrient uptake and metabolism. We use two different evolutionary approaches with the model: implicit and explicit. Our results reveal that significantly different characteristics of both efficiency and timescale emerge in the resulting evolved systems depending on the which particular approach is used.

The genetic bi-stable switch (toggle switch) consists of two genes which repress one another through their synthesised proteins. In the stochastic bi-stable case, the toggle switch exhibits noise induced flips. Using simulation based techniques, we identified three methods for reducing the random flipping probability, namely: (i) increasing the system size, (ii) increasing the synthesis rate, and (iii) increasing the Hill coefficient. Our heat-maps showed that for large parts of parameter space the toggle switch displays extremely low noise induced flipping probability, and consequently high robustness. We also analysed the noise analytically. The area of validity of the analytical approach was found to be constrained to low Hill coefficients and high synthesis rate (both conditions within biological realistic parameters). Most importantly, we observed that increasing the synthesis rate is more efficient in reducing the noise compared to the increase in system size, for systems affected by negative feedback.

In various nuances of evolutionary algorithms it has been observed that variable sized genomes exhibit large degrees of redundancy and corresponding undue growth. This phenomenon is commonly referred to as bloat. The present contribution investigates the role of crossover operators as the cause for length changes in variable length genetic algorithms and linear GP. Three crossover operators are defined; each is tested with three different fitness functions. The aim of this article is to indicate suitable designs of crossover operators that allow efficient exploration of designs of solutions of a wide variety of sizes, while at the same time avoiding bloat.

Recently there has been significant interest in evolving genetic regulatory networks with a user-determined behaviour. It is unclear whether or not artificial evolution of biochemical networks can be of direct benefit for or biological relevance to systems biology. This article highlights some pitfalls when concluding from artificially evolved genetic regulatory networks to real networks. This article also gives a (brief) review of some previous attempts to evolve genetic regulatory networks with oscillatory behaviour; it also describes a new system to evolve networks and describes the networks that have been evolved. These networks seem to be very diverse sharing no apparent common motifs either with one another or with their real-life counterparts

In this article we present an agent-based simulations of the spread of a vector borne disease in a population with limited mobility. The model assumes two types of agents, namely "vectors" and "people agents"; infections can only be transmitted between agents of different type. We discuss how the infection levels of the population depend on the mobility of agents.

In a recent article Chu et al. presented a computational model investigating the evolutionary origin of so-called uptake signal sequences in bacteria. In that contribution the authors used an agent-based approach. The main aim of this article is to understand the fitness-landscape on which the agents operate. We propose such a fitness- landscape and discuss its implications. This opens the possibility to use GAs for future simulations rather than the computationally expensive agent-based model.

Mathematical modelling of the spread of infectious diseases is a well established field with high practical importance. Underlying most analytical approaches is the assumption of "perfect mixing", that is the idea that the spatial structure of the population can be neglected. This assumption is crucial to the solvability of the models, but can be dropped when using computational models instead of analytical approaches. Using methods from Artificial Life, we investigate under which conditions the perfect mixing assumption becomes a good approximation to describe the spread of vector borne disease in a population with spatial structure.

We present an iterative neural synthesis approach to train Convolutional Spiking Neural Networks for classification problems. Unlike previous neural synthesis methods which primarily compute the neuron firing rates, our method is designed to compute multiple spikes at arbitrary timings. As such, our approach is directly applicable to spatio-temporal problems using spiking network models. In our approach, each weight update is formulated as a linear Constraint Satisfaction Problem, which can then be solved using existing numerical techniques. On the MNIST, EMNIST, and ETH-80 image classification benchmarks, our approach demonstrates competitive with other models in the literature, while requiring relatively few training samples to converge to a good solution.

The digital processing of electronic documents is widely exploited across many domains to improve the efficiency of information extraction. However, paper documents are still largely being used in practice. In order to process such documents, a manual procedure is used to inspect them and extract the values of interest. As this task is monotonous and time consuming, it is prone to introduce human errors during the process. In this paper, we present an efficient and robust system that automates the aforementioned task by using a combination of machine learning techniques: optical character recognition, object detection and image processing techniques. This not only speeds up the process but also improves the accuracy of extracted information compared to a manual procedure.

Embedded systems acquire information about the real world from sensors and process it to make decisions and/or for transmission. In some situations, the relationship between the data and the decision is complex and/or the amount of data to transmit is large (e.g. in biologgers). Artificial Neural Networks (ANNs) can efficiently detect patterns in the input data which makes them suitable for decision making or compression of information for data transmission. However, ANNs require a substantial amount of energy which reduces the lifetime of battery-powered devices. Therefore, the use of Spiking Neural Networks can improve such systems by providing a way to efficiently process sensory data without being too energy-consuming. In this work, we introduce a low-powered neuron model called Integrate-and-Fire which exploits the charge and discharge properties of the capacitor. Using parallel and series RC circuits, we developed a trainable neuron model that can be expressed in a recurrent form. Finally, we trained its simulation with an artificially generated dataset of dog postures and implemented it as hardware that showed promising energetic properties.

When micro-organisms are in environments with multiple nutrients, they often preferentially utilise one first. A second is only utilised once the first is exhausted. Such a two-phase growth pattern is known as diauxic growth. Experimentally, this manifests itself through two distinct exponential growth phases separated by a lag phase of arrested growth. The dura- tion of the lag phase can be quite substantial. From an evolu- tionary point of view the existence of a lag phase is somewhat puzzling because it implies a substantial loss of growth op- portunity. Mutants with shorter lag phases would be prone to outcompete those with longer phases. Yet in nature, diauxic growth with lag phases appears to be a robust phenomenon. We introduce a model of the evolution of diauxic growth that captures the basic interactions regulating it in bacteria. We observe its evolution without a lag phase. We conclude that the lag phase is an adaptation that is only beneficial when fit- ness is averaged over a large number of environments.

The evolution of quantitative details (i.e. "parameter values") of biological systems is highly under-researched. We use evolutionary algorithms to co-evolve parameters for a generic but biologically plausible topological differential equation model of nutrient uptake. In our model, evolving cells compete for a finite pool of nutrient resources. From our investigations it emerges that the choice of values is very important for the properties of the biological system. Our analysis also shows that clonal populations that are not subject to competition from other species best grow at a very slow rate. However, if there is co-evolutionary pressure, that is, if a population of clones has to compete with other cells, then the fast growth is essential, so as not to leave resources to the competitor. We find that this strategy, while favoured evolutionarily, is inef- ficient from an energetic point of view, that is less growth is achieved per unit of input nutrient. We conclude, that competition can lead to an evolutionary pressure towards inefficiency.

A simplified stochastic model of bacterial nutrient uptake and metabolism is presented. An evolutionary algorithm is used to explore the parameter space of this model under three different conditions: Unlimited nutrient supply, a limited nutrient supply that is replenished and a limited nutrient supply that is replenished after a period of no nutrient. A comparison of evolved parameters shows that the solutions are specific to the particular parameters for which they have been evolved. Given such solutions, this article then explores the parameter space of the evolved solutions. This shows that there is a noise-speed-cost trade-off in parameters which manifests itself through a tension between the growth rate of the cell and the likelihood of the cell running out of nutrients.

Parameter values of biological systems have received comparatively little attention in the past. Particularly computational modellers tend to see them as necessary, but often illusive pieces of information that somehow needs to be inferred. This article is a first attempt to understand whether quantitative information about biological systems, i.e. kinetic parameters, reaction rates, or other constants contain valuable information about the adaptive pressures for which they have evolved. To do this, genetic algorithms are used to evolve parameters for a fixed network structure. The results indicate that the parameters are rather specific to the particular circumstances for which they have evolved. If this is also true for real biological systems, then this suggests that parameter values may be a valuable source of biological information.

We present a model of how transcription factors scan DNA to find their specific binding sites. Following the classical work of Winter et al. (1981), our model assumes two modes of transcription factor dynamics. Adjacent moves, where the proteins make a single step movement to one side, or short walks where the transcription factors slide along the DNA several binding sites at a time. The purpose of this article is twofold. Firstly, we discuss how such a system can be efficiently modeled computationally. Secondly, we analyse how the mean first binding times of transcription factors to their specific time depends on key parameters of the system.

Using a set of genetic logic gates (AND, OR and XOR), we constructed a binary full-adder. The optimality analysis of the full-adder showed that, based on the position of the regulation threshold, the system displays different optimal configurations for speed and accuracy under fixed metabolic cost. In addition, the analysis identified an optimal trade-off curve bounded by these two optimal configurations. Any configuration outside this optimal trade-off curve is sub-optimal in both speed and accuracy. This type of analysis represents a useful tool for synthetic biologists to engineer faster, more accurate and cheaper genes.

We present an individual agent-based model of transcription factor binding to DNA. We explicitly represent every single transcription factor and every single binding site in our simulation. In order to obtain statistically significant results from our model, it is necessary to perform a large number of simulations. This has previously been hampered by the computational demands of individual agent-based models. Here we present an approach to make such simulations feasible using approaches related to dynamic memory allocations in computer systems

We present a model of multi-level selection. By this we mean the idea that there are multiple units of selection each of which operates on a different hierarchical level. Concretely we consider here a model of 3 hierarchical levels and various selection scenarios of adaptive conflict between levels. The main finding of this contribution is that in order for selection at higher level units to be effective, it has to occur at a high frequency compared to low level selection. From this we conclude that multi-level selection is biologically not very plausible.

Novelty detection is a machine learning technique which identifies new or unknown information in data sets. We present our current work on the construction of a new novelty detector based on a dynamical version of predictive coding. We compare three evolutionary algorithms, a simple genetic algorithm, NEAT and FS-NEAT, for the task of optimising the structure of an illustrative dynamic predictive coding neural network to improve its performance over stimuli from a number of artificially generated visual environments. We find that NEAT performs more reliably than the other two algorithms in this task and evolves the network with the highest fitness. However, both NEAT and FS-NEAT fail to evolve a network with a significantly higher fitness than the best network evolved by the simple genetic algorithm. The best network evolved demonstrates a more consistent performance over a broader range of inputs than the original network. We also examine the robustness of this network to noise and find that it handles low levels reasonably well, but is outperformed by the illustrative network when the level of noise is increased.

This book presents a detailed introduction to the use of a wide range of software tools and modelling environments for use in the biosciences, as well as some of the fundamental mathematical background. The practical constraints and difficulties presented by each modelling technique are described in detail, enabling the researcher to determine quickly which software package would be most useful for their particular problem.

We are in the midst of a culture war between science and religion. In this struggle science portrays itself as the white knight of enlightenment truth defending humanity against the dark forces of religious fundamentalism, ignorance and gullibility that threaten free thinking and progress. But is there really the fundamental difference between science and religion that the culture warriors like us to believe? This book takes the reader on an inside journey through science showing how scientific beliefs are made. It will show science as a human activity that is shaped by power struggles, personal interests, cultural prejudices, beliefs and values...and yes, experimental data as well.

In this book we seek to provide a detailed introduction to a range of modeling techniques that are appropriate for modeling in biosciences. The book is primarily intended for bioscientists, but will be equally useful for anybody wishing to start modeling in biosciences and related fields. The topics we discuss include agent-based models, stochastic modeling techniques, differential equations and Gillespie's stochastic simulation algorithm. Included in the book are practical introductions to a number of useful tools, such as the Maxima computer algebra system, the PRISM model checker, and the Repast Simphony agent modeling environment.