Memoryless systems generate the class of all discrete systems

Automata are machines, which receive inputs, accordingly update their internal state, and produce output, are a common abstraction for the basic building blocks used in engineer-ing and science to describe and design complex systems. These arbitrarily simple machines can be wired together—so that the output of one is passed to another as its input—to form more complex machines. Indeed, both modern computers and biological systems can be described in this way, as assemblies of transistors or assemblies of simple cells. The complexity is in the network, i.e., the connection patterns between simple machines. The main result of this paper is to show that the range of simplicity for parts as compared to the complexity for wholes is in some sense complete: the most complex automaton can be obtained by wiring together direct-output memoryless components. The model we use—discrete-time automata sending each other messages from a ﬁxed set of possibilities—is certainly more appropriate for computer systems than for biological systems. However, the result leads one to wonder what might be the simplest sort of machines, broadly construed, that can be assembled to produce the behavior found in biological systems, including the brain.


Introduction
Automata represent systems that receive inputs, alter their internal states, and produce outputs. The state set of the automaton is to be interpreted as the set of all potential memories, or storable experiences. In automata theory, the state set is typically finite. In this case, one can view this memory capacity as limited. On the contrary, when the memory of the automaton is not assumed to be limited (human brain), or its capacity can always be extended (RAM-machines or Turing machines as models of computers in computation theory), the automaton should have an infinite state set.
In the theory of dynamical systems, we use a generalisation of automata in which the size of the state space is not restricted to finiteness, or even to countability. Dynamical systems with the behaviour of an automaton, that is, taking inputs in discrete time, are called discrete systems. The state space of such a system acts as a sort of memory of the inputs. Each input influences the current state of the automaton, and the current state is the result of the system's own form-how it deals with inputs-together with the system's history.
One can imagine a dynamical system whose state space is that of all possible input-histories; a new input simply appends to the existing history to become a new history. On the other hand, one can imagine the "opposite" kind of system: one that completely forgets the previous inputs. These systems are referred to as "simple-reflex" in [1, p. 49], reactive, or memoryless in this paper. The transitions of these automata depend only on the input, as no experience is stored. The system decides according to the current perception of the world, rather than current perception together with past perception. In fact, these memoryless systems could act by making a single distinction in the input-a yes/no Boolean response-and nothing more; we call these Boolean reactive systems.
In this paper, we will study the links between systems that have memory and those that do not. More precisely, we prove that systems with memory can be simulated by wiring together systems without memory. Our result provides a theoretical framework that supports artificial neural network approaches. Memory is carried by connections, and not only by individuals, within a compositional hierarchy of parts.
This article lies between two fields of mathematics: category theory and dynamical systems. In Section 2, we introduce all the background related to category theory and its use in the study IMTA-RR-2018-02-SC 2 2. BACKGROUND

Background in category theory
of discrete systems. Readers with a background in category theory are invited to skip Section 2.1. We need no more than the basic definitions of categories, functors, natural transformations, monoidal categories, and monoidal functors. Readers already familiar with the study of boxes and discrete systems from a category-theoretic point of view (as in [2]) can skip Section 2.2. Here again, we only need the basic understanding of C -typed-finite sets, C -boxes, wiring diagrams and discrete systems inside a C -box.
In Section 3, we introduce discrete systems and a specific mapping that will serve our purposes (Section 3.1). We then introduce two equivalence relations between discrete systems. Both are bigger than the usual bisimulation used in automata theory (in the sense of set inclusion). One corresponds to an external point of view; two systems are equivalent if they transform input streams into output streams in the same way (Section 3.2). The other relation corresponds to an internal point of view: two systems are equivalent if they have "the same structure" (in a sense that is defined in Section 3.3). We prove that these are just two perspectives on the same relation.
This equivalence relations plays a crucial role in the two results we show in Section 4. First, we show that any discrete system is equivalent to some wiring-together of memoryless systems (Section 4.2). Second, we show that any discrete system with a finite state set is equivalent to a combination of finitely many Boolean reactive systems (Section 4.3).

Notation
In this article, we will use the following notation.
Let N denote the set of all natural numbers, N {0, 1, 2, . . .}. By default, the variable n will refer to a natural number: n ∈ N. We will also see the integers n as their set-theoretic counterparts, that is 0 = ∅, and n = {0, 1, . . . , n − 1}; in that context, i ∈ n simply means i ∈ {0, . . . , n − 1}. Note that the set n contains exactly n elements and this is what really matters in this notation. When the size of a sequence (x 0 , x 1 , . . . , x n−1 ) does not matter, it will be denoted x, which makes it easier to write and read. If each x i is an element of the same set X, then we will write x ∈ X, instead of defining an n = length (x) and writing x ∈ X n . If x i ∈ X i for possibly different X i , and if there exists a compact notation X for X 0 × X 1 × · · · × X n−1 , then we will write x ∈ X too. Sets is the usual category of sets; it will be defined in section 2.1.

Background
In this chapter, we will present the background necessary for the understanding of this paper, namely that of category theory and dynamical systems. Both will be reduced to the absolute minimum used in this article.

Background in category theory
This section will present some basic notions about category theory. If the reader is already familiar with the notions of category, functor, natural transformation, product, monoidal category, and lax monoidal functor, they can skip directly to section 2.2.
The composition law is required to be associative: ∀A, B, C, D ∈ Ob C , ∀ f : A → B and g : B → C and h : . Identity morphisms are required to act like identities: By abuse of notation, we will often write C ∈ C instead of C ∈ Ob C , and ( f : C → D) ∈ C instead of ( f : C → D) ∈ Mor C . It should be clear from the way these are written that we refer respectively to an object and a morphism.
In the rest of the article, a category C will be described according to the following presentation: Objects: An object in C is...

Morphisms:
A morphism in C is...

Composition:
The composition law for morphisms is... Usually, the description of morphisms suffices to implicitly define dom and cod, as in the following example.
Example 2.2. We define the category Sets as the following: Objects: An object in Sets is any set Morphisms: A morphism in Sets is any function f : A → B Identities: An identity morphism is an identity function id A : A → A Composition: The composition law for morphisms is the usual composition of functions Similarly, we can define the category FinSets whose objects are the finite sets.
We also define mappings somewhat similar to functions, or homomorphisms, between categories. [3]). Let C and D be categories.

Definition 2.3 (Functor
A functor F : C → D is a mapping from C to D such that:

Background in category theory
In other words, a functor F : C → D sends the objects (resp. morphisms) in C to objects (resp. morphisms) in D, preserving domains and codomains of morphisms, as well as identities and composition.
Functors are maps between categories; there also exist maps between functors. Definition 2.4 (Natural transformation [3]). Let C and D be two categories, and let F, G : C → D be functors. A natural transformation θ : F → G consists of a collection of morphisms (θ C : F(C) → G(C)) C ∈Ob C such that, for all C, D ∈ C , and for all h : C → D, the following square commutes: For each object C ∈ C , the morphism θ C is called the C-component of θ.
Depending on the context, and for the sake of readability, the C-component of a natural transformation θ can be written θ C as above (C as an index) or θ(C) (C as a parameter). Definition 2.5 (Natural isomorphism [3]). A natural transformation θ : F → G is called a natural isomorphism when all of its components θ C : F(C) → G(C) are isomorphisms.
Basic constructions inside categories include the product of a pair of objects, defined as follows.
Definition 2.6 (Product [3]). Let C be a category and let A 1 and A 2 be objects of C .
A product of A 1 and A 2 , written A 1 × A 2 , is an object in C , together with two morphisms π 1 : A 1 × A 2 → A 1 and π 2 : A 1 × A 2 → A 2 such that, for all objects P with two morphisms p 1 : P → A 1 and p 2 : P → A 2 , there exists a unique morphism u : P → A 1 × A 2 such that π 1 • u = p 1 and π 2 • u = p 2 , that is, such that the following diagram commutes: We call π 1 , π 2 projections, and we denote u by u = (p 1 , p 2 ).
The above can be generalized from n = 2 to any n ∈ N; the result is called an n-ary product. The 1-ary product of A 1 is just A 1 and the projection π 1 : A 1 → A 1 is the identity. The 0-ary product is an object * such that for all objects P, there exists a unique morphism u : P → * ; it is called a terminal object Note that products need not exist in arbitrary categories, however they do exist in many categories of interest here. For example, in Sets, 2-ary products are given by the usual product of sets, and 0-ary products is any set { * } with one element. Definition 2.7 (Category with finite products). A category C is said to be a category with finite products when ∀n ∈ N, ∀A 1 , . . . , A n objects of C , the product A 1 × · · · × A n exists.

Background in category theory
An important class of categories is that of monoidal categories.

Definition 2.8 (Monoidal category [4]).
A monoidal category is a 6-tuple (C , ⊗, I, a, r, l), consisting of a category C , a functor ⊗ : C × C → C , an object I ∈ C , and three natural isomorphisms a, r, and l of the following form such that, for all objects A, B, C and D of C , the following rules hold: 1. The following diagram commutes: The following diagram commutes too: We also say that (⊗, I, a, r, l) forms a monoidal structure on C .
Roughly, a monoidal category is a category with an operation ⊗ which can be seen as associative (items 1 and 2), and a distinguished element I that behaves like a unit for the operation (items 2, 3 and 1).
The category (Sets, ×, 1, a, r, l) is monoidal for the usual product of sets, where a, r, l are the obvious isomorphisms and 1 is the singleton 1 = { * }.
then a is defined as the natural transformation a : F → G. The same holds for r and l. Define:

Background in category theory
then r is the natural transformation r : R → id C and l is the natural transformation l : L → id C . Definition 2.10 (Symmetric monoidal category [4]). A symmetric monoidal category is a 7-tuple (C , ⊗, I, a, r, l, s) such that (C , ⊗, I, a, r, l) is a monoidal category, and s is a natural isomorphism s(A, B) : A ⊗ B → B ⊗ A such that, for all A, B and C of C , the following diagrams commute: A symmetric monoidal category has an associative and commutative law, with a unit object. Again, Sets is a symmetric monoidal category. Definition 2.11 (Lax monoidal functor [5]). Let (C , , I, a C , r C , l C ) and (D, ⊗, J, a D , r D , l D ) be monoidal categories.
A monoidal functor between C and D is a 3-tuple (F, σ, σ ) where: such that, for all A, B, C ∈ C , the following three diagrams commute: IMTA-RR-2018-02-SC 7

Boxes and wiring diagrams
The pair (σ, σ ) is called the coherence maps of F. We sometimes refer to σ as the first coherence map of F. Remark 2.12. Just like we call R a field without clarifying its two laws and its two units, we often write (C , ⊗, I), omitting the natural isomorphisms a, r and l, because they are only a matter of "bookkeeping". We may even write C for a monoidal category when the context makes it clear what the unit and monoidal product are.

Boxes and wiring diagrams
We will now apply the categorical framework to build discrete systems. Our approach is different from the one in [6]. The dynamical systems presented here are defined as a generalisation of automata whose input and output spaces are predetermined. We will define a category of lists, a category of boxes, and diverse operations on them.
In this section, C will be any category with finite products (typically Sets). Most of the following notions were already defined in [2]; we only recall them without proving their properties. We also give examples in order to help for comprehension.

The category of typed finite sets
Before defining proper boxes, we need to define the notion of input and output ports. These will eventually be the sides of our boxes. Definition 2.13 (Category of C -typed finite sets [2]). The category TFS C of C -typed finite sets is defined as follows: Objects: An object is any pair (P, τ) such that P is a finite set and τ : P → Ob C is a function Morphisms: A morphism from (P, τ) to (P , τ ) is a function γ : P → P such that τ = τ • γ Identities: The identity morphism on (P, τ) is the identity function of the set P Composition: The composition of morphisms is the usual composition of functions An object in TFS C is called a C -typed finite set; a morphism in TFS C is called a C -typed function.
We can rewrite a C -typed finite set (P, τ) as the finite sequence τ(p 0 ), . . . , τ(p n−1 ) , where P = {p 0 , . . . , p n−1 }. A C -typed finite set is simply a list of objects in C , indexed by a finite set P. If C = Sets, a Sets-typed finite set is a list of sets.
Example 2.14. Let A, B and C be objects of C and consider the following two C -typed finite sets: The list A, B can be obtained from the list B, C, A by taking its third and first elements in this order. A C -typed function from (2, τ 2 ) to (3, τ 3 ) could be γ : 2 → 3 such that γ(0) = 2 and γ(1) = 0.
We let the reader find the morphisms (4, τ 4 ) → (3, τ 3 ) and the morphism (4, τ 4 ) → (2, τ 2 ), that is, the morphisms that transform the list B, C, A into the list A, A, A, B and the (unique) morphism that transforms the list A, B into the list A, A, A, B .
Proposition 2.17. The category TFS C has the following properties: The sum of C -typed finite sets is a coproduct. There is only one C -typed finite set (P, τ) where P = ∅. We denote it by 0. TFS C has a symmetric monoidal structure for the sum +, with 0 as the unit.

Dependent products
In this subsection, we define the dependent product functor. If a C -typed finite set can be viewed as a list of objects of C , then the dependent product of this list is simply the product of its elements. Definition 2.18 (Dependent product [2]). We define the dependent product as the functor − : TFS op C → C such that: Action on objects: (P, τ) = p ∈P τ(p) Action on morphisms: If γ : (P, τ) → (P , τ ), then γ : (P , τ ) → (P, τ) is defined as the function γ : p ∈P τ (p ) → p ∈P τ(p) such that ∀ a p p ∈P ∈ (P , τ ) , γ a p p ∈P = a γ(p) p ∈P .

Boxes and wiring diagrams
The interpretation of the dependent product is actually quite straightforward: the dependent product of a C -typed finite set, viewed as a list, is the product of the elements of the list in the same order as they appear in the list.
We remind that C has finite products; as a consequence, the dependent product always exists.
Example 2.19. Consider the same A, B and C objects of C and C -typed finite sets as in Example 2.14: The dependent products of these C -typed finite sets are: In order to see what the dependent product does to morphisms, take a morphism γ : 2 → 4 that converts the list A, A, A, B to A, B , for example the morphism defined by γ(0) = 0 and γ(1) = 3. Its dependent product γ will be the function (4, We let the reader find the other dependent products as an exercise.
The dependent product is thus a functor that packages the usual operations of diagonal Proposition 2.20. There is a natural isomorphism (P 0 , τ 0 ) + (P 1 , τ 1 ) (P 0 , τ 0 ) ×(P 1 , τ 1 ) ; in other words, the dependent product functor sends coproducts in TFS C to products in C .
Proof. See [2]. This property is also quite intuitive: if one views the coproduct in TFS C as the concatenation of lists, and the dependent product as the product of the elements of the list, then the dependent product of the concatenation of two lists is the product of the dependent products of each lists.

The category of boxes and wiring diagrams
The category TFS C is not the main purpose of this article; however its properties will be useful for the rest of this article.
In the following, by abuse of notation, we will write X ∈ TFS C for (X, τ), and X for (X, τ) . Definition 2.21 (C -box [2]). We call C -box any pair X = X in , X out ∈ TFS C × TFS C .
A C -box is a pair of C -typed finite sets X in , X out , where X in represent the list of inputs ports, and X out represent the list of outputs ports.
Example 2.22. From the typed finite sets in Example 2.14, we can build the following C -boxes:

Boxes and wiring diagrams
These C -boxes are represented here: In the rest of the paper, the ports will no longer be labelled, for the sake of readability.
A wiring diagam ϕ : X → Y is a pair of C -typed functions ϕ in , ϕ out such that: The C -typed function ϕ in tells what feeds the input ports of the box X: each input port of X is either connected to an input port of Y or to an output port of X (in case of feedback); the C -typed function ϕ out tells what feeds the output ports of Y : each output port of Y is connected to some output port of X.
Example 2.24. Given B 2,2 and B 2,4 defined in Example 2.22, the wiring diagrams ϕ : B 2,2 → B 2,4 will have the following form: We can build specific wiring diagrams such as: Let us consider a wiring diagram ψ : B 4,3 → B 2,2 (defined in Example 2.22). It will look like:

Boxes and wiring diagrams
We can build specific wiring diagrams: What about the reverse wiring diagram ρ : B 2,2 → B 4,3 ? It will look like: There is no problem with the first C -typed function, but we already know that there is no C -typed function (3, τ 3 ) → (2, τ 2 ) (cf. Example 2.14).
We can now compose the wiring diagrams: Definition 2.25 (Composition of wiring diagrams [2]). Let ϕ : X → Y and ψ : Y → Z be two wiring diagrams. We define their composition, denoted ψ • φ, as the pair (ψ • φ) in , (ψ • φ) out , where (ψ • φ) in is defined such that the following diagram commutes: and (ψ • φ) out is defined such that the following diagram commutes: Definition 2.27 (Category of C -boxes and wiring diagrams [2]). The category W C of C -boxes and wiring diagrams is defined as follows: Objects: An object in W C is a C -box

Morphisms:
A morphism between two C -boxes X and Y is a wiring diagram φ : X → Y Identities: An identity morphism on X is the identity wiring diagram Composition: The composition of wiring diagrams is the composition defined in definition 2.25

Monoidal structure of the category of boxes
The category W C has a monoidal structure for the parallel composition of boxes, that corresponds to the intuitive idea of parallelising boxes.
Definition 2.28 (Parallel composition of boxes [2]). Let X = X in , X out and Y = Y in ,Y out be two C -boxes.
The parallel composition, or sum, of X andY , denoted X Y , is the box X Y = X in + Y in , X out + Y out , where + is the sum of C -typed finite sets (cf. Definition 2.15).
The parallel composition of two C -boxes summarises to the concatenation of both input ports, and both output ports.  [2]). Let ϕ : X → Y = ϕ in , ϕ out and ψ : Y → Z = ψ in , ψ out be two wiring diagrams.
Example 2.31. Using the notations of Example 2.24, we can build ϕ i ψ i : Proposition 2.32. The category W C has the following properties: The closed box , defined as = (0, 0), where 0 is C -typed finite set (∅, ∅ → C ) defined in 2.17, is the unit for the sum of boxes . W C has a symmetric monoidal structure for the sum of boxes , with as the unit.

Dependent product of boxes
The aim of this section is to extend the notion of dependent product (Definition 2.18) to C -boxes and wiring diagrams.
Definition 2.33 (Dependent product of a C -box [2]). The dependent product X of the C -box X = X in , X out is the pair X = X in , X out .
Definition 2.35 (Dependent product of wiring diagrams [2]). The dependent product X of the wiring diagram ϕ : X → Y is the pair ϕ = ϕ in , ϕ out .
Remark 2.38. The dependent product of C -boxes and wiring diagrams could be described in terms of monoidal functors; however the codomain of this functor is not C × C as expected, but a category that has the same objects (pairs of objects (A, B) of C ) but whose morphisms are pairs of morphisms f in , f out : in C and f out is the morphism f out : B 0 → B 1 in C . The composition law is the one given in Proposition 2.37.
Until now, we have only defined a category of C -boxes, with interesting properties. These C -boxes are exactly as their name suggests: empty boxes. The extension of the dependent product to C -boxes is a necessary step in order to define the "inhabitants" of C -boxes.

Discrete systems and their equivalences
In this chapter and in the rest of this paper, we will consider the special case where C = Sets. Thus, in general, we will simply call "boxes" what we introduced as "Sets-boxes". We denote the symmetric monoidal category of boxes as W Sets .

Definition and basic properties
The notions introduced in this section come from [2]. The properties stated here are proven in the same article. [2]). Let X = X in , X out ∈ W Sets be a box.

Definition 3.1 (Discrete systems
A discrete system for the box X, or discrete system for short, is a 4-tuple F = S F , f rdt , f upd , s F ,0 where: S F ∈ Sets is the state set of F f rdt : S F → X out is its readout function f upd : X in × S F → S F is its update function s 0 ∈ S F is its initial state We denote by DS (X) the set of all discrete systems for the box X.
Its dependent product is = p ∈ ∅ τ(p), p ∈ ∅ τ(p) (1 , 1 ), where 1 is any typed finite set of the form (1, τ : 1 → Sets). As a consequence, we have: In other words, an inhabitant of a closed box is a dynamical system with no inputs and no outputs, just a set S and a function S → S. Remark 3.3. From a set-theoretic point of view, DS (X) is too big to be a set. A potential solution is to define the DS (X) within a set big enough for our purposes; for example, the set V ω 1 from the von Neumann hierarchy of sets, which contains the usual sets, vector spaces, measurable spaces, Hausdorff spaces, fields, etc. used in mathematics (V ω×2 suffices [7, Lemma 2.9]).
In the following, we will continue to write DS (X) (and similarly for mappings) with the state set in S F ∈ Sets for the sake of understandability, but in case set-theoretic problems emerge, we should not write S F ∈ Sets but S F ∈ V ω 1 .
Note that discrete systems can be viewed as a generalisation of automata. They have no final states, the transition function is always a function, i.e. all discrete systems are deterministic, the input alphabet can be infinite, and the transition function is always defined on every input and every state. Discrete systems are not automata that recognize a language, but rather, automata that take any input stream and return an output stream based on the states it transitioned to; that is, discrete systems are a generalisation of transducers as defined in [8]. Alternatively, discrete systems exactly correspond to the sequential automata in [9].
Example 3.4. For this example, we generalise the notation seen in Definition 2.13 to the set TFS R of R-typed finite sets, seen as lists of real numbers, that is, finite sequences of real numbers: Here, this is a set, not a category; besides, we use n ∈ N instead of P ∈ FinSets so that we define a set. As a discrete category (having only identity morphisms), it is equivalent to TFS C R obtained by also considering R as a discrete category.
We also generalise the sum of finite sequences, seen as the concatenation.
Consider the box X 0 = ( R , R ) ∈ W Sets . We define the following discrete system F 0 ∈ DS (X 0 ): , a 0 , . . . , a n−1 ) → a 0 , . . . , a n−1 , a f rdt 0 : TFS R → R a 0 , . . . , a n−1 → max ( a 0 , . . . , a n−1 ) s F 0 ,0 = 0 (where 0 is the empty list as defined in Proposition 2.17) In this example, the state set S F 0 = TFS R is uncountably infinite and clearly works as a memory. This discrete system F 0 takes a real number as an input, appends it to its memory, and computes the maximum value of the stored list.
A more complicated discrete system could return several results; for example, in the box X 1 = ( R , R, R, R, R ) ∈ W Sets , we can define F 1 ∈ DS (X 1 ) such that:

Definition and basic properties
and s F 1 ,0 = 0 (same state set, update function and initial state as F 0 ) f rdt 1 : . . , a n−1 → (max ( a 0 , . . . , a n−1 ) , min ( a 0 , . . . , a n−1 ) , mean ( a 0 , . . . , a n−1 ) , var ( a 0 , . . . , a n−1 )) Or we could add a switch so that we can decide what output we want; in the box X 2 = ( R, 4 , R ), we define F 2 as: : a, b, a 0 , . . . , a n−1 , c) → ( a 0 , . . . , a n−1 , a , b) f rdt 2 : We could even add a reset button on the discrete system; in the box X 3 = ( R, 4, 2 , R ), we define F 3 as: : (a, b, r, a 0 , . . . , a n−1 , c) → s F 3 ,0 if r = 0 ( a 0 , . . . , a n−1 , a , b) otherwise We previously viewed general boxes (objects in W Sets ) as empty frames. Discrete systems are the objects that "live" inside. One can draw a parallel with programming: a C -box is the signature of the function, that is, its accepted types of inputs and outputs, and the discrete system is the actual code of the function.
In the rest of the article, we will often represent a discrete system F = S F , f rdt , f upd , s F ,0 as the following two-arrow graph: The first function describes how a state and an input are transformed into a new state; the second describes how the state is output, or "read out". In general, the initial state s F ,0 ∈ S F will not be represented in these diagrams, though it is implicitly there.
Discrete systems are part of the more general class of dynamical systems. We can define other types of dynamical systems depending on the category C that we are interested in. If C is the category Euc of Euclidean spaces, then we will refer to continuous systems. For more examples, see [2].

Definition and basic properties
Definition 3.5 (DS-application of a wiring diagram [2]). Let ϕ : X → Y be a wiring diagram.
The DS-application of ϕ to F, denoted DS (ϕ) (F), is the discrete system DS (ϕ) (F) = S G , g rdt , g upd , s G,0 ∈ DS (Y ) such that: We can view DS (ϕ) (F) as the discrete system we obtain from F by implementing the wiring diagram ϕ. Definition 3.6 (Parallel composition of discrete systems [2]). Let X 0 , X 1 be boxes and let F i = The parallel composition of F 0 and F 1 , denoted F 0 F 1 , is the discrete system S G , g rdt , g upd , s G,0 ∈ DS (X 0 X 1 ) such that: We also define the parallel composition of DS (X 0 ) and DS (X 1 ), denoted DS (X 0 ) DS (X 1 ), by: DS Proof. See [2].

An external equivalence relation on dynamical systems
Via the monoidal functor DS, a box contains a specified sort of discrete system (depending on the ports of the box). For an exterior spectator, the content of the box does not matter; what matters is the way it transforms input streams to output streams. Thus, even if two boxes contain different discrete systems, for example one with an infinite state set, and the other with a finite state set, as long as they both give the same output in response to the same input, then they are viewed as "equivalent" from an external point of view.
The following definitions formalise this idea.

Definition 3.9 (Input and output streams). Let
An input stream (for X) is a finite sequence The output stream produced by F when given x in , denoted F x in , is the stream x out defined by the following recursive system: We refer to the state s that F reaches after having processed the input stream x in as resulting state of F, and denote it F res (x in ). Formally, if x in = x in i i ∈n , then according to the previous recursive system, the resulting state of F is F res (x in ) = s n .
Remark 3.10. According to the notation proposed in section 1.1, x in = x in i i ∈n ∈ X in n will be written x in ∈ X in .
Remark 3.11. Definition 3.9 is a continuation of the definitions of run maps and behaviours in [9], which are functions that assign respectively the resulting state and the last output of the automaton given an input stream. The results we obtain with our notations are similar to those in [9]. Definition 3.12 (Equivalence as stream transducers). Let F = S F , f rdt , f upd , s F ,0 and G = S G , g rdt , g upd , s G,0 be two discrete systems.
We say that F and G are equivalent as stream transducers, and we write F ≡ G, when, ∀x in ∈ It is easy to see that: Proposition 3.13. The relation ≡ is an equivalence relation on the set DS (X), for any box X.

An internal equivalence relation on dynamical systems
The relation ≡ defined above does not give any information on the links between two discrete systems that are equivalent as stream transducers. In this subsection, we define another equivalence relation that provides an internal point of view. We then prove that the two equivalence relations are the same.
In the following, X = X in , X out is any box. Definition 3.14 (Simulation relation). Suppose given F = S F , f rdt , f upd , s F ,0 and G = S G , g rdt , g upd , s G,0 in DS (X).
We say that F simulates G, and we write F G, if there exists α : S F → S G such that s G,0 = α s F ,0 and such that α • f upd = g upd • id X in , α , and f rdt = g rdt • α; that is, preserving the initial state and making the following two diagrams commute: We refer to α as a simulation function: it witnesses the simulation F G.
A priori, the simulation relation does not relate the output of the two discrete systems F and G (though this does follow; see Lemma 3.19); it only declares a correspondence between both their state sets and update and readout functions. Both discrete systems can work in parallel; their state sets need not be the same, nor even of the same cardinality, but they somehow coordinate via the map α. The function α draws the parallel between the internal machinery of F and that of G.
For the rest of the article, we will be more interested in the simulation relation F G than any particular simulation function witnessing it: any one will do. Remark 3.15. Definition 3.14 refers to the existence of morphisms between two automata as described in the automata theory litterature [9]. The existence of such morphisms suffices for our purposes. We are a bit more restrictive here, as the outputs need to be the same in both automata, while in the usual definition of morphisms, automata can have different output alphabets, as long as there is a function to convert one output into the other.
The simulation relation is not necessarily an equivalence relation and is not enough for our purpose, but we can use it to generate the equivalence relation we actually need. Definition 3.16 (Internal equivalence relation on DS (X)). Let F, G ∈ DS (X).
We say that F and G are simulation-equivalent, and we write F ∼ G, if there exists a finite sequence (H i ) i ∈N ∈ DS (X) such that: It is not hard to check that: Theorem 3.17. The equivalence relation ∼ is the equivalence relation generated by , that is, ∼ is the smallest equivalence relation R such that ⊆ R.
Finally, we need to show that the equivalence relation ∼ actually groups discrete systems that have the same behaviour as a stream transducer, in the sense of Definition 3.12; that is, the external and the internal equivalence relation are the same. Take H = S H , h rdt , h upd , s H ,0 such that: Take as simulation functions the respective projections π S F and π S G .
It is easy to see that the required diagrams in Definition 3.14 do commute. For all (s, s ) ∈ S H , we have h rdt • π S F (s, s ) = f rdt (s) by defintion of h rdt . Also, for all (s, s ) ∈ S H , h rdt • π S G (s, s ) = f rdt (s) = g rdt (s ) because there exists some stream x in such that F results in s and G results in s ; besides, as F ≡ G, we have F x in = G x in , which implies f rdt (s) = g rdt (s ). Consequently, the diagrams commute and H simulates both F and G.
Proof. Follows by induction on the length of an input stream x in i i ∈n ∈ X in .
Suppose first that F, G ∈ DS (X) are dynamical systems such that F ≡ G. According to Lemma 3.18, there exists a H ∈ DS (X) such that H F and H G, and hence F ∼ G by Definition 3.16. This establishes ≡ ⊆ ∼.
We now show that ∼ ⊆ ≡. According to Proposition 3.13, ≡ is an equivalence relation, and according to Lemma 3.19, ≡ contains . Theorem 3.17 states that ∼ is the smallest equivalence relation that contains ; necessarily, we have ∼ ⊆ ≡.
The goal of this article is to show that the behaviour of a general discrete system can be emulated by some specific wiring of some other discrete system, chosen with constraints (for example, on its internal structure). As far as we know, this result cannot be obtained with a pure equality. However, we have a description of what it means to be equivalent, both from an internal and from an external point of view, with the assurance that, seen as a blackbox, the "inhabited" box remains unchanged.
As we are not using real equalities, we need to define relations between sets that correspond to the usual inclusion and equality. We say that A is a subset of B up to equivalence, and we write A B, when ∀a ∈ A, ∃b ∈ B, a ∼ b.
We say that A is equal to B up to equivalence, or A is equivalent to B, and we write A ≈ B, when A B and A B.

Memoryless systems
If F, G : W Sets → Sets are functors, then we write F G when, for all box X, we have F(X) G(X). We write F ≈ G, when F G and F G.
If M, N : W Sets → Sets are mappings (not necessarily functors), then we write M ⊆ N when, for all boxes X and Y , we have M(X) ⊆ N(X) and Mor Sets (M(X), M(Y )) ⊆ Mor Sets (N(X), N(Y )).

Main results
Before we introduce the actual results of the paper, we need a few more notions.

Algebras and closures
Definition 4.1 (Algebra). Given a monoidal category C , a functor F : C → Sets is called an algebra over C when it is a lax monoidal functor.
In our case, C = W Sets , and DS is an algebra by Theorem 3.8.

Definition 4.2 (Subalgebra). Let
A : W Sets → Sets be an algebra over W Sets . Let σ X,Y : A(X) × A(Y ) → A (X Y ) denote its first coherence map (we recall that is the parallel composition of boxes (cf. Definition 2.28)).
A functor B : W Sets → Sets is called a subalgebra of A when: Here, A and B are functors that transform boxes into sets. In our setting, the conditions can be interpreted as follows: (First item) Discrete systems generated by B are included in those generated by A; (Second item) The parallel composition of two discrete systems F and G generated by B is also generated by B. (Third item) B is stable through wiring diagrams: wiring a discrete system generated by B gives another discrete system generated by B.
Note that a subalgebra is itself an algebra. Let B : Ob W Sets → Ob Sets be any map such that ∀X ∈ W Sets , B(X) ⊆ A(X). The closure of B, denoted Clos (B), is the intersection of all subalgebras of A that contain B(X) for all X ∈ W Sets . (Any intersection of subalgebras is a subalgebra.) The closure of a map B can be understood as the minimal lax monoidal functor (or algebra) containing B.

Memoryless systems
Our first main result concerns the subclass of discrete systems that we call memoryless. We show that wiring together memoryless systems can lead to systems that have memory. . Let X = X in , X out be a box.
A memoryless discrete system for the box X, or memoryless discrete system for short, is a discrete system F = S F , f rdt , f upd , s F ,0 ∈ DS (X) such that f upd immediately discards the previous state and uses only the current input; more precisely, such that f upd factors as We denote by DS ML (X) the set of all memoryless discrete systems for the box X : DS ML (X) = We call these discrete systems memoryless because we see the states as a kind of memory (as in Example 3.4). The discrete systems defined above transition from one state to another without checking their current state, i.e. without checking their memory.
The following definition is a natural restriction of memoryless discrete systems; as these systems are memoryless, the only goal of their states is to produce the output via their readout function. The simplest case is when the readout function is the identity. Definition 4.5 (Direct-output discrete systems). Let X = X in , X out be a box.
A direct-output memoryless discrete system for the box X, or direct-output discrete system for short, is a discrete system F = S F , f rdt , f upd , s F ,0 ∈ DS (X) such that: We denote by DS ML out (X) the set of all direct-output discrete systems for the box X: Remark 4.6. The maps DS ML : W Sets → Sets and DS ML out : W Sets → Sets are not functors, because they are not closed under wiring. Indeed, the whole point is that the result of wiring together memoryless systems is not necessarily memoryless.
We can now prove one of the main results of this paper, which is that every discrete system can be obtained by memoryless systems and feedback loops. The feedback loop is responsible for holding the state that was originally in the discrete system.
Here is the formal statement.

Memoryless systems
Let Y = Y in ,Y out ∈ W Sets , and let G = S G , g rdt , g upd , s G,0 ∈ DS (Y ). We will find X ∈ W Sets , F ∈ DS ML out (X) and ϕ : X → Y such that DS (ϕ) (F) ∼ G. Let δ S G = S G ∈ TFS Sets be the list with one element, S G , and consider the box δ S G , δ S G with only that port on the left and the right. We define X as the parallel composition of this box δ S G , δ S G and Y , that is: Note that X in = S G ×Y in and X out = S G ×Y out . Thus, if x in ∈ X in , then x in = s, y in . Similarly, if x out ∈ X out , then x out = s, y out .
We choose ϕ : X → Y as the pair ϕ in , ϕ out of coproduct inclusions: It follows from 2.35 that their dependent products ϕ in : Y in × S G × Y out → S G × Y in and ϕ out : Recall that the goal is to find F = S F , f rdt , f upd , s F ,0 ∈ DS ML out (X) such that DS (ϕ) (F) ∼ G. So define F as follows: x out = f upd s, y in , s , y out = g upd y in , s , g rdt g upd y in , s s F ,0 = s G,0 , g rdt s G,0 It is easy to see that F is in DS ML out (X) because f rdt and f upd have the correct form. So let S H , h rdt , h upd , s H ,0 = DS (ϕ) (F); we need to show it is equivalent to G. We compute each part of DS (ϕ) (F) according to Definition 3.5.
Its state set is as follows: Its readout function is defined on an arbitrary x out as follows: h rdt x out = h rdt s, y out = ϕ out f rdt s, y out = π Y out id X out s, y out = y out IMTA-RR-2018-02-SC 24

Memoryless systems
Its update function is defined on an arbitrary y in , s, y out as follows: h upd y in , s, y out = f upd ϕ in y in , f rdt s, y out , s, y out = f upd s, y in , s, y out = g upd y in , s , g rdt g upd y in , s Finally, its start state is as follows: s F ,0 = s G,0 , g rdt s G,0 Consequently, the following diagram commutes: where α = id S G , g rdt . This yields G DS (ϕ) (F) and hence DS (ϕ) (F) ∼ G, which concludes the proof.  Proof. In the proof of Theorem 4.7, take H = DS (ϕ) (F), but instead of S F = S G × Y out , take S F = S G × g rdt (S G ) ⊆ S G × Y out . If S G is finite, so is S F .
In that case, H is no more in Clos DS ML out (Y ) but in Clos DS ML (Y ). In that case, H is no more in Clos DS ML out (Y ) but in Clos DS ML (Y ).
Theorem 4.7 states that systems without memory can be wired together to form systems with memory. In fact, the result is more subtle. It states that for any discrete system, we can find (or build) a memoryless discrete system with the certain wiring such that both systems are equivalent as stream transducers. The internal equivalence relation described in Theorem 3.20 is instrumental to prove Theorem 4.7, while the result is stated with regard to the external equivalence relation.

Finite-state systems
The second result is a refinement of Theorem 4.7, and is somewhat similar to it. We show that wiring together two-state discrete systems can generate a finite-state system with memory.
We can view the result as the generalisation of transistors being wired together in order to build a computer, or a system of neurons wired together to form a brain with finite memory. Definition 4.11 (Finite-state systems). Let X = X in , X out be a box.
A finite-state discrete system for the box X, or finite-state system for short, is a discrete system F = S F , f rdt , f upd , s F ,0 ∈ DS (X) such that S F is a finite set.
We denote by DS Fin (X) the set of all finite-state discrete systems for the box X : DS Fin (X) = S F , f rdt , f upd , s F ,0 ∈ DS (X) | card (S F ) ∈ N . For a wiring diagram φ, we set DS Fin (ϕ) = DS (ϕ).
It is easy to see that: Definition 4.13 (Boolean systems). Let X = X in , X out be a box.

CONCLUSION
In this paper, we use a category-theoretic framework to give a constructive proof that any discrete system with memory can be simulated by some correctly-wired memoryless system. This result can be understood as a phenomenon of emergence in a complex system. This construction opens a number of new questions. A possible question might consist in finding the "best" memoryless system, where "best" could depend on the definition of some valuation function, e.g. the most parsimonious in terms of state set. A similar question could be asked with respect to wiring diagrams, whose number of feedback loops could be bounded by a cost function.
Possible extensions of this work could concern dynamical systems other than DS. For instance, can we establish the same kind of results when considering measurable or continuous dynamical systems?