We look at small Turing machines (TMs) that work with just two colors (alphabet symbols) and either two or three states. For any particular such machine
In the first part of the paper, we will define the basic notions we work with. In particular, we will fix on a computational model: small Turing machines with a one-way infinite tape. For these machines, we will define the so-called
Complexity measures are designed to capture complex behavior and quantify
The relation we have between both frameworks is as follows. We start in the framework of computations and algorithms and for simplicity assume that they can be modeled as using discrete time steps. Now, suppose we have some computer
We look at the spatial representation
One can set this up in such a way that
It is this main question that is being investigated in this paper. The computational model that we choose is that of Turing machines. In particular, we look at small one-way infinite Turing machines (TMs) with just two or three states and a binary tape alphabet.
For these particular machines, we define a notion of dimension along the lines sketched above. In exhaustive computer experiments, we compute the dimensions of all machines with at most three states. Among the various relations that we uncover is the notion that such a TM runs in at most linear time if the corresponding dimension is 2. Likewise, if a TM (in general) runs in superpolynomial time and uses polynomial space, we see that the corresponding dimension is 1.
Admittedly, the way in which fractal geometry measures complexity is not entirely clear and one could even sustain the view that fractal geometry entirely measures something else. Nonetheless, dimension is clearly related to degrees of freedom and as such related to an amount of information storage.
In [
The results presented in this paper were found by computer experiments and proven in part. To the best of our knowledge, it is the first time that a relation is studied between computational complexity and fractal geometry, of a nature as presented here.
More in detail, in Section
In Section
Section
We conclude the paper with Section
As mentioned before, this paper forms part of a larger project where the authors exhaustively mine and investigate a set of small Turing machines. In this section, we will briefly describe the raw data that was used for the experiments in this paper and refer for details to the relevant sources.
A TM can be conceived as both a computational device and a dynamical system. In our studies, a TM is represented by a
The head of a TM can be in various
A computation of a TM proceeds in discrete time steps. The tape content at the start of the computation is called the
A TM
Clearly, each TM in
We say that a TM
We will refer to the input consisting of the first
By Rice’s theorem, it is in principle undecidable if two TMs compute the same function. Nonetheless, for spaces
As previously mentioned in this paper, a central role is played by the so-called
The figure shows a sequence of space-time diagrams corresponding to the TM in (2,2) space with number 346 (according to Wolfram’s enumeration scheme [
Since these space-time diagrams are such a central notion to this paper, let us briefly comment on Figure
Remember that the computation starts with the head of the TM in State 1 in the rightmost cell. Each lower row represents the tape configuration of a next step in the computation. So, there can at most be one cell of different color between two adjacent rows in a space-time diagram. We see that this particular (2,2) TM with number 346 first moves over the tape input erasing it. Then, it gradually moves back to the edge of the tape writing alternatingly black and white cells to eventually fall of the tape, whence it terminates.
Clearly, these space-time diagrams define spatiotemporal objects by focusing on the black cells. We wish to measure the geometrical complexity of these spatiotemporal objects. Subsequently, we wish to see if there is a relation between this geometrical complexity and the computational complexity (space or time usage) of the TM in question.
In Section
Note that for this paper it is entirely irrelevant how to numerically interpret the output tape configuration whence we will refrain from giving such an interpretation. However, it has been a restrictive choice to represent our input in a unary way. That is to say, the notion of a function in our context only looks at a very restricted class of possible inputs: blocks of
Basically, the Strips Theorem boils down to the following. Let us consider a TM
We have chosen our input-output convention in such a way to prevent the Strips Theorem. There are two undesirable side effects of our coding. Firstly, it is clear that any TM that runs in less than linear time actually runs in constant time. Secondly, the thus defined functions are very fast growing if we were to represent the output in binary. In particular, the tape identity represents an exponentially fast growing numerical function in this way.
A positive feature of our input convention is that the amount of symmetry present in the input coding facilitates various types of analysis and in particular automated function-completion seems to run more smoothly.
We will here describe an alternative way of representing the input
In order to represent the input
For each
It is clear that
In this section, we will briefly recall the definition of and ideas behind the box-counting dimension which is a particular fractal dimension having the better computational properties whence better suited for applications. In Section
After revisiting the notion of box-counting dimension, we see how to apply these to Turing machines and their space-time diagrams.
We will use the notion of box dimension. This notion of fractal dimension can be seen as a simplification of the well-known Hausdorff dimension (see [
Let us briefly recall the definition of the box dimension and the main ideas behind it. The intuition is as follows. Suppose we have a mathematical object
If
If
Likewise, for a three-dimensional object, to estimate its volume, we would have
The idea behind the definition of the box dimension is to take (
The reflections above form the main ideas behind the definition of box dimension that we will use in this paper.
Let
Let us see how we can adapt the notion of box dimension to our space-time diagrams. The spatiotemporal figure
Firstly,
The second objection is more serious. As for each
The first objection is that
The second objection is that this new definition seems hard to numerically approximate at first glance. We will see how to overcome the second objection which will yield to us automatically a solution to the first objection.
As we mentioned before, we cannot first approximate
There seems to be a canonical choice though. The approximation of the dimension
For
We will sometimes write
By the nature of our input-output protocol, there exist no TMs whose runtime is sublinear but not constant. Let us first concentrate on the TMs that run in at least linear time and deal with the constant time TMs later. If a TM halts in nonconstant time, the least it should do is read all the input, do some calculations, and then go back to the beginning of the tape. Thus, clearly
Recall that
In this definition, we could address the issue of undefinedness by replacing
In the current paper, however, we have only considered TMs with either two or three states and just two colors. It turned out that in this setting we could determine both
For TMs with constant runtime, we know that only a constant number of cells will be visited and possibly changed color. For these TMs, the figure
Let
We will define the box dimension of a TM
Note that our definition of dimension can readily be generalized to nonterminating computations. Also, restricting to computational models with discrete time steps is not strictly necessary.
For certain TMs
Thus, after scaling each space-time diagram so that the vertical time axis is rescaled to 1, we will always have a little surface in the shape of a black triangle in the scaled space-time diagram. The box dimension of a triangle is of course 2. We may conclude that
Let
We fix some TM
Let us consider the case that
However,
Above we saw that for linear time TMs we can actually compute the corresponding dimension. However, for nonlinear TMs, we can only prove an upper bound on the box dimension.
For a given TM
The box dimension is maximal in case all cells under consideration are black. This number is bounded above by
As we will see, in all cases, the upper bound given by the Space-Time Theorem is actually attained in our experiment. It is unknown, however, whether it holds in general.
We first observe that for any Turing machine
The dimension
Let
The method in proving the lower bound seems very crude: no blocks of
In Theorem
For each TM
In certain cases, the Space-Time Theorem (Theorem
In case a TM
More in general, if for a TM
By combining our general lower and upper bound as proven in Theorems
Lemma
We conjecture that for each
Thus, Lemma
For each TM
Since
In case
We may assume that
The following proposition provides an almost equivalent formulation of the Upper Bound Conjecture.
For each TM
Moreover, if the Upper Bound Conjecture holds uniformly for some TM
If
For the other direction, we assume
As usual, we denote by
Likewise, we denote by
By
Let
Clearly, this does not constitute a real strategy since, for one, in general it is undecidable whether
In this second part of the paper, we describe the experiment we have performed to empirically test whether the theoretical results also hold in cases that do not satisfy the necessary requirements for the theoretical results to be applied.
We have already proven on purely theoretical grounds that there is a relation between runtimes and fractal dimension of the space-time diagrams. However, our theoretical results only apply to a restricted class of TMs.
In the experiment, we wanted also to study the fractal dimension of the space-time diagrams in cases where our theoretical results do not apply. Moreover, guided by the first outcomes of our experiment, we formulated the Upper Bound Conjecture (Conjecture
For TMs
A substantial complication in this project is caused by the occurrence of logarithms in the definition of
The figure shows an estimate of the box dimension of 2,2 TM with TM number 346. On the horizontal axis, the input is shown and the vertical axis shows the corresponding approximation of the box dimension. Note that we know that the function converges to 2 when the input tends to infinity.
Our way out here is to apply numerical and mathematical analysis to the functions involved so that we can retrieve their limit behavior. In particular, we were interested in three different functions.
As before, for
With these functions and knowledge of their asymptotic behavior, we can compute the corresponding dimension
It is important to bear in mind this process and the fact that we work with guesses that can be wrong in principle. For example, if we speak of a TM
In this section, we will describe the steps that were performed in obtaining our results. Basically, the methodology consists of the following steps: Each TM that lives in 2,2 space also occurs in (3,2) space so for the final results it suffices to focus on this data set. The TMs that diverge on all inputs were removed from the initial list of 2 985 984 TMs in the (3,2) space, since for them the dimension is simply not defined. For the remaining TMs, we erased all diverging inputs from the sequence to which we were to apply our analysis. Since we are only interested in limit behavior of any subsequences, this does not alter our final results. We isolated the TMs for which there is no theorem that predicts the corresponding dimension. By Lemmas In addition, there are 1 792 TMs that perform in exponential time and linear space, but clearly they needed no further analysis since we know on theoretical grounds that their corresponding dimension is 1. All other machines in (3,2) space were very simple in terms of time computational complexity; that is, they perform at most in linear time. Per TM Per TM Per TM Per TM Per TM Per TM Per TM
Distribution of those TMs in (3,2) space of which we had to compute the corresponding dimension over their complexity classes. By
Boxes | Runtime | Space | Machines |
---|---|---|---|
|
|
|
3358 |
|
|
|
6 |
|
|
|
14 |
Some of the Turing machines possessed alternating asymptotic behavior. This has been already observed in [
The differences between the alternating subsequences can be rather drastic though. The most extreme example we found is reflected in Figure
Alternating linear and exponential runtime behavior for TM 1 728 529.
Figure
This machine runs in linear time for even inputs and exponential time for odd inputs. The runtime is given by
Moreover, we note that the sequence of outputs is of a very simple and regular nature. The outputs can be grouped in series of two, where the output on input
We have found alternating sequences of periodicities 2, 3, and 6. Like we noted in [
Alternating values for
In Figure
This alternating behavior reflects the richness of what we sometimes refer to as
In this section, we would mainly like to stress that most of the computational effort for this paper has actually been put into determining/guessing the functions
As may have become manifest from the previous section, it is hard to automatically guess perfect matches for these functions in case there is alternating behavior present. Finally, we could deal with all functions in a satisfactory way. Notwithstanding our confidence, it is good to bear in mind that all classifications provided in this paper are given the current methodology.
We will here briefly describe how we proceeded to guess our functions. The methodology is fairly similar to that performed in [ We collected the sequences for time usage These sequences
Thus, we obtain two lists: a list Moreover, 288 runtime sequences and 85 space sequences in We performed a check on our guesses as collected in From the list For the TMs in For the sequences in
In most of the cases, there was alternating behavior present. We could read off the periodicity from looking at graphs as, for example, in Figure One alternating TM did not succumb to this methodology. This was TM 582 263 whose treatment is included in Section In some cases, the regularity was not obvious to Mathematica but was evident when looking at space-time diagrams and/or the binary expansion of the output. In these cases, we could manage by just feeding our insight into Mathematica in that we let it work, for example, on the binary expansion of the sequences. In some cases, the recurrences were just too complicated for Mathematica version 8. In these cases, we carefully studied the space-time diagrams analyzing what kind of recurrences were present. Then, the observed recurrences were fed into After having successfully (allegedly) found the functions
In this section, we will present the main results of our investigations. The space of TMs which employ only 2 colors and 2 states is clearly contained in (3,2) space. However, we find it instructive to dedicate first a section to the findings in (2,2) space. Apart from the first section, all other results in this section refer to our findings in (3,2) space.
In (2,2) space, there was a total of 74 different functions. Of these functions, only 5 of them where computed by some superlinear time TMs. Note that this does not mean that all TMs computing this function performed in superlinear time. For example, the tape identity has many constant time performing TMs that compute it but also some exponential time performing TMs that compute it.
In total, in (2,2) space, there are only 7 TMs that run in superpolynomial time. Three of them run in EXP-time, all computing the tape identity. The other four TMs compute different functions. These functions do roughly compute a function that doubles the tape input (see Figure
The figure shows the four different functions that are computed by the four TMs that have quadratic runtime in 2,2 space. The diagrams show the outputs on increasing inputs. So, for example, in the leftmost diagram, we see that TM with number 1383 (recall this is the code in (2,2) space) outputs two black consecutive cells on input 1, and more in general it outputs
All these four TMs perform in quadratic time and linear space. We computed the dimension for these functions and all turned out to have dimension
The only three exponential time performers used linear space so by Lemma
We saw that a TM in (2,2) space runs in superpolynomial time if and only if its dimension equals 1. This observation is no longer valid in (3,2) space though.
In the remainder of this section, we will focus on the TMs in (3,2) space. That space contains 2 985 984 many different TMs which compute 3 886 different functions. Almost all TMs used at most linear space for their computations. The only exception to this was when the TM used exponential space. Curiously enough, in (3,2) space, there was no space usage in between linear and exponential space.
In [
Classically speaking, the
Figure
Execution of the Busy Beaver on the first three inputs.
That is, if one looks at the amounts of used cells for consecutive inputs and their differences, then modulo a small error term, there is a clear tendency. Let
The structure of the space sequence of machine 666 364 in (3,2) space.
|
|
|
|
Difference |
---|---|---|---|---|
1 | 3 | — | — | — |
2 | 7 | 4 | — | — |
3 | 13 | 6 | 6 | 0 |
4 | 22 | 9 | 9 | 0 |
5 | 36 | 14 |
|
|
6 | 57 | 21 | 21 | 0 |
7 | 88 | 31 |
|
|
8 | 135 | 47 |
|
|
9 | 205 | 70 |
|
|
10 | 310 | 105 | 105 | 0 |
So, ignoring the exact nature of the error term, the recurrence equation for the space is given in the following:
The runtime depends on the space and we found the following recurrence relation for it:
Using these recurrence equations, we could finally compute the limits. We computed the limits by both standard methods on limits of recurrence relations and employing
One of our most important empirical findings is that the upper bound as given by the Space-Time Theorem is actually always attained in (3,2) space. Moreover, we found two related empirical facts for (3,2) space. We mention them in this section.
In Proposition
In Lemma
1/9, 1/6, 7/30, 1/4, 5/18, 5/16, 1/3, 3/8, 8/21, 7/18, 5/12, 3/7, 4/9, 7/15, 1/2, 5/9, 9/16, 2/3, 3/4, 7/9, 1.
It is possible that a few other limit values exist but were not found by the way we computed the functions generating
For two of the EXP-space performers, we could not find the boxes function. These TMs were 582 263 (and their twin machine), whose execution for inputs 1 to 6 is shown in Figure
Execution of machine 582 263 on the first six inputs and their corresponding space-time diagrams.
For the sequences of even number of consecutive black input cells, we found that the fraction
The authors have explored the space of small Turing machines before. On occasion, they have been so much impressed by the rich structures present there that they came to speak of
In particular, Figure
Symmetric performers.
Of course, this can only happen in case the TM computes the tape identity since the input must equal the output in order to yield a symmetric image. At first, one might be tempted to think that this phenomenon is bound to occur since we can define for each TM
Let us denote by
Indeed, it comes as a surprise that all these constraints can be met in (3,2) space, if only just for the even inputs.
In the third and final part of the paper, we will try to locate our results within the landscape of known theoretical results that link fractal dimensions to other notions of complexity.
In this paper, we have worked with a variant of box-counting dimension and with space and time complexity for processes implemented on Turing machines. These are just some out of a myriad of different complexity measures in the literature. Since eventually the notion of being complex or not is relative to a framework and the ultimate framework in which all these complexity notions can be embedded in is our own cognitive system, on philosophical grounds, one can expect relations between the various
In this final section, we wish to place our results in the context of other results in the literature that link different complexity notions. Our point of departure will be fractal dimensions and possible relations to complexity notions of a computational nature.
Neither is the current section self-contained nor do we pretend to give an exhaustive overview of the literature. Rather, we will try to provide sufficient pointers so that this section at least can serve as a point of departure for a more exhaustive and self-contained study.
In this paper, we decided to work with a variant of box-counting dimension since this has many desirable computational properties and applications. Let us first see where box-counting dimension fits into the landscape of various versions of fractal and other dimensions.
Edgar divides geometrical dimensions in two main groups,
The most basic of all topological dimensions is the so-called
The
The
Fractal dimensions on the other hand can have noninteger values. In a sense, the fractal dimension of some object
The most fundamental and most common notion of fractal dimension is that of
In order to relate our box-counting dimension to the more common Hausdorff dimension, we will outline the definition and some basic properties of Hausdorff dimension.
For
This unique
The Hausdorff dimension comes with a natural dual dimension called
The main idea behind packing dimension of some spatiotemporal object
This unique
A fundamental property that is not hard to prove of the dimensions we have seen so far is that
We can now see how box-counting dimensions (or box dimensions for short) naturally fit the scheme of fractal dimensions we have seen above. In particular, the box dimension is like Hausdorff dimension only that we now cover the spatial object by balls/boxes of
Alternatively and equivalently, in order to define the box dimension, we can divide space into a regular mesh with mesh size
Again, there is a cut-off value
In case
Notwithstanding the good computational behavior, box dimension has various undesirable mathematical properties: in particular, a countable union of measure zero sets can have positive box dimension. For example, one can show that in
Mathematically, this undesirable properties can be impaired with the same trick that was applied to the packing dimension by defining
But, of course, by doing so, we would lose all the good computational properties. In general, we have that
Moreover, if
Let
So, in various situations, box counting coincides with Hausdorff dimension. The most famous example is probably that this equality holds for the Mandelbrot set. In addition, there are various other situations where box-counting and Hausdorff dimension coincide [
As a first link between fractals and computability properties, we want to mention that of various fractal objects one has studied the computational complexity.
Probably the most famous examples of fractals are Julia sets and the corresponding “roadmap Mandelbrot set.” Let us briefly recall some basic definitions. By
One can express that
The
Likewise, we say that a set
We call a set
The Turing degree of a set, the equivalence class under
We can conceive a real number as a set of natural numbers. Let us restrict ourselves to the real interval
We will shortly discuss that one can set up real analysis in such a way that it also makes sense to speak about the Turing degree of nondiscrete objects like
Braverman and Yampolsky have studied (see [
Chong has generalized this result [
It is good to stress that all these results are sensitive to the underlying model of computation and real analysis and the results would change drastically if one were to switch to other models like the so-called Blum-Schub-Smale model (see [
The results presented in this section relate the Turing complexity of the fractal to the complexity of the parameter generating it. However, there are no links from the Turing degrees of the Julia sets to the corresponding dimensions. In the next section, we will discuss various results of this sort.
In this section, we will present certain results that relate Hausdorff dimension to other notions of complexity. In order to do so, we will first rephrase the notion of Hausdorff dimension in the setting of binary strings. Next, we will define the so-called effectivizations of Hausdorff dimension. It is these effectivizations that can be related to other notions of complexity. Again, this section will be far from self-contained. We refer the reader to [
Thus, let us reformulate the definition of Hausdorff dimension in the realm of binary sequences, that is, in the realm of Cantor space which we will denote by
For
Thus, for any
Within the context of Cantor space, we will now give a definition of what is called
One can now show [
Let
In the same paper, Hitchcock also proves an equality for
However, for other important classes, they differ. In particular, we have that the Hausdorff dimension of any sequence in Cantor space equals zero. However, there may be no simple effective covers around so that a single sequence can have positive effective Hausdorff dimension.
There is a link between Turing degrees and effective Hausdorff dimension although this link is not very straightforward. Recall that for
Thus, in a sense, having nonzero effective Hausdorff dimension is an indication of containing complexity. And in fact it can be shown that if
This result establishes a relation between effective Hausdorff dimension and computational complexity in the guise of degrees of undecidability. However, the relation between effective dimension and computable content is not monotone nor simple. In particular, one can show that if
Let
It is exactly this kind of results that we are interested in here in this section: theorems that relate different notions of complexity. Another classical result links Kolmogorov complexity to effective Hausdorff dimension. Let us briefly and loosely define Kolmogorov complexity referring to, for example, [
For a string
Let
Moreover, there is a link from effective Hausdorff dimension to a notion that is central to probability theory:
An
This is a generalization of “gales” (as introduced/simplified by [
We say that a certain gale
The
Consider
In the context of this paper, it is good to mention that other notions of dimension also have their effective counterparts. In particular, Reimann studied an effectivization of box-counting dimension in [
Also, for various dimensions, the
In this section, we have tried to present a selection of readily accessible results in the literature that relate fractal dimension to other notions of complexity.
We mentioned results of Braverman and Yampolsky and the generalization thereof by Chong in Section
Next, in Section
Effective Hausdorff dimension however works with highly idealized notions relatively high up in the computational hierarchy. Our results involve complexity classes which are more down-to-earth like
The authors declare that they have no competing interests.
Joost J. Joosten wishes to thank José María Amigó García for his kind support and an invited research stay at the University of Miguel Hernandez in Elche. Joost J. Joosten received support from the Generalitat de Catalunya under Grant no. 2009SGR-1433 and from the Spanish Ministry of Science and Education under Grant nos. MTM2011-26840 and MTM2011- 25747. The authors acknowledge support from the project FFI2011-15945-E (Ministerio de Economía y Competitividad, Spain).