Saturday, 26 July 2014

Randomly Infinite

As some of you might know, I am writing a popular book about Bayesian probabilities and science. The first draft, which is a work in progress, can be found for download here and I appreciate any feedback:

I am working on a revised version right now and I decided to expand the sections on randomness and make them a whole chapter. The problem is to know when to stop. Randomness is such an interesting and intriguing topic that one can write a whole book only on it. There are many things that I would like to say about randomness, but I have to leave out otherwise it would change the main theme of the book. 

I will consider writing a book only on randomness in the future, but I still have some consolation on the fact that I can at least share with you those things that, at present, will be left out. One of those things is related to the mind-boggling things you get when you mix the two dangerous concepts of randomness and infinity.

SPOILER ALERT. If you haven't read Contact by Carl Sagan yet, be aware that I will be talking about something that happens literally at the end of the book. Be warned. At the very end, the main character is running a program to find a message that has been left hidden in the digits of the number Pi by the supposed 'designers of the universe'. The program suddenly spills out a sequence of numbers that somehow form the picture of a circle. Now, you might be very truly amazed to know that Carl is right: there is indeed such a sequence hidden in the digits of Pi! That exact sequence, by the way! Have I just changed your life? Before you starting twitting about these amazing news, let me tell you a couple of things about Pi.

Well, everybody knows that Pi is a number which is a result of dividing the length of a circle by its diameter. In flat Eulidean space, which is the one obeying the geometric properties you have learned in the school, this works for every circle. But Pi is a very interesting number in many other aspects. One of them is the fact that it is an irrational number. This means that there is no way to write Pi as a fraction, or a rate, between two other integer numbers. A consequence of this is that the decimal digits of Pi cannot (ever, never) be periodic. What does that mean?

A periodic sequence is one that repeats itself after a certain amount of time. Examples are:


I am assuming these sequences repeat forever (I call the last one the 'Mambo Sequence' by the way). The first sequence has period one, the second has period 2 and the Mambo Sequence has period 6. The period is then, clearly, the number of digits that are repeating. A rational number, one that can be written as a fraction between two integers, always finishes with a periodic sequence. It can take a while to reach that sequence, but it is always there. For instance:


where the last digit '6' repeats forever is a rational number. In irrational numbers, like Pi, this never happens. The odd consequence behind this is that the digits of Pi, which start like:


have all the properties of an infinite sequence of randomly distributed integers from 0 to 9! Each one with equal probability. If you are skeptical, take a look at the two graphs below.

These two graphs appear in my book. They represent two sequences of digits from 0 to 9 with a total of 100 digits. Can you see the difference between them? There is no regularity in any of the two graphs, but one of the above sequences is the digits of the number Pi and the other is a sequence of digits randomly generated by a computer program. Try to identify which one is Pi. There is a way, but it is definitely not by looking at their overall appearance.

Another detail is that the number of decimal digits in Pi is infinite. That is because any number whose sequence of decimal digits is finite IS a rational number. All you need to do to find its representation as a fraction is to multiply it by an appropriate power of ten until it becomes an integer. The number is then that integer divided by the power of ten.

Many of you must know Jorge Luis Borges' story The Library of Babel. In it, Borges imagine a library containing books in which every combination of the letters of the alphabet are present, in a random order. This means that, if you only look at the books with say 400 pages, the library contains all stories and all scientific books that have been ever written or that will one day be written as long as they fit in 400 pages! Even things that haven't been discovered yet! Even stories that nobody wrote yet, but that one day someone will! In fact, because the library is infinite, it contains all books that have ever been written or that will ever be.

Although Borges' library is fictional, it illustrates a truly amazing property of the infinite. When you put together infinity and randomness, you get something even more amazing. It can be proved that in an infinite random sequence, ANY finite sequence of characters appears an infinite number of times! Now, the punchline:

Every finite sequence of numbers appears an infinite number of times
in the sequence of decimal digits of Pi.

And so what? Think about this. In the same way as you can encode computer files in binary form, you can also encode any information in decimal form. If you doubt, just write down the binary representation of any file. That is an integer number. Write that integer in decimal base and voila! This means that every text that has ever been written or that will ever be written can be found somewhere in the sequence of decimal digits of Pi! An infinite number of times! This means that, whatever Sagan's character found in the sequence of digits of Pi, it is not a message from another race, but simply the result of good and old randomness!

If you are worried that there is so much information hidden in Pi or maybe trying to devise a plan to extract future information from it like the Bible Code, be aware that this is useless. Because the digits are random, there is no way to know where the information is before hand, or even which information is correct or not, because the same information appears with all possible mistakes!

Reading the text above again, I cannot stop being amazed myself. I am having second thoughts if I include this in the book or not already...

Update: Some friends are telling me that it's not proved that the digits of Pi are random. Maybe I should have been more cautious when I wrote it. It is absolutely true that there is no proof that the digits of Pi are random and, in fact, there seems to be evidence that it is actually pseudo-random, in the sense that one might predict them if we have a certain formula that was found by a couple of mathematicians.

Still, the possibly pseudo-random sequence of digits of Pi continues to satisfy most tests of randomicity, which although is not a proof that they are random, makes it more probable that it satisfies the most interesting properties, specially the one of having every other sequence written in it. 

Friday, 18 July 2014

The Circle of Life

I have just read this funny circular definition of life on June's Scientific American:

"(...) would be strong evidence of life, widely defined: a biological system that encodes information and uses this information to build complex molecules."

I'll leave to the reader to find out why it is circular.

Wednesday, 16 July 2014

The (New) Meaning of Science

Every word, in every language has a life cycle. Words are used by humans in their daily affairs and humans are complicated creatures whose decisions are affected by a complex interplay between reason and emotion. Because of this, words evolve in such a way that, with time, their meanings go through “mutations” which might eventually lead to such a radical change that they cannot be used anymore within their original scope. The word “science” is not different.

Most modern discussions concerning what science actually is, end up falling in the semantic category. The reason for that is the fact that, at some point, the meaning of ‘true knowledge’ was attached to the word science. The study of methods to discern what kind of knowledge is ‘true’ and what is ‘false’ ended up being associated to the word and, as these methods became successful, the word science acquired a respected status. Humans are attracted by reputation as this result in better chances of satisfying emotional goals. Therefore, the importance of guaranteeing being associated with the word and the status it provides.

The original meaning of the word “science” seems to have been very little ambitious, simply meaning any kind of knowledge. Greek philosophers seem to be responsible of seeking a way to separate knowledge that would actually describe how the world works from that which would not. This was when the word science started to acquire its respectability. 

The S-Method

Let us forget the word “science” for now and consider the following problem. It is undeniable that there are repeating patterns in nature. That is a trivial observation whose simplest example is the fact that the sun rises with some predictable regularity every single day. In fact, that creates the basis on which we define what a ‘day’ is.

The fact that patterns exist allows us to write down sets of rules for these patterns. The problem I want to propose is that of checking if a pattern we think we found is really there or not. This can be thought in terms of a competition. 

The competition consists in the judges writing down a set of rules that generates a sequence of numbers. The judges hire a programmer to create an app that uses the rules to generate the required sequence of numbers. Using that program, the judges generate a dataset which is then given for different groups of people and their task is to find the original pattern, the judges’ rules, that generated the data. Once each group has prepared their entry to the contest, one has to decide which one is the winner. In this case, of course, all that is needed is to check which group gave the correct rules.

The way it is, it is easy to decide who is the winner. Suppose now that, somehow, the judges lost the original rules and cannot remember then. All they still have is the app, but the programmer has already gone on holiday and cannot be contacted. They need to decide which group is the winner. Can they do that?

Indeed, there is a way to select the winner and we will call this the S-Method (‘s’ for ‘selection’). The S-Method is an elimination method. The judges start to generate additional numbers beyond the original dataset and ask the groups to do the same with their rules. Each time a group generates a number which is different from the one generated by the app, the group is eliminated.

Unless two groups have equivalent rules, which means that they always generate exactly the same numbers, the S-Method guarantees that at some point one will find a winner. This can take time, but will eventually happen. 

But there is still one limitation of the S-Method. It serves the objective of finding a winner, but it cannot guarantee that the rule given by the winner corresponds to the correct one. No matter how long you test, although you might be able to catch a failure and debunk the winner’s method based on its predicted next number, one will never be able to tell if the generated numbers will always work for sure. 

Notice that there is one key idea of the S-Method: it requires each group to make predictions about the next number. That is because one wants to check if the rules, or in other words the inferred pattern, are indeed the correct ones.

There is no way to check if the rules work without testing them against the data. If one of the groups simply created a fancy story that would generate only the original dataset but could not be used to generate additional numbers, they would not have identified the original rules. 

The possibility of generating a prediction that can be checked against data generated by the original rule has the name of falsifiability. Entries to the contest which are not falsifiable, cannot be judged. In the case of the contest, they are automatically wrong as the original rules do generate more numbers.

Consider now that we are dealing with nature. We do not really know if there are indeed patterns in every phenomenon. Experience indicates that there is by the simple observation that we were able to find many up to this day. If our guesses about a phenomenon are falsifiable, then we can apply the S-Method to select the best guess and even to eliminate all of them.

However, it might be possible that in nature there are phenomena to which we cannot find a pattern in principle. In those cases, the phenomenon cannot be attacked using the S-Method. It is out of its reach. Fortunately, those situations are rare and do not affect our lives significantly, only emotionally.

The Certified Scientist 

You can appreciate that both the effectiveness and the limits of applicability of the S-Method are well established above. It turns out that, at some point in history, the word ‘science’ started to be associated only with knowledge that could be checked using the S-Method.

Because the S-Method is clear, objective and powerful, it started to yield results. Those who dedicated themselves to check which of those guessed patterns up to that point were valid or not using the S-Method succeeded in selecting the rules that actually worked.

It did not take long for people to see that explanations in terms of gods and spiritual entities for the natural phenomena were not falsifiable. This would not be too critical in principle, the greatest problem is that people started to actually find falsifiable descriptions for those phenomena.

Those people who started to dedicate themselves to tailor falsifiable models for natural phenomena then became the new ‘scientists’. They gathered together and started to teach others. 

The success of this new meaning of ‘science’ made the title of ‘scientist’ a desirable one. Desirable because of the credibility associated with it. And then the scientists started to give certifications for those who studied with them. They created the ‘certified scientists’ and this was the beginning of a new change in meaning.

The problem with certifications is that, at some point, they stop being about the original qualities of the product and become a matter of politics. Those who receive a certification that cannot be revoked will tend to ignore the very rules that allowed them to earn it in the first place when those rules are against their personal beliefs. Because the individuals themselves have the power of certifying others, the certifications start to become degenerate with time.

The unintended effect is that the original meaning of the very words that defined the certification start to drift away. Because now you have ‘certified scientists’ that will not admit losing their certification, they will lobby to include in the meaning of ‘science’ whatever they personally do or think that they should do.

New Science

Finally, the term ‘science’ starts to be associated not with the S-Method anymore, but is now used to describe a profession whose definition bends according to the wills and necessities of those who have the power to give certifications.

Here lies the kernel of all modern discussions about what is ‘science’. Discussing the validity of the S-Method is not an issue, the issue became whether give or not the ‘certified scientist’ title even when one ignores the S-Method. 

Model Engineering

The S-Method, as powerful as it is, is just a selection procedure. It requires models to select. This guaranteed ‘model engineering’ as an important part of what became known as science.

As more data about natural phenomena accumulated and models of the simpler ones were selected, model engineering became more complex. Whenever complexity increases in an area, specialisation naturally follows. This resulted in many certified scientists becoming specialised in model engineering.

Today’s model engineering is a very sophisticated process and mathematics plays a key role on it. Mathematics allows us to concisely describe patterns in natural phenomena, including the ones used by humans to reason. Once these patterns are codified and selected as valid ones, they can be trusted until a reason appears not to do so.

Model engineering is a very difficult area and requires a lot of ingenuity and creativity. In modern times, it also requires a good knowledge of mathematics and a certain ability to work with it. Many of the most famous certified scientists are theoretical physicists because their mathematical ability is recognised outstanding.

The use of mathematics provided a means to build models that go beyond the practical reaches of the S-Model in terms of economic and technological feasibility. There is no known limits to the kind of models that can be engineered, the only constraint being that they should agree with collected data and not contradict those which have already been selected by the S-Method within their limits of applicability.

Many certified scientists will only concentrate on model engineering and leave the task of selecting models to other specialists. There is nothing wrong with that in terms of profession as long as they remember that the fact that a model has been engineered using valid methods still does not mean that the model is the correct rule to describe some natural phenomenon.

Science without the S-Method 

What happens if we keep model engineering and discard the S-Method? 

Many people today are lured into believing that, as long as a model involves mathematics, it is a good model, but model engineering can be completely detached from the S-Method. As a consequence, using mathematics does not per se provide any extra credibility to a model.

Religion and mysticism contain many examples of models which can even be based on mathematics, but nevertheless would either not be vindicated by the S-Method or even not falling under the scope of its application. Model engineering, without the S-Method, falls into the same category. 

Questioning is not Enough

Rebellion against rigid impositions is a good practice. It is by questioning traditional rules that reasoning flaws can be found. However, rebellion for the sake of rebellion is as useless as conformism. One must question things with a reason, otherwise the questioning becomes senseless.

Critics attack the S-Method, or the falsifiability principle, as being too rigid, but ignore what is the original objective that led to it. 

If one wants to change the meaning of science once again from a method to find correct models to describe nature’s patterns to a list of professional obligations, there is very little to do to prevent this. What cannot be tolerated is that this new meaning of science still demands to be recognised as something that achieves this.

Tuesday, 15 July 2014

Game of Life

I have just found this application that allows you to simulate Conway's Game of Life and other cellular automata:

It's an open source program that allows you to change the update rules for two dimensional automata. I have been playing with it a bit and seems very simple and potentially very useful (not mentioning very entertaining). The picture in the beginning is a screenshot of one of the rules.

Friday, 11 July 2014

Post-Empiricism and Data Tables

I was reading Peter Woit's blog and stumbled with this post-modern word: post-empiricism. Apparently, a guy named Richard Dawid, said to be a physicist-turned-philosopher whatever that means, wrote a book about this and String Theory. I haven't read the book and I doubt I will, because I have already read one thousand similar arguments and not even one of them had anything new to add to the discussion. 

But to give you an idea of what Dawid means by "post-empiricism", I will reproduce part of an interview given by him which Woit put in his blog:

I think that those critics make two mistakes. First, they implicitly presume that there is an unchanging conception of theory confirmation that can serve as an eternal criterion for sound scientific reasoning. If this were the case, showing that a certain group violates that criterion would per se refute that group’s line of reasoning. But we have no god-given principles of theory confirmation. The principles we have are themselves a product of the scientific process. They vary from context to context and they change with time based on scientific progress. This means that, in order to criticize a strategy of theory assessment, it’s not enough to point out that the strategy doesn’t agree with a particular more traditional notion.

Let me start by saying that, as Sokal has made explicit, the fact that you find an intellectually good looking word for something does not make that true. In particular, although attaching the suffix 'post-' to a word gives to it an air of modernity and rebellion, that also doesn't give any extra credibility to the concept.
Okay, as I am not reading the book, I have to extract what I understand by Dawid's post-empiricism from the post. It seems to me is that he is simply rephrasing in the most Sokal-like fashion the argument that we have to relax the condition that theories should be testable. He talks about 'god-given principles', principles that 'change with time' and 'traditional notion'. All this, of course, are discourse techniques which mean nothing concrete.

I really, really understand the desperation of string theorists to defend their line of research given the fact that people cannot give credit for theoretical exploration of ideas, but that's not reason to turn to religion and mysticism or starting believing in ghosts, which is exactly what happens when one argues that one does not need to test if something works or not as long as it is interesting. Of course, not all evidence comes from direct experiments. A theory can be tested by comparing it with other tested theories to see if there is any inconsistent, but ultimately, a theory that does not make any testable prediction is nothing more than a data table. It can be a beautifully decorated table, but it is still just a table. I will explain myself. 

Think about the following toy phenomenon: a ball is in a field divided in two sides and it changes sides once in a while. My data set consists of the times at which the ball passes through the central line that divides the field. Suppose now that I have five data points: t = 1, 5, 6, 11, 20. Now, I say to you that I have a theory describing this data. My theory is 

-6600 + 9950 t - 3941 t^2 + 633 t^3 - 43 t^4 + t^5 = 0.

In other words, my theory is that the times at which the ball passes the central line are the zeros of the above polynomial. There is only one problem: there are only five zeros for the above equation and they are exactly the data in my data set. This means that the above equation, even being an equation, is nothing more than the list of points I had before written in a different way.

Any reasonable person will then complain: wait! But you are making the prediction that the ball is never going to cross the line again! And then I say to you: don't worry, it's such a nice-looking equation! Be more of a post-empiricist and give less importance to predictions. Who needs to test such a beautiful theory? Besides, do you have a better theory to describe this data?

I rest my case.