Quantifying the Scientific Method

This morning I realized that you can consider error in terms of uncertainty rigorously. I explained the mathematics of this in a previous article, showing that you can associate the distance between two vectors x and y with an amount of information given by, I = \log(l), where l = ||x - y|| is the norm of the difference between the two vectors x and y. We can therefore also associate the total error \epsilon between some prediction function f, and the correct underlying function F, over some domain, with some amount of information given by log(\epsilon).

Separately, I also showed that there’s a simple equation that relates information, knowledge, and uncertainty, as follows:

I = K + U.

Intuitively, your uncertainty with respect to your prediction function f is a function of your error \epsilon, for the simple reason that as error increases, your confidence in your prediction decreases. So let’s set the uncertainty in the equation above to,

U = \log(1 + \epsilon).

What this implies is that when your error is zero, your uncertainty is also zero, and moreover, grows as an unbounded function of your error.

The total information of a static system should be constant, and so the value I should in this case also be constant, since we are considering the relationship between two static functions, f and F. This implies that whatever the value of our knowledge K is, it must be the case that,

K + U = C,

For some constant C.

Because we have assumed that U = \log(1 + \epsilon), the only function of \epsilon that satisfies this equation is the function,

K = - \log(1 + \epsilon).

What’s interesting about this is that this set of equations together implies that,

I = 0.

Moreover, for all non-zero error, your knowledge is a negative number.

At first, I was a puzzled by this, but upon reflection, it makes perfect sense, and is consistent with the scientific method generally:

If you have no error, then you know nothing;

If you have any error at all, then you know you’re wrong.

This is in contrast to the knowledge that is possible when dealing with systems that have components that can be identified, and defined with certainty, which I discuss in the articles linked to above. In this case, what you’re measuring is your absolute uncertainty given only your error with respect to some true underlying function over some domain. It is therefore simply not the case that this would allow you to make any claims about the behavior of the function outside of that domain, absent other assumptions limiting the possibilities for the underlying function. Said otherwise, at best, in the absence of additional assumptions, you know nothing, in that you’re not wrong over that domain, but you have no knowledge about F outside of that domain, by virtue of your prediction function f.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s