Forest Gump and the Normal Distribution

"I may not be a smart man, but I know what love is." Forrest is a fictional character and was deemed by the experts as having a well below average IQ; however as far as fiction goes, I think he was a lot more intelligent than Jenny. Maybe he listened to his Mama. Psychologist Alfred Binet, said:


"It seems to us that in intelligence there is a fundamental faculty, the alteration or the lack of which, is of the utmost importance for practical life. This faculty is judgment, otherwise called good sense, practical sense, initiative, the faculty of adapting one's self to circumstances. A person may be a moron or an imbecile if he is lacking in judgment; but with good judgment he can never be either. Indeed the rest of the intellectual faculties seem of little importance in comparison with judgment" (Binet & Simon, 1916, 1973, pp.42-43).


So, what does Forrest have to do with the normal distribution? IQ scores are said to follow a bell-shaped curve, or normal distribution. Technically, the normal distribution does not really work, because there is a chance that a test score could be negative, unless you truncate (fix the distribution so as not to allow negative scores). However, the normal distribution works well in describing the distribution of standardized test scores, for example IQ, SAT, etc.



A Little History

The Stanford Binet Scale is widely known for measuring intelligence levels of individuals between the ages of 2 to 85+ years. This test was the result of the revision by Lewis Terman in 1916 to the Binet-Simon Scale. Since 1916 there have been many revisions to the test. It was the first published intelligence test to supply detailed administration and scoring procedures was the first American test to use the idea of the intelligence quotient (IQ). (Laurent, Swerdlik, & Ryburn, 1992.) Currently, it is in its Fifth Edition. The Stanford-Binet metric is based on an average or mean score of 100.

Importance of the Normal Distribution

The term "normal" possibly arose because of the various attempts made to establish this distribution as the underlying rule governing all continuous variables. These attempts were based on false premises and consequently failed. Nonetheless, the normal distribution rightly occupies a preeminent place in the field of probability. In addition to portraying the distribution of many types of natural and physical phenomena (such as the heights of men, diameters of machined parts, etc.), it also serves as a convenient approximation of many other distributions which are less tractable. Most importantly, it describes the manner in which certain estimators of population characteristics vary from sample to sample and, thereby, serves as the foundation upon which much statistical inference from a random sample to population are made.

About 68% of values drawn from a normal distribution are within one standard deviation σ > 0 away from the mean μ; about 95% of the values are within two standard deviations and about 99.7% lie within three standard deviations. This is known as the 68-95-99.7 rule, or the empirical rule, or the 3-sigma rule.




IQ Scores and the Normal Distribution


IQ Scores are said to follow a normal distribution with a mean of 100 and a standard deviation of 10 (approximately). Thus, if you have a IQ score between 90 and 110, you are within 0ne standard deviation of the mean. (If you have taken any standardized test, you have probably seen or heard these terms before.) About 15% of the population have scores greater than 110 or less that 90. Approximately 2% of the population have IQ scores above 120 or below 80. Less than 1% have scores greater than 130 or less than 70.


Now, I am not saying that I agree with IQ scores. If we take fictional Forrest as a representative of some actual person, and apply Alfred Binet's definition of intelligence, IQ scores may not tell the whole story about intelligence.

Incidentally, What was Forrest Gump's IQ score?

Random Number Generators

Random numbers are used in things like the lottery and encryption, but this article is about their use in simulation.

Random number are critical building block for introducing variation is simulation experiments. They are used for providing variation between simulation runs, and within each run of a simulation, by providing the basis for producing random variates (samples values or outcomes from particular probability distributions).

Random numbers in simulation provide real, uniformly generated numbers between 0 and 1, sometimes written as Uniform ~ (0, 1). A real Uniform ~ (0, 1) probability distribution would appear as a solid rectangle from 0 to 1 with a height of 1, graphically. The distribution would have a mean of 0.5 and a variance of approximately 0.0833.

Classical uniform random number generators have some major defects, such as, short period length and lack of higher dimension uniformity. However, nowadays there are a class of rather complex generators which is as efficient as the classical generators while enjoy the property of a much longer period and of a higher dimension uniformity.

Computer programs that generate "random" numbers use an algorithm. That means if you know the algorithm and the seed-values you can predict what numbers will result. Because you can predict the numbers they are not truly random - they are pseudo-random. For statistical purposes "good" pseudo-random numbers generators are good enough.

Below, I have evaluated four Real Uniform (0, 1) random number generators using the correlation test and frequency test, as well as calculating estimates for their means and variances. The four random number generators are the Additive Congruence Generator, the Linear Congruential Generator, the Excel 2007 (based on Algorithm AS 183, Appl. Statist (1982) vol. 31, no. 2), and the ExtendSim 8 (a trademark of Imagine That Inc.) random number generator. The results of the tests for all four generators are generally, "good enough" for most discrete event simulation needs ("most" is relative and subject to debate). The ExtendSim 8 random number generator performs particularly well for more complex systems simulation. I will caveat this with the fact that I did not examine cycles or conduct a runs test. Furthermore, I did not perform more rigorous goodness of fit test, such as the Anderson-Darling or Kolmogorov-Smirnov tests.

Additive Congruence Generator

For n := 5 and k := 99991

Statistics

The mean and variance for 1000 random numbers generated by this formula as very close to the "actual" mean and variance of the Uniform distribution. Again, we would have to perform more rigorous tests to determine if these estimates are "good enough," from the perspective of statistical significance.

Mean = 0.483182

Variance = 0.087944

Correlation

Correlation indicates the degree to which the data are linearly related. A correlation of -1 indicates the there is a negative linear relationship, or that the data points approximately fit a line that slopes downward from left to right. A correlation of 1 indicates that the data approximately fits a line sloping upward from left to right. A correlation near zero indicates that the data is not linearly related. Graphically, that would look like the figure below.

	w i	w i+1
w i	1
w i+1	0.083997	1

Frequency

The frequency distribution shows how many data point fall within specified ranges called bin. I have selected bins from 0 to 1 with an interval of 0.1 for each bin. The data consisted of 1000 pseudo-random numbers from a uniform distribution. If the data is uniformly distributed between 0 and 1 the histogram (bar-chart_ would look like a rectangle. The graph below is approximately the shape of a rectangle and is a good indication that the data is uniformly distributed. A more rigorous statistical test (a hypothesis test for a goodness of fit) would be required to determine that the data is not uniformly distributed between 0 and 1.

Linear Congruential Generator
Where a, c, and m determine the statistical quality of the generator

Good parameter choices:
a = 16807 (IBM), or 630360016 (Simscript)
c = 0
m = 2³¹-1=2147483647
x₀ = 123457

Statistics

Mean = 0.490559

Variance = 0.081979

Correlation

	w i	w i+1
w i	1
w i+1	0.131844	1

Frequency

Random Function in Excel

N = 1000

Statistics

Mean = 0.501519

Variance = 0.080906

Correlation

	w i	w i+1
w i	1
w i+1	-0.01739	1

Frequency

See http://support.microsoft.com/kb/828795

real function random()
c
c Algorithm AS 183 Appl. Statist. (1982) vol.31, no.2, FORTRAN Code c
c Returns a pseudo-random numbers with rectangular distribution.
c between 0 and 1. The cycle length is 6.95E+12 (See page 123
c of Applied Statistics (1984) vol.33), not as claimed in the
c original article.
c
c IX, IY and IZ should be set to integer values between 1 and
c 30000 before the first entry.
c
c Integer arithmetic up to 30323 is required.
c
integer ix, iy, iz
common /randc/ ix, iy, iz
c
ix = 171 * mod(ix, 177) - 2 * (ix / 177)
iy = 172 * mod(iy, 176) - 35 * (iy / 176)
iz = 170 * mod(iz, 178) - 63 * (iz / 178)
c
if (ix .lt. 0) ix = ix + 30269
if (iy .lt. 0) iy = iy + 30307
if (iz .lt. 0) iz = iz + 30323
c
c If integer arithmetic up to 5212632 is available, the preceding
c 6 statements may be replaced by:
c
c ix = mod(171 * ix, 30269)
c iy = mod(172 * iy, 30307)
c iz = mod(170 * iz, 30323)
c
random = mod(float(ix) / 30269. + float(iy) / 30307. + float(iz) / 30323., 1.0)
return
end
c
c

Random Numbers in ExtendSim 8

N = 1000

Statistics

Mean = 0.497612

Variance = 0.08407

Correlation

	w i	w i+1
w i	1
w i+1	-0.02746	1

Frequency

ExtendSim 8 Distribution Plot Example (Real Uniform)

References & Further Readings:
Aiello W., S. Rajagopalan, and R. Venkatesan, Design of practical and provably good random number generators, Journal of Algorithms, 29, 358-389, 1998.
Dagpunar J., Principles of Random Variate Generation, Clarendon, 1988.
Fishman G., Monte Carlo, Springer, 1996.
James, Fortran version of L'Ecuyer generator, Comput. Phys. Comm., 60, 329-344, 1990.
Knuth D., The Art of Computer Programming, Vol. 2, Addison-Wesley, 1998.
L'Ecuyer P., Efficient and portable combined random number generators, Comm. ACM, 31, 742-749, 774, 1988.
L'Ecuyer P., Uniform random number generation, Ann. Op. Res., 53, 77-120, 1994.
L'Ecuyer P., Random number generation. In Handbook on Simulation, J. Banks (ed.), Wiley, 1998.
Maurer U., A universal statistical test for random bit generators, J. Cryptology, 5, 89-105, 1992.
Sobol' I., and Y. Levitan, A pseudo-random number generator for personal computers, Computers & Mathematics with Applications, 37(4), 33-40, 1999.
Tsang W-W., A decision tree algorithm for squaring the histogram in random number generation, Ars Combinatoria, 23A, 291-301, 1987

Horses Use Mathematics

Horse's might be smarter than fifth graders. They use math frequently. The picture shows my daughter and niece lopping across Coogan's Ranch near Buena Vista, CO., in late July.

The lope (or relaxed canter) is a three-beat Western riding gait, meaning that there are three footfalls heard per stride. Each footfall is the "grounding" phase of a leg. The three footfalls are evenly spaced, and followed by the "suspension" phase of the gait, which is when all four legs are off the ground. The three beats and suspension are considered one stride. The movement for one stride is as follows:

1. Beat One: the grounding phase of the outside hind leg. There are many riders who think a front leg is the first beat of the canter, which is incorrect. At this time, the other three legs are off the ground.
2. Beat Two: the simultaneous grounding phase of the inside hind leg and outside fore leg. The inside fore leg is still off the ground. The outside hind leg (beat one), is still touching the ground, but is about to be lifted off.
3. Beat Three: The grounding phase of the inside foreleg. The outside hind leg (beat one), is off the ground. The inside hind leg and outside foreleg are still touching the ground, but are about to be lifted up.
4. The inside hindleg and outside foreleg (beat two) are lifted off the ground. The inside foreleg is the only foot supporting the horse's weight.
5. The inside foreleg is lifted off the ground.
6. Suspension: The horse has all four legs off the ground.

The faster the horse is moving, the longer is the time of the phase of suspension relative to the time of the three beats.

The lope and gallop are related gaits, as the rider simply asks the horse to gallop from the canter by allowing it to lengthen its stride until it is four-beat, rather than three-beat. When the stride is sufficiently lengthened, the diagonal pair of beat two breaks, resulting in the inside hind striking first, before the outside fore. The horse is able to easily move in and out of the gallop using the canter.

The trot is a two beat diagonal gait of the horse where the diagonal pairs of legs move forward at the same time. There is a moment of suspension between each beat.

What is a Mathematical Model?

Recently I saw a question on Answer.com that was not answered, so I took the opportunity to provide a response. First, not all models are mathematical. Modeling in general is to pretend that one deals with a real thing while really working with an imitation. In operations research the imitation is a computer model of the simulated reality. A flight simulator on a PC is also a computer model of some aspects of the flight: it shows on the screen the controls and what the "pilot" (the youngster who operates it) is supposed to see from the "cockpit" (his armchair).

In order to understand what it means to model a phenomena or process, we must first understand the term "model" and understand its limitation. A model is a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process . A model can also be thought of as an abstraction of the real world, or an approximation of it. If you think about the problem of modeling a human being, or just the mind of a human, you can immediately see the limitations of modeling. We can use the term "system" to encompass systems, entities, phenomenon, or process. Since a modeling is a representation, abstraction, or approximation of the "system" being modeled, we must understand that it is not an "exact" representation, i.e., we can't model every aspect of the system. First, we don't know everything we need to know in order to model the system. We may not be able to define a process of the system with mathematical precision, or with heuristic algorithms, and many of the processes may not appear logical. Second, even if we were to know everything about the system, we may not have enough computing power to model every process, at least for complex systems, e.g., a human being, the earth's ecosystem, etc.

Yet with their limitations, models are a good way to gain understanding of how a system operates. George E.P. Box said, "All models are wrong; some models are useful." They are wrong in that there is not a one-to-one mapping form the real system to the model; they are useful in that we can use them to understand the system, or at least certain aspects of the system. Even though we often use models to predict behavior, this is a dangerous process, and we must do it with caution.

There are many types of models that one can use when trying to represent a system. These can be divided into several classes, and for simplicity we will consider two classes: physical models and symbolic models . Physical models include mock-up models (e.g., a vehicle mock-up), scale models (e.g., a 1:48 scale aircraft model), Iconic model, natural model, or a fashion model (which represent what we all want to look like).
Symbolic models include narrative models, graphical models, tabular models, software models, and mathematical models. These models are not necessarily mutually exclusive. For example a tabular model might contain data derived from a mathematical model, or a mathematical model may be embedded in software code.

So, a mathematical model is a symbolic model, or a symbolic representation or imitation of a real system or phenomenon. An example of a model is the equation for a line (a liner model) y = mx + b. A more complicate mathematical model is a model model is the waiting time for the drive through window at McDonalds at lunchtime (I will not confuse you with the mathematical formula). We do have models that cover all times during the day (at the drive through), but the solutions to these models may not be derived, "mathematically." We use numeric approximations, heuristic models (or algorithms) and simulations to solve these.

Selected mathematical modeling references:

1. http://en.wikipedia.org/wiki/Mathematical_model
2. http://www.math.montana.edu/frankw/ccp/modeling/topic.htm
3. http://www.meaningfulmath.org/modeling
4. http://www.learn.motion.com/products/modeling/index.html
5. http://www.stt.msu.edu/~mcubed/modeling.html
6. http://www.blurb.com/bookstore/detail/1461238
7. http://www.mat.univie.ac.at/~neum/model.html
8. http://www.math.colostate.edu/~pauld/M331/

You Can't Simulate Everything

Things you cannot simulate include: (1) the effect of coffee and chocolate on human behavior, (2) what my youngest daughter will say next, (3) whether or not I will sleep tonight, and (4) what happens when 180 children meet together, unsupervised. Well, maybe chaos theory can help us with #4.

Simulation Validation Using Test Data

In simulation, real world data is not always available for validation, or anchoring. Often we must rely on the results of live tests, e.g., flight tests. In the execution of "Results" Validation, we must consider the relationship between the test and M&S environments.

While test is probably the highest fidelity representation of the prototype system, limitations in the test program usually will not allow exercise of the system in its true production configuration or through out its complete envelope. The portion of production environment exercised by the test is depicted in the circles in the Venn diagram in Figure 1 (proportions are for illustration only - no scale is intended).

From Figure 1 we see that some functionality (data) of the test are representative of the production system while others are not. Now consider a simulation of both the test (by a HWIL simulator) and of the production system as shown in the rectangles of Figure 1. Simulation of each environment/configuration of is not an exact representation. The differences between the test and HWIL simulation as well as the difference between the production system and the system simulation must be accounted for.

Figure 1: Simulation of the Test and Production Systems

Two points are worth noting from the above figure. First, while the test may be a higher fidelity representation of the prototype, only some of the functionality applies to the production system. Secondly, the simulation of the system/test is not exact. It does not cover all of the behaviors and also may provide outputs that would not be allowed by the actual configurations. Therefore, care must be taken in the selection of the data that will be used for Results Validation.
The Results Validation Process must define the “accuracy” of the data and determine which data can be used for the purpose of ensuring the fidelity of the M&S. In general, the following assumptions are made for the Results Validation process:

1. The tests are simulations of the prototype (as is) system.

2. The test is the most accurate representation of the prototype system.

3. Once validated by test, the HWIL simulation is the next most accurate simulation of the prototype system.

4. Once validated, the digital simulation follows the HWIL simulator as a depiction of the prototype system.

5. Once validated, the digital simulation is the best representation of the production system in its’ intended environment.

Toast to the Flag

What does this have to do with simulation? In one sense, it has nothing to do with simulation. In another sense it has everything to do with it, at least for me. Now, I may die a POOR man, but I will also die as a man of integrity and honor. For me, the flag represents an ideal America and the ideas of our founding fathers (since it is a representation, I suppose it is a model). I don't always like what our country or its leaders do. I don't always like the direction in which we're heading. But I do love the ideal America, and this is what the flag really represents.

I served in the military for 24 yours to protect the flag (not the cloth--the idea). I hope that as you read, you'll consider your own flag, whether it be that of India, Indonesia, China, Russia, or South Africa. The flag is why I do simulation.

I delve into simulation to find the means for a safer and more secure country, an economically stable country, an environmentally friendly country, and even a better educated country. I believe that I am serving the idea of freedom and people of this country, not so much its leaders. If you're reading this from India, Argentina or another country, I hope you'll have the same heart for your country. With simulation we can experiment with ways to have a safer world, better communication, etc.

Finally, here is a Toast to the Flag:

Here’s to the red of it —
There’s not a thread of it,
No, nor a shred of it
In all the spread of it
From foot to head.
But heroes bled for it,
Faced steel and lead for it,
Precious blood shed for it,
Bathing it red!

Here’s to the white of it —
Thrilled by the sight of it,
Who knows the right of it
But feels the might of it
Through day and night?
Womanhood’s care for it
Made manhood dare for it,
Purity’s prayer for it
Keeps it so white!

Here’s to the blue of it —
Beauteous view of it,
Heavenly hue of it,
Star-spangled dew of it
Constant and true;
Diadems gleam for it,
States stand supreme for it,
Liberty’s beam for it
Brightens the blue!

Here’s to the whole of it —
Star, stripes and pole of it,
Body and soul of it,
O, and the roll of it,
Sun shining through;
Hearts in accord for it
Swear by the sword for it,
Thanking the Lord for it,
Red, White and Blue!