What is Time-Series Analysis, Part 1

This article first appeared on bicorner.com

This article shows you how to use the R statistical software to carry out some simple analyses that are common in analyzing time series data. If the reader has some basic knowledge of time series analysis it will serve them well since the principal focus of the article is not to explain time series analysis, but rather to explain how to carry out these analyses using R.

 

Sometimes the time series data set that you have may have been collected at regular intervals that were less than one year, for example, monthly or quarterly. In this case, you can specify the number of times that data was collected per year by using the ‘frequency’ parameter in the ts() function. For monthly time series data, you set frequency=12, while for quarterly time series data, you set frequency=4.

You can also specify the first year that the data was collected, and the first interval in that year by using the ‘start’ parameter in the ts() function. For example, if the first data point corresponds to the second quarter of 1986, you would set start=c(1986,2).

 

An example is a data set of the monthly live births (adjusted) in thousands for the United States, 1946-1979. The set ‘birth’ is part of the astsa package.

> require(astsa)
> birth ## monthly live births (adjusted) in thousands for the United States, 1946-1979.

Once you have read the time series data into R, the next step is to store the data in a time series object in R, so that you can use R’s many functions for analyzing time series data. To store the data in a time series object, we use the ts() function in R. For example, to store the data in the variable ‘birth’ as a time series object in R, we type:

< birthtimeseries < birthtimeseries
     Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
1946 295 286 300 278 272 268 308 321 313 308 291 296
1947 294 273 300 271 282 285 318 323 313 311 291 293
1948 297 273 294 259 276 294 316 325 315 312 292 301
1949 304 282 313 296 313 307 328 334 329 329 304 312
1950 312 300 317 292 300 311 345 350 344 336 315 323
1951 322 296 315 287 307 321 354 356 348 334 320 340
1952 332 302 324 305 318 329 359 363 359 352 335 342
1953 329 306 332 309 326 325 354 367 362 354 337 345
1954 339 325 345 309 315 334 370 383 375 370 344 355
1955 346 317 348 331 345 348 380 381 377 376 348 356
1956 344 320 347 326 343 338 361 368 378 374 347 358
1957 349 323 358 331 338 343 374 380 377 368 346 358
1958 338 329 347 327 335 336 370 399 385 368 351 362
1959 358 333 356 335 348 346 374 386 384 372 343 346
1960 346 318 359 328 333 329 366 373 367 363 337 346
1961 355 314 343 322 336 327 362 366 361 358 327 330
1962 336 326 337 316 331 331 359 350 356 347 328 336
1963 315 292 322 291 302 310 330 335 333 318 305 313
1964 301 281 302 291 297 291 311 319 317 317 296 307
1965 295 265 300 271 291 290 310 318 310 304 285 288
1966 277 260 282 274 288 287 308 312 306 304 282 305
1967 284 273 286 284 294 288 315 322 317 309 295 306
1968 300 275 301 292 298 306 326 332 329 328 308 324
1969 299 284 306 290 292 285 295 306 317 305 294 287
1970 278 261 275 256 270 264 265 284 284 275 269 275
1971 259 244 267 255 260 253 267 277 277 264 255 260
1972 261 238 257 246 254 255 273 276 286 283 261 276
1973 264 243 259 250 262 253 280 288 270 273 241 266
1974 257 242 266 241 252 250 281 278 286 278 260 272
1975 274 256 276 259 273 272 297 296 290 282 262 275
1976 262 251 285 260 272 265 296 312 289 282 274 281
1977 277

Plotting Time Series

Once you have read a time series into R, the next step is usually to make a plot of the time series data, which you can do with the plot.ts() function in R. To plot the time series of the monthly live births (adjusted) in thousands for the United States, we type:

< plot.ts(birthtimeseries)

We can see from this time series that there seems to be seasonal variation in the number of births per month: there is a peak every summer, and a trough every winter. Again, it seems that this time series could probably be described using an additive model, as the seasonal fluctuations are roughly constant in size over time and do not seem to depend on the level of the time series, and the random fluctuations also seem to be roughly constant in size over time.

Decomposing Seasonal Data

A seasonal time series consists of a trend component, a seasonal component and an irregular component. Decomposing the time series means separating the time series into these three components: that is, estimating these three components.

 

To estimate the trend component and seasonal component of a seasonal time series that can be described using an additive model, we can use the “decompose()” function in R. This function estimates the trend, seasonal, and irregular components of a time series that can be described using an additive model.

The function “decompose()” returns a list object as its result, where the estimates of the seasonal component, trend component and irregular component are stored in named elements of that list objects, called “seasonal”, “trend”, and “random” respectively.

 

For example, as discussed above, the time series of the monthly live births (adjusted) in thousands for the United States is seasonal with a peak every summer and trough every winter, and can probably be described using an additive model since the seasonal and random fluctuations seem to be roughly constant in size over time:

> birthtimeseriescomponents

The estimated values of the seasonal, trend and irregular components are now stored in variables birthtimeseriescomponents$seasonal, birthtimeseriescomponents$trend and birthtimeseriescomponents$random. For example, we can print out the estimated values of the seasonal component by typing:

> birthtimeseriescomponents$seasonal # get the estimated values of the seasonal component
Jan   Feb   Mar   Apr   May   Jun   Jul   Aug   Sep   Oct   Nov   Dec
1946  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1947  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1948  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1949  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1950  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1951  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1952  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1953  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1954  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1955  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1956  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1957  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1958  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1959  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1960  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1961  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1962  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1963  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1964  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1965  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1966  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1967  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1968  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1969  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1970  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1971  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1972  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1973  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1974  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1975  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1976  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
1977  -4.3

The estimated seasonal factors are given for the months January-December, and are the same for each year. The largest seasonal factor is for August (about 23.6), and the lowest is for February (about -25.1), indicating that there seems to be a peak in births in July and a trough in births in February each year.

 

We can plot the estimated trend, seasonal, and irregular components of the time series by using the “plot()” function, for example:

> plot(birthtimeseriescomponents)
 

The plot below shows the original time series (top), the estimated trend component (second from top), the estimated seasonal component (third from top), and the estimated irregular component (bottom). We see that the estimated trend component shows a small increase from about 300 in 1947 to about 360 in 1959, followed by a steady decrease from then on to about 280 in 1966, followed by a slight increase to about 320 in 1969, followed by a steady decease to about 260 from about 1971 through 1974 and the a slight increase to about 280 in 1979.

Seasonally Adjusting

If you have a seasonal time series that can be described using an additive model, you can seasonally adjust the time series by estimating the seasonal component, and subtracting the estimated seasonal component from the original time series. We can do this using the estimate of the seasonal component calculated by the “decompose()” function.

 

For example, to seasonally adjust the time series of the number of births per month in New York city, we can estimate the seasonal component using “decompose()”, and then subtract the seasonal component from the original time series:

> birthtimeseriescomponents > birthtimeseriesseasonallyadjusted

We can then plot the seasonally adjusted time series using the “plot()” function, by typing:

> plot(birthtimeseriesseasonallyadjusted)

You can see that the seasonal variation has been removed from the seasonally adjusted time series. The seasonally adjusted time series now just contains the trend component and an irregular component.

Forecasts using Exponential Smoothing

Exponential smoothing can be used to make short-term forecasts for time series data.

Simple Exponential Smoothing

If you have a time series that can be described using an additive model with constant level and no seasonality, you can use simple exponential smoothing to make short-term forecasts.

 

The simple exponential smoothing method provides a way of estimating the level at the current time point. Smoothing is controlled by the parameter alpha; for the estimate of the level at the current time point. The value of alpha; lies between 0 and 1. Values of alpha that are close to 0 mean that little weight is placed on the most recent observations when making forecasts of future values.

To make forecasts using simple exponential smoothing in R, we can fit a simple exponential smoothing predictive model using the “HoltWinters()” function in R. To use HoltWinters() for simple exponential smoothing, we need to set the parameters beta=FALSE and gamma=FALSE in the HoltWinters() function (the beta and gamma parameters are used for Holt’s exponential smoothing, or Holt-Winters exponential smoothing, as described below).

 

The HoltWinters() function returns a list variable, that contains several named elements.

For example, to use simple exponential smoothing to make forecasts for the time series of monthly live births (adjusted) in thousands for the United States, 1946-1979, we type:

## Holt-Winters exponential smoothing without trend and without seasonal component.> birthtimeseriesforecasts > birthtimeseriesforecasts
Call:
HoltWinters(x = birthtimeseries, beta = FALSE, gamma = FALSE)
Smoothing parameters:
  alpha: 0.71
  beta : FALSE
  gamma: FALSE
Coefficients:
  [,1]
a  278

The output of HoltWinters() tells us that the estimated value of the alpha parameter is about 0.71. This is very close to zero, telling us that the forecasts are based on both recent and less recent observations (although somewhat more weight is placed on recent observations).

 

By default, HoltWinters() just makes forecasts for the same time period covered by our original time series. In this case, our original time series included monthly live births (adjusted) in thousands for the United States, 1948-1979.

 

In the example above, we have stored the output of the HoltWinters() function in the list variable “birthstimeseriesforecasts”. The forecasts made by HoltWinters() are stored in a named element of this list variable called “fitted”, so we can get their values by typing:

> birthtimeseriesforecasts $fitted
xhat level
Feb 1946  295   295
Mar 1946  289   289
Apr 1946  297   297
May 1946  283   283
Jun 1946  275   275
Jul 1946  270   270
Aug 1946  297   297
Sep 1946  314   314
Oct 1946  313   313
Nov 1946  310   310
Dec 1946  296   296
Jan 1947  296   296
Feb 1947  295   295
Mar 1947  279   279
Apr 1947  294   294
May 1947  278   278
Jun 1947  281   281
Jul 1947  284   284
Aug 1947  308   308
Sep 1947  319   319
Oct 1947  315   315
Nov 1947  312   312
Dec 1947  297   297
.
.
.
Jan 1976  273   273
Feb 1976  265   265
Mar 1976  255   255
Apr 1976  276   276
May 1976  265   265
Jun 1976  270   270
Jul 1976  266   266
Aug 1976  287   287
Sep 1976  305   305
Oct 1976  294   294
Nov 1976  285   285
Dec 1976  277   277
Jan 1977  280   280

We can plot the original time series against the forecasts by typing:

> plot(birthtimeseriesforecasts)

The plot shows the original time series in black, and the forecasts as a red line. The time series of forecasts is much smoother than the time series of the original data here.

As a measure of the accuracy of the forecasts, we can calculate the sum of squared errors for the in-sample forecast errors, that is, the forecast errors for the time period covered by our original time series. The sum-of-squared-errors is stored in a named element of the list variable “birthtimeseriesforecasts” called “SSE”, so we can get its value by typing:

> birthtimeseriesforecasts$SSE
[1] 97559

That is, here the sum-of-squared-errors is 97559.

It is common in simple exponential smoothing to use the first value in the time series as the initial value for the level. For example, in the time series for monthly live births (adjusted) in thousands for the United States, 1946-1979, the first value is 295 in 1946. You can specify the initial value for the level in the HoltWinters() function by using the “l.start” parameter. For example, to make forecasts with the initial value of the level set to 295, we type:

> HoltWinters(birthtimeseries, beta=FALSE, gamma=FALSE, l.start=295)

As explained above, by default HoltWinters() just makes forecasts for the time period covered by the original data, which is 1946-1979 for the birth time series. We can make forecasts for further time points by using the “forecast.HoltWinters()” function in the R “forecast” package. To use the forecast. HoltWinters() function, we first need to install the “forecast” R package (for instructions on how to install an R package, see How to install an R package).

Once you have installed the “forecast” R package, you can load the “forecast” R package by typing:

> library("forecast")

When using the forecast.HoltWinters() function, as its first argument (input), you pass it the predictive model that you have already fitted using the HoltWinters() function. For example, in the case of the birth time series, we stored the predictive model made using HoltWinters() in the variable “birthstimeseriesforecasts”. You specify how many further time points you want to make forecasts for by using the “h” parameter in forecast.HoltWinters(). For example, to make a forecast of births for the years Feb 1977 to Sep 1978 (8 more months) using forecast.HoltWinters(), we type:

> birthtimeseriesforecasts2 > birthtimeseriesforecasts2 Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
Feb 1977            278   257   299   246   310
Mar 1977            278   252   303   239   317
Apr 1977            278   248   307   233   323
May 1977            278   245   311   227   328
Jun 1977            278   242   314   223   333
Jul 1977            278   239   317   218   338
Aug 1978            278   236   320   214   342
Sep 1978            278   234   322   210   346

The forecast.HoltWinters() function gives you the forecast for a year, a 80% prediction interval for the forecast, and a 95% prediction interval for the forecast. For example, the forecasted births for 1979 is about 275 births, with a 95% prediction interval of (250, 320).

 

To plot the predictions made by forecast.HoltWinters(), we can use the “plot.forecast()” function:

> plot.forecast(birthtimeseriesforecasts2)

Here the forecasts for 1946-1979 are plotted as a blue line, the 80% prediction interval as a gray shaded area, and the 95% prediction interval as a light gray shaded area.

The ‘forecast errors’ are calculated as the observed values minus predicted values, for each time point. We can only calculate the forecast errors for the time period covered by our original time series, which is 1946-1979 for the birth data. As mentioned above, one measure of the accuracy of the predictive model is the sum-of-squared-errors (SSE) for the in-sample forecast errors.

The in-sample forecast errors are stored in the named element “residuals” of the list variable returned by forecast.HoltWinters(). If the predictive model cannot be improved upon, there should be no correlations between forecast errors for successive predictions. In other words, if there are correlations between forecast errors for successive predictions, it is likely that the simple exponential smoothing forecasts could be improved upon by another forecasting technique.

 

To figure out whether this is the case, we can obtain a correlogram of the in-sample forecast errors for lags 1-20. We can calculate a correlogram of the forecast errors using the “acf()” function in R. To specify the maximum lag that we want to look at, we use the “lag.max” parameter in acf().

For example, to calculate a correlogram of the in-sample forecast errors for the birth data for lags 0-30, we type:

> acf(birthtimeseriesforecasts2$residuals, lag.max=30)

You can see from the sample correlogram that the autocorrelation at lag 0 is just crosses the significance bounds. To test whether there is significant evidence for non-zero correlations at lags 1-30, we can carry out a Ljung-Box test. This can be done in R using the “Box.test()”, function. The maximum lag that we want to look at is specified using the “lag” parameter in the Box.test() function. For example, to test whether there are non-zero autocorrelations at lags 1-30, for the in-sample forecast errors for monthly live births for the United States data (1946-1977), we type:

> Box.test(birthtimeseriesforecasts2$residuals, lag=20, type="Ljung-Box")          Box-Ljung test
data:  birthtimeseriesforecasts2$residuals
X-squared = 470, df = 20, p-value < 2.2e-16

Here the Ljung-Box test statistic is 470, and the p-value is 0.001, so there is evidence of non-zero autocorrelations in the in-sample forecast errors at lags 1-30.

 

To be sure that the predictive model cannot be improved upon, it is also a good idea to check whether the forecast errors are normally distributed with mean zero and constant variance. To check whether the forecast errors have constant variance, we can make a time plot of the in-sample forecast errors:

> plot.ts(birthtimeseriesforecasts2$residuals)

The plot shows that the in-sample forecast errors seem to have roughly constant variance over time, although the size of the fluctuations in the start of the time series (1846-1861) may be slightly less than that at later dates (e.g., 1862-1877).

 

To check whether the forecast errors are normally distributed with mean zero, we can plot a histogram of the forecast errors, with an overlaid normal curve that has mean zero and the same standard deviation as the distribution of forecast errors. To do this, we can define an R function “plotForecastErrors()”, below:

> plotForecastErrors function(forecasterrors)
{ 
# make a histogram of the forecast errors:   mybinsize    mysd    mymin    mymax    # generate normally distributed data with mean 0 and standard deviation mysd   mynorm    mymin2    mymax2       if (mymin2 < mymin) { mymin       if (mymax2 > mymax) { mymax    # make a red histogram of the forecast errors, with the normally distributed data overlaid:   mybins    hist(forecasterrors, col="red", freq=FALSE, breaks=mybins)
   # freq=FALSE ensures the area under the histogram = 1
   # generate normally distributed data with mean 0 and standard deviation mysd
   myhist FALSE, breaks=mybins)  
   # plot the normal curve as a blue line on top of the histogram of forecast errors:
   points(myhist$mids, myhist$density, type="l", col="blue", lwd=2)
}

You will have to copy the function above into R in order to use it. You can then use plotForecastErrors() to plot a histogram (with overlaid normal curve) of the forecast errors for the birth predictions:

> plotForecastErrors(birthtimeseriesforecasts2$residuals)

The plot shows that the distribution of forecast errors is roughly centered on zero, and is more or less normally distributed, although it seems to be slightly skewed to the right compared to a normal curve. However, the right skew is relatively small, and so it is plausible that the forecast errors are normally distributed with mean zero.

The Ljung-Box test showed that there is little evidence of non-zero autocorrelations in the in-sample forecast errors, and the distribution of forecast errors seems to be normally distributed with mean zero. This suggests that the simple exponential smoothing method provides an adequate predictive model for the births for the United States data (1946-1977), which probably cannot be improved upon. Furthermore, the assumptions that the 80% and 95% predictions intervals were based upon (that there are no autocorrelations in the forecast errors, and the forecast errors are normally distributed with mean zero and constant variance) are probably valid.

---

Authored by: Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

• Operations Research using Open-Source Tools
• Discrete Event simulation using ExtendSim
• Crime Analysis and Mapping
• Missile Flight Simulation
• Mathematical Modeling of Warfare and Combat Phenomenon
• Predictive Modeling and Analytics
• Using Math to Defeat the Enemy
• Verification and Validation for Modeling and Simulation
• Simulation Conceptual Modeling
• System Engineering Process and Practices
• Weird Scientist: the Creators of Quantum Physics
• Albert Einstein: No one expected me to lay a golden eggs
• The Men of Manhattan: the Creators of the Nuclear Era
• Fundamentals of Combat Modeling
• LinkedIn Memoirs
• Quantum Phaith
• Dear Mister President
• Handbook of Handguns
• Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

Where Did All The Thinking Go?

 

Some people are saying that statistical methods in data science and analytics are obsolete. These people have either just grown tired of thinking or have forgotten how to.

What is wrong with this picture?

This view has two major problems. First, espousing the idea that machine learning algorithms is the only method required for providing analytic solutions to business problems is a very naïve view. Second, this idea is philosophically dangerous and reeks with an undertone of quantitative inadequacy.

How can you be so naïve?

Naïve is being kind. What you really have is extreme arrogance. You have some people that practically no one has ever heard of, essentially saying they are smarter than the late George Box, who is not here to defend himself. They apparently know more about probability and statistics than Andrey Kolmogorov, Nikolai Smirnov, Andrey Markov, Richard Jeffrey, Adrien-Marie Legendre, John Herschel, Friedrich Bessel and Richard Cox. They want o throw away statistical models and only use machine learning algorithms, which reminds me of the King James version only movement. What I really see is a desperate cry of “We do not understand mathematics, probability or statistics, so we’ll assume it away.”

Why is this dangerous?

To me this is a no brainer, but those who propose this seem to be brainless. We (in the United States) already have a math-phobic society and an educational system that is substandard relative to many other countries. As if we have not dumbed down quantitative skills enough, we add the “for Dummies” series to add salt to the wound.

 

It seems that undergraduate programs are teaching tools, and when you ask a recent graduate to solve a real problem with a customer's licensed tool, you may hear, “Can I do it in MATLAB? That’s what I know.” We tend to want to force every problem into our favorite tool or technique, rather than solve the problem with the appropriate tool, or actually think.

 

The cry is, “Give me a tool that does not require me to apply much thought in order to use!” And many are providing such tools, along with courses to learn them, and making lots of money in the process. What we get is a society of people who do not have any critical thinking skills. Moreover, critical thinking skills are not only required for the quantitative sciences, but also in disciplines like biology (my undergraduate degree) as well. Though I am not a great writer, I am critically thinking about sentence structure, grammar, logic and so on, as I write.

Can Machines Think?

Alan Turing said they could, but he qualified his statement by saying they think differently than humans. Roger Penrose basically said “Ditto” when addressing artificial intelligence. So, are machine learning algorithms the way to solve problems? Certainly, except they are not the only way, as some might propose. If you are trying to solve a problem where all the assumptions of a linear program are met, will a genetic algorithm give a better answer? Not necessarily and probably not.

There has to be a decision process involved in choosing the best functional form for solving various problems. Decision points, like whether or not data pathologies exist, have to be weighed. Generally, if the assumptions of traditional methods are not violated, they usually yield the best results. Do an experiment. Take a problem were all the assumption of a logistic regression are met and compare the results with an artificial neural network. I performed such an experiment with a real business problem and two different logistic regression models outperformed a neural network. However, when used together in an ensemble, the logistic regression and neural network combination (using averaging) outperformed everything else in performance testing. In very simple terms, this takes the strengths of both and negates the weaknesses of either.

I also checked the results of a logistic regression uplift model built in SAS by employing a random forest in R. Although the distribution among pentile was a little different, the overall net lift was the same. So, I am not saying that machine learning algorithms should not be used, only that some logic has to be used for selecting them as the functional form of your solution method.

Should Humans Think?

They should, but there seems to be a growing thesis to not do so. “I don’t want to think!” “It makes my brain hurt!” When solving problems, we usually examine the “What” or the “So what”. However, the “Why”, though it may not be important for the business owner, should be important to the analyst. Anytime our methods produce answers, we should be asking “Why?” (and probably “How?”). I would never give my customer a solution without knowing the “Why” and the “How”. I may never be asked questions that requires my understanding of either, but as the analyst, I have to know.

 

If my solution method is a black-box, I must try to make it as “gray” as possible. One of the things we have a tendency to do is forget intuition as a legitimate problem solving process. When I produce a solution through the logical approach, I have to ask, “Does this intuitively make sense?” Does the period required for underwriting have a bearing on a decision to buy insurance from company X? Does the possession of a reward card from Citibank have a bearing on a decision to buy insurance from company X? There latter is not so intuitively clear, but we have to know why the relation exists.

Conclusion

If we were asked to build a house, would we show up with just a screwdriver? Probably not. We wound bring our complete set of tools to bear. If we were asked to make a decision for financing our new home with a mortgage, would we choose the type and mortgage company at random? Would you force the problem into a model with an unsupervised learning algorithm? (You would probably just ask who has the lowest interest rate.)

The analysis of data should produce information that is useful for making a decision. Yet, that is not all of the information. This is the fallacy of taking “Human” out of HR. When we screen every resume with software and reject some based on certain criteria, are we possibly eliminating the very best candidate for the job? The human element must be involved in decisions, no matter what the question is or in what discipline it occurs. Blindly accepting solutions is naïve and dangerous. Believing you know better than George Box is arrogant.

“All models are wrong; but some are useful”

—George Box

About The Author

Serving in the military for 24 years as a cavalry unit officer and operations research analyst, Jeffrey Strickland has been applying quantitative methods in decision making for 34 years. He has been involved in the design of long-range unmanned aerial vehicles (UAV), manned space launch systems, missile defense systems, satellite systems, and communication systems. He has developed models for predicting combat outcomes, weapon systems effectiveness, vulnerability to cyber-attacks, occurrences of crime, propensity to purchase, propensity to engage, and propensity to churn. He holds a Masters and Doctorate in Mathematics and is a Certified Modeling and Simulation Professional (CMSP). Jeffrey has published over 20 technical books and written over 300 articles and blogs.

What the Heck is Operations Research?



 

This article was first published on bicorner.com.

 

Many people probably never heard the term “Operations Research” used. Operations Research (OR), or operational research in the U.K, is a discipline that deals with the application of advanced analytical methods to help make better decisions. The terms management science and analytics are sometimes used as synonyms for operations research. Yet, in my experience OR extends far beyond either. The figure shows a hierarchy of operations research activities, and I’ll let you decide if they are also performed in analytics.

• Data Mining and Machine Learning
• Artificial Intelligence and Expert Systems
• Financial Engineering
• Games, Decision, and Strategic Planning
• Marketing Research
• Investment Science
• Experimental and Engineering Design
• Manufacturing and Production
• Logistics and Transportation
• Supply Chain Management
• Enterprise Resource Planning

 Modeling Systems and Optimization Services is an interface part that bridges OR modeling with OR tools. When implemented smoothly, it is the part that is not noticed by modelers or users.

What do Operations Research Analysts do?

The Operations Research Analyst is a jack-of-all-trades (one guy's opinion), or at least that has been my experience. Some tend to specialize in a particular area, like mathematical optimization, but I think this is a mistake. One can find people who specialize in a particular methodology or discipline, but they would be challenged to find a good Operations Research analyst without a holistic view of the problem space.

An OR’s view of the problem space is really what defines them and describes what they do. The list above displayed some of the activities that ORs engage in, but not without a holistic view of the problem space. Figure 1 depicts the entire problem space. Mathematically, we could look at it like this:

({(Analysis Space)⊂Research Space}⊂Operations Space)⊂Problem Space

 

Figure 1. The OR Problem Space (I made this up yesterday)

 

The OR Analyst must enter the problem space with the following in mind: (1) the potential operational domains, (2) the types of research that may be used, and (3) the types of analyses that may be appropriate. If one goes in having done nothing more than math programming for 10 years, that analyst is NOT an operations research analyst—they are just a math programmer.
Operations research analysts provide this holistic view, which then allows for the definition of the right problem within any domain, and application of the most appropriate research methodology, using the most appropriate analyses. You cannot build a house with just a screwdriver, unless you are MacGyver[1].

If we look at the historical context of OR, as we discussed earlier, we should be able to ascertain that anything short of a holistic point of view may have resulted in operational chaos much worse than missed dropped zones. Operation Overlord—the most complex operation ever executed—could have easily failed.

Where do you find them?

Operation Research Analysts work in many industries, including maritime, space operations, defense, airlines, train lines, financial service, entertainment and many more. Wherever operation occur, operations research analysts are usually there. The following lists several key functional areas for operations research analysts.

• Communications
• Interfaces
• Networks
• Scheduling
• Routing
• Manpower
• Modeling

What are Their tools?

Underlying Tools is the level that is typically regarded as what uniquely defines Operations Research.

• Mathematical Programming
• Computing Technology
• Probability and Statistics
• Stochastic Simulation
• Systems Analysis
• Organization Theory
• Accounting Principles
• Engineering Economics
• Decision Analysis
• Game Theory
• Heuristics
• Computer Programming
• Numeric Methods
• Stochastic Analysis
• Queuing Theory
• Evolutionary Algorithms
• Dynamic Programming

What are they built upon?

Foundations upon which OR are built include:

• Mathematical Theory
• Statistical Theory
• Computing Theory
• Economic Theory

What is their history?

Operational Research was born during the early year of WWII and matured rapidly. One of its primary function was the planning of Operation Overlord or the Normandy Invasion. It has its foundations in mathematics, computing and economic theories, on which basic tools in optimization and simulation are built.  Today OR’s are employed by airlines, train lines, logistic systems, delivery systems (e.g., FedEx), defense systems, military, oil companies, insurance companies, financial institutions, manufacturing, marketing and many more.

---

Who Wrote This?

Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

• Operations Research using Open-Source Tools (new)

• Operations Research using Open-Source Tools (new)
• Discrete Event simulation using ExtendSim
• Crime Analysis and Mapping
• Missile Flight Simulation
• Mathematical Modeling of Warfare and Combat Phenomenon
• Predictive Modeling and Analytics
• Using Math to Defeat the Enemy
• Verification and Validation for Modeling and Simulation
• Simulation Conceptual Modeling
• System Engineering Process and Practices
• Weird Scientist: the Creators of Quantum Physics
• Albert Einstein: No one expected me to lay a golden eggs
• The Men of Manhattan: the Creators of the Nuclear Era
• Fundamentals of Combat Modeling
• LinkedIn Memoirs
• Quantum Phaith
• Dear Mister President
• Handbook of Handguns
• Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

#PredictiveAnalytics

What the Heck are Predictive Analytics Models?

Predictive Modeling and Predictive Analytics does not lie solely in the domain of Big Data Analytics or Data Science. I am sure that there are a few “data scientist” who think they invented predictive modeling. However, predictive modeling has existed for a while and at least since World War II. In simple terms, a predictive model is a model with some predictive power. I will elaborate on this later.
I have been building predictive models since 1990. Doing the math, 2015 – 1990 = 25 years, I have been engaged in the predictive modeling business longer that data science has been around. My first book on the subject, "Fundamentals of Combat Modeling (2007), predates the "Data Science" of 2009 (see below).

How old is Data Science?

It is really a trick question. The term was first used in 1997 by C. F. Jeff Wu. In his inaugural lecture for the H. C. Carver Chair in Statistics at the University of Michigan, Professor Wu (currently at the Georgia Institute of Technology), calls for statistics to be renamed data science and statisticians to be renamed data scientists. That idea did not land on solid ground, but the topic reemerges in 2001 when William S. Cleveland publishes “Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics.” But it is really not until 2009 that data science gains any significant following and that is also the year that Troy Sadkowsky created the data scientists group on LinkedIn as a companion to his website, datasceintists.com (which later became datascientists.net). [1]

What is Predictive Modeling?

It is not a field of statistics! Yes, we do predictive modeling in statistics, but it is really a multidisciplinary field and is based more in mathematics than in other fields. Now, if you consult the most authoritative source of factual information available to the world, Wikipedia, you will find an incorrect view of predictive modeling (of course, I do not believe what I said about Wikipedia). It was formed by people with too much time on their hands and too little exposure to other disciplines, such as physics and mathematics.

Predictive modeling may have begun as early as World War II in the Planning of Operation Overlord, the Normandy Invasion, but was certainly used in determining air defenses and bombing raid sizes (it may have appeared as early as 1840 [2]). Now, this is not an article about the history of operations research, so suffice it to say that the modern field of operational research arose during World War II. In the World War II era, operational research was defined as “a scientific method of providing executive departments with a quantitative basis for decisions regarding the operations under their control.”[3]

What is a Predictive Model?

The answer is easy: a model with some predictive power. I say that with caution, and use the word “some”, because more often than not, decision makers think that these model are absolute. Of course, they become very disappointed when the predictions do not occur as predicted. Rather than expand on my simplistic definition, I think some examples my help.

Examples of Predictive Models

The taxonomy of predictive models represented here is neither exhaustive of exclusive. In other words, there are other ways to classify predictive models, but here is one.
Times Series Models/Forecasting Models. This kind of model is a statistical model based on time series data. It uses “smoothing” techniques to account for things like seasonality in predicting or forecasting what may happen in the near future. These models are based on time-series data.

Regression Models. Time series model are technically regression models, but machine learning algorithms like auto neural networks have been employed recently in Time Series Analysis. Here I am referring to logistic regression models used in propensity modeling, and other regression models like linear regression models, robust regression models, etc. These models are based on data.

Physical models. These models are based on physical phenomena. They include 6-DoF (Degrees of Freedom) flight models, space flight models, missile models, combat attrition models (based on physical properties of munitions and equipment).

Machine Leaning Models. These include auto neural networks (ANN), support vector machines, classification trees, random forests, etc. These are based on data, but unlike statistical models, they “learn” from the data.

Weather models. These are forecasting models based on data, but the amount of data, the short interval of prediction windows and the physical phenomena involved make them much different that statistical forecasting models.

Mathematical Models. These are usually restricted to continuous time models based on differential equations or estimated using difference equations. They are often used to model very precise processes like the dynamics solid fuel rockets, or to approximate physical phenomena in the absence of actual data, like attrition coefficients approximation or direct fire effects in combat models.

Statistical Models. The first two examples, Time Series and Regression models, are statistical models. However, I list it separately because many do not realize that statistical models are mathematical models, based on mathematical statistics. Things like means and standard deviations are statistical moments, derived from mathematical moment generating functions. Every statistic in Statistics is based on a mathematical function.

What Predictive Models have I Built?

I have built predictive models in all example categories except weather models. Models I have built include Reliability, Availability and Maintainability (RAM) models for Unmanned Aerial Vehicle design; unspecified models involving satellites (unspecified because they are classified); unspecified missile models; combat attrition models; 6-DoF missiles models; missile defense models; propensity to purchase, propensity to engage, and share or wallet models regression models; time-series forecasting models for logistics; uplift (net-lift models) marketing models; ANN models as part of ensembles, classification trees, and random forests marketing models. I have also worked on descriptive and prescriptive models.

Models I have consulted on include the NASA Ares I Crew Launch Vehicle Reliability and Launch Availability; The Extended Range Multi-Purpose (ERMP) Unmanned Aerial Vehicle RAM Model, The Future Combat Systems (FCS) C4ISR family of models; FCS Logistic Decision Support System Test-Bed Model; Unspecified models (unspecified because they are classified).

References

1. Press, G. “A Very Short History Of Data Science”, Forbes, May 28, 2013 @ 7:09 AM, Retrieved 05-29-2015.
2. P. W. Bridgman, The Logic of Modern Physics, The MacMillan Company, New York, 1927.
3. Operational Research in the British Army 1939–1945, October 1947, Report C67/3/4/48, UK National Archives file WO291/1301. Quoted on the dust-jacket of: Morse, Philip M, and Kimball, George E, Methods of Operations Research, 1st Edition Revised, pub MIT Press & J Wiley, 5th printing, 1954

About the Author

Jeffrey Strickland, Ph.D., is the Author of Predictive Analytics Using R and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and is considered one of the worlds foremost experts in mathematical modeling of combat phenomena. He has published over 250 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 21 books including:

• Operations Research using Open-Source Tools
• Discrete Event simulation using ExtendSim
• Crime Analysis and Mapping
• Missile Flight Simulation
• Mathematical Modeling of Warfare and Combat Phenomenon
• Predictive Modeling and Analytics
• Using Math to Defeat the Enemy
• Verification and Validation for Modeling and Simulation
• Simulation Conceptual Modeling
• System Engineering Process and Practices

Connect with Jeffrey StricklandContact Jeffrey Strickland

#PredictiveModeling #PredictiveAnalytics #Analytics

What the Heck is Predictive Analytics?

[Excerpt from my new book, Predictive Analytics using R, downloadable from my profile for free]

Predictive analytics—sometimes used synonymously with predictive modeling—is not synonymous with statistics, often requiring modification of functional forms and use of ad hoc procedures, making it a part of data science to some degree. It does however, encompasses a variety of statistical techniques for modeling, incorporates machine learning, and utilizes data mining to analyze current and historical facts, making predictions about future.

In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions. Predictive models are not restricted to business, for they are used to predict anything from the reliability of an electronic component to the success of a manned lunar landing. These model, however, are usually stochastic models that can be used in a simulation.
Predictive analytics is used in actuarial science (Conz, 2008), marketing (Fletcher, 2011), financial services (Korn, 2011), insurance, telecommunications (Barkin, 2011), retail (Das & Vidyashankar, 2006), travel (McDonald, 2010), healthcare (Stevenson, 2011), pharmaceuticals (McKay, 2009), defense (Strickland, 2011) and other fields.

Definition

Predictive analytics is an area of data science that deals with extracting information from data and using it to predict trends and behavior patterns. Often the unknown events of interest is in the future, but predictive analytics can be applied to any type of unknown whether it be in the past, present or future. For example, identifying suspects after a crime has been committed, or credit card fraud as it occurs (Strickland J., 2013). The core of predictive analytics relies on capturing relationships between explanatory variables and the predicted variables from past occurrences, and exploiting them to predict the unknown outcome. It is important to note, however, that the accuracy and usability of results will depend greatly on the level of data analysis and the quality of assumptions.

Not Statistics

Predictive analytics uses statistical methods, but also machine learning algorithms, and heuristics. Though statistical methods are important, the Analytics professional cannot always follow the “rules of statistics to the letter.” Instead, the analyst often implements what I call “modeler judgment”. Unlike the statistician, the analytics professional—akin to the operations research analyst—must understand the system, business, or enterprise where the problem lies, and in the context of the business processes, rules, operating procedures, budget, and so on, make judgments about the analytical solution subject to various constraints. This requires a certain degree of creativity, and lends itself to being both a science and an art.

For example, a pure statistical model, say a logistic regression, may determine that the response is explained by 30 independent variables with a significance of 0.05. However, the analytics professional knows that 10 of the variables cannot be used subject to legal constraints imposed for say a bank product. Moreover, the analytics modeler is aware that variables with many degrees of freedom can lead to overfitting the model. Thus, in their final analysis they develop a good model with 12 explanatory variables using modeler judgment. The regression got them near to a solution, and their intuition carried them to the end.

Additionally, the Analytics professional does not always look for a hypothesis a priori. Consequently, they may use a machine learning algorithm, such as Random Forests, that does not depend upon statistical assumptions, but instead they "learn" from the data.

Types

Generally, the term predictive analytics is used to mean predictive modeling, “scoring” data with predictive models, and forecasting. However, people are increasingly using the term to refer to related analytical disciplines, such as descriptive modeling and decision modeling or optimization. These disciplines also involve rigorous data analysis, and are widely used in business for segmentation and decision making, but have different purposes and the statistical techniques underlying them vary.

Predictive models

Predictive models are models of the relation between the specific performance of a unit in a sample and one or more known attributes or features of the unit. The objective of the model is to assess the likelihood that a similar unit in a different sample will exhibit the specific performance. This category encompasses models that are in many areas, such as marketing, where they seek out subtle data patterns to answer questions about customer performance, such as fraud detection models. Predictive models often perform calculations during live transactions, for example, to evaluate the risk or opportunity of a given customer or transaction, in order to guide a decision. With advancements in computing speed, individual agent modeling systems have become capable of simulating human behavior or reactions to given stimuli or scenarios.

The available sample units with known attributes and known performances is referred to as the “training sample.” The units in other sample, with known attributes but un-known performances, are referred to as “out of [training] sample” units. The out of sample bear no chronological relation to the training sample units. For example, the training sample may consists of literary attributes of writings by Victorian authors, with known attribution, and the out-of sample unit may be newly found writing with unknown authorship; a predictive model may aid the attribution of the unknown author. Another example is given by analysis of blood splatter in simulated crime scenes in which the out-of sample unit is the actual blood splatter pattern from a crime scene. The out of sample unit may be from the same time as the training units, from a previous time, or from a future time.

Descriptive models

Descriptive models quantify relationships in data in a way that is often used to classify customers or prospects into groups. Unlike predictive models that focus on predicting a single customer behavior (such as credit risk), descriptive models identify many different relationships between customers or products. Descriptive models do not rank-order customers by their likelihood of taking a particular action the way predictive models do. Instead, descriptive models can be used, for example, to categorize customers by their product preferences and life stage. Descriptive modeling tools can be utilized to develop further models that can simulate large number of individualized agents and make predictions.

Decision models

Decision models describe the relationship between all the elements of a decision—the known data (including results of predictive models), the decision, and the forecast results of the decision—in order to predict the results of decisions involving many variables. These models can be used in optimization, maximizing certain outcomes while minimizing others. Decision models are generally used to develop decision logic or a set of business rules that will produce the desired action for every customer or circumstance.

Applications

Although predictive analytics can be put to use in many applications, I outline a few examples where predictive analytics has shown positive impact in recent years.

Clinical decision support systems

Experts use predictive analysis in health care primarily to determine which patients are at risk of developing certain conditions, like diabetes, asthma, heart disease, and other lifetime illnesses. Additionally, sophisticated clinical decision support systems incorporate predictive analytics to support medical decision making at the point of care. A working definition has been proposed by Robert Hayward of the Centre for Health Evidence: “Clinical Decision Support Systems link health observations with health knowledge to influence health choices by clinicians for improved health care.” (Hayward, 2004)

Customer retention

With the number of competing services available, businesses need to focus efforts on maintaining continuous consumer satisfaction, rewarding consumer loyalty and minimizing customer attrition. Businesses tend to respond to customer attrition on a reactive basis, acting only after the customer has initiated the process to terminate service. At this stage, the chance of changing the customer's decision is almost impossible. Proper application of predictive analytics can lead to a more proactive retention strategy.

Direct marketing

When marketing consumer products and services, there is the challenge of keeping up with competing products and consumer behavior. Apart from identifying prospects, predictive analytics can also help to identify the most effective combination of product versions, marketing material, communication channels and timing that should be used to target a given consumer. The goal of predictive analytics is typically to lower the cost per order or cost per action.

Fraud detection

Fraud is a big problem for many businesses and can be of various types: inaccurate credit applications, fraudulent transactions (both offline and online), identity thefts and false insurance claims. These problems plague firms of all sizes in many industries. Some examples of likely victims are credit card issuers, insurance companies (Schiff, 2012), retail merchants, manufacturers, business-to-business suppliers and even services providers. A predictive model can help weed out the “bads” and reduce a business's exposure to fraud.

The Internal Revenue Service (IRS) of the United States also uses predictive analytics to mine tax returns and identify tax fraud (Schiff, 2012).

Recent advancements in technology have also introduced predictive behavior analysis for web fraud detection. This type of solution utilizes heuristics in order to study normal web user behavior and detect anomalies indicating fraud attempts.

Portfolio, product or economy-level prediction

Often the focus of analysis is not the consumer but the product, portfolio, firm, industry or even the economy. For example, a retailer might be interested in predicting store-level demand for inventory management purposes. Or the Federal Reserve Board might be interested in predicting the unemployment rate for the next year. These types of problems can be addressed by predictive analytics using time series techniques. They can also be addressed via machine learning approaches which transform the original time series into a feature vector space, where the learning algorithm finds patterns that have predictive power.

Risk management

When employing risk management techniques, the results are always to predict and benefit from a future scenario. The Capital asset pricing model (CAM-P) and Probabilistic Risk Assessment (PRA) examples of approaches that can extend from project to market, and from near to long term. CAP-M (Chong, Jin, & Phillips, 2013) “predicts” the best portfolio to maximize return. PRA, when combined with mini-Delphi Techniques and statistical approaches, yields accurate forecasts (Parry, 1996). @Risk is an Excel add-in used for modeling and simulating risks (Strickland, 2005). Underwriting (see below) and other business approaches identify risk management as a predictive method.

Underwriting

Many businesses have to account for risk exposure due to their different services and determine the cost needed to cover the risk. For example, auto insurance providers need to accurately determine the amount of premium to charge to cover each automobile and driver. A financial company needs to assess a borrower's potential and ability to pay before granting a loan. For a health insurance provider, predictive analytics can analyze a few years of past medical claims data, as well as lab, pharmacy and other records where available, to predict how expensive an enrollee is likely to be in the future. Predictive analytics can help underwrite these quantities by predicting the chances of illness, default, bankruptcy, etc. Predictive analytics can streamline the process of customer acquisition by predicting the future risk behavior of a customer using application level data. Predictive analytics in the form of credit scores have reduced the amount of time it takes for loan approvals, especially in the mortgage market where lending decisions are now made in a matter of hours rather than days or even weeks. Proper predictive analytics can lead to proper pricing decisions, which can help mitigate future risk of default.

Technology and big data influences

Big data is a collection of data sets that are so large and complex that they become awkward to work with using traditional database management tools. The volume, variety and velocity of big data have introduced challenges across the board for capture, storage, search, sharing, analysis, and visualization. Examples of big data sources include web logs, RFID and sensor data, social networks, Internet search indexing, call detail records, military surveillance, and complex data in astronomic, biogeochemical, genomics, and atmospheric sciences. Thanks to technological advances in computer hardware—faster CPUs, cheaper memory, and MPP architectures—and new technologies such as Hadoop, MapReduce, and in-database and text analytics for processing big data, it is now feasible to collect, analyze, and mine massive amounts of structured and unstructured data for new insights (Conz, 2008). Today, exploring big data and using predictive analytics is within reach of more organizations than ever before and new methods that are capable for handling such datasets are proposed (Ben-Gal I. Dana A., 2014).

Analytical Techniques

The approaches and techniques used to conduct predictive analytics can broadly be grouped into regression techniques and machine learning techniques. [condensed]

Regression techniques

Regression models are the mainstay of predictive analytics.

• Linear regression model
• Ridge regression
• LASSO (Least Absolute Shrinkage and Selection Operator)
• Logic regression
• Quantile regression
• Multinomial logistic regression
• Probit regression

Classification and regression trees

• Hierarchical Optimal Discriminant Analysis (HODA)
• Classification and regression trees (CART)
• Decision trees
• Multivariate adaptive regression splines (MARS)

Machine learning techniques

Machine learning, a branch of artificial intelligence, was originally employed to develop techniques to enable computers to learn.

• Neural networks
• Multilayer Perceptron (MLP)
• Radial basis function (RBF)
• Naïve Bayes
• K-Nearest Neighbor algorithm (k-NN)

Criticism

There are plenty of skeptics when it comes to computers and algorithms abilities to predict the future, including Gary King, a professor from Harvard University and the director of the Institute for Quantitative Social Science. People are influenced by their environment in innumerable ways. Trying to understand what people will do next assumes that all the influential variables can be known and measured accurately. “People’s environments change even more quickly than they themselves do. Everything from the weather to their relationship with their mother can change the way people think and act. All of those variables are unpredictable. How they will impact a person is even less predictable. If put in the exact same situation tomorrow, they may make a completely different decision. This means that a statistical prediction is only valid in sterile laboratory conditions, which suddenly isn't as useful as it seemed before.” (King, 2014)

Tools

Tools change often, but SAS appears to be the industry standard, and I relay heavily on SAS Enterprise Modeler for my job. Be that as it may, I use R a great deal and find SPSS (particularly SPSS Modeler) useful for some things. Personally, I prefer R.

▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄

About the Author

Jeffrey Strickland is the Author of "Predictive Analytics Using R" and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries. He is also the author of 20 books including:

• Discrete Event simulation using ExtendSim
• Crime Analysis and Mapping
• Missile Flight Simulation
• Mathematical modeling of Warfare and Combat Phenomenon
• Predictive Modeling and Analytics
• Using Math to Defeat the Enemy
• Verification and Validation for Modeling and Simulation
• Simulation Conceptual Modeling
• System Engineering Process and Practices
• Weird Scientist: the Creators of Quantum Physics
• Albert Einstein: No one expected me to lay a golden eggs
• The Men of Manhattan: the Creators of the Nuclear Era
• Fundamentals of Combat Modeling

Connect with Jeffrey Strickland
Contact Jeffrey Strickland

▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄

References

Barkin, E. (2011). CRM + Predictive Analytics: Why It All Adds Up. New York: Destination CRM. Retrieved 2014, from http://www.destinationcrm.com/Articles/Editorial/Magazine-Features/CRM---Predictive-Analytics-Why-It-All-Adds-Up-74700.aspx
Conz, N. (2008). Insurers Shift to Customer-focused Predictive Analytics Technologies. New York: Insurance & Technology. Retrieved 2014, from http://www.insurancetech.com/business-intelligence/insurers-shift-to-customer-focused-predi/210600271
Das, K., & Vidyashankar, G. (2006). Competitive Advantage in Retail Through Analytics: Developing Insights, Creating Value. New York: Information Management. Retrieved 2014, from http://www.information-management.com/infodirect/20060707/1057744-1.html
Fletcher, H. (2011). The 7 Best Uses for Predictive Analytics in Multichannel Marketing. Philadelphia: Target Marketing. Retrieved 2014, from http://www.targetmarketingmag.com/article/7-best-uses-predictive-analytics-modeling-multichannel-marketing/1#
Hayward, R. (2004). Clinical decision support tools: Do they support clinicians? FUTURE Practice, 66-68.
Korn, S. (2011). The Opportunity for Predictive Analytics in Finance. San Diego: HPC Wire. Retrieved 2014, from http://www.hpcwire.com/2011/04/21/the_opportunity_for_predictive_analytics_in_finance/
McDonald, M. (2010). New Technology Taps 'Predictive Analytics' to Target Travel Recommendations. Oyster Bay: Travel Market Report. Retrieved 2014, from http://www.travelmarketreport.com/technology?articleID=4259&LP=1,
McKay, L. (2009, August). The New Prescription for Pharma. Destination CRM. Retrieved 2014, from http://www.destinationcrm.com/articles/Web-Exclusives/Web-Only-Bonus-Articles/The-New-Prescription-for-Pharma-55774.aspx
Parry, G. (1996, November–December). The characterization of uncertainty in Probabilistic Risk Assessments of complex systems. Reliability Engineering & System Safety, 54(2-3), 119–1. Retrieved 2014, from http://www.sciencedirect.com/science/article/pii/S0951832096000695
Schiff, M. (2012, March 6). BI Experts: Why Predictive Analytics Will Continue to Grow. Renton: The Data Warehouse Institute. Retrieved 2014, from http://tdwi.org/Articles/2012/03/06/Predictive-Analytics-Growth.aspx?Page=1
Stevenson, E. (2011, December 16). Tech Beat: Can you pronounce health care predictive analytics? Times-Standard. Retrieved 2014, from http://www.times-standard.com/business/ci_19561141
Strickland, J. (2013). Introduction toe Crime Analysis and Mapping. Lulu.com. Retrieved from http://www.lulu.com/shop/jeffrey-strickland/introduction-to-crime-analysis-and-mapping/paperback/product-21628219.html

Angels and Demons

Yes, most people think I am weird, and little John Nash-like. Well, I consider that a complement. This key character in this non-movie is a professor of a different sort—a math professor, or at least I once was. But, I do see both angels and demons. The angels are usually people-like yet translucent. The demons are dark and shadowy. The angels bring me messages, which is their job. The demons try to thwart me, which is theirs. Their attacks used to be subtle, but they bring open battle now, except for the scouts. Fearless, I fight. Not because I have no capacity for fear, rather no reason to fear:

[38]  For I am convinced that neither death, nor life, nor angels, nor principalities, nor things present, nor things to come, nor powers, [39] nor height, nor depth, nor any other created thing, will be able to separate us from the love of God, which is in Christ Jesus our Lord. (Romans 8:38-39) 

I do Life Safety at our church for nearly all events involving youth and children. I am licensed to carry, which does not help in the spiritual realm. So I also carry another weapon, the Bible on my i-Phone. I am skilled in Koine Greek and a novice in Hebrew, and "my hands are trained for war and my fingers for battle". Our Life Safety verse is Nehemiah 4:9 (emphasis mine):

And we prayed to our God AND set a guard as protection against them day and night. ESB

This will sound strange, but I have faced death on three occasions. After those experiences I begin to see in my heart what Christ did when he conquered death, and what Paul meant in Romans 8. Now the punch-line. I fear nothing on this earth physical or spiritual. "There is no fear in love (ἀγάπῃ), but perfect love (ἀγάπῃ) cast out all fear (1 John 4:18).

φόβος οὐκ ἔστιν ἐν τῇ ἀγάπῃ, ἀλλ᾿ ἡ τελεία ἀγάπη ἔξω βάλλει τὸν φόβον. (Α΄ ΙΩΑΝΝΟΥ 4:18a NTPT)

Oh, I am cautious and I do not pick up prairie rattlers, I shoot them, because ἀγάπῃ is not stupid either. When the dark ones come I fight them without hesitation. I had one last month that was so powerful, I fought with it all night long. I read the book ok Hebrews to it over and over. "Since therefore the children share in flesh and blood, he himself likewise partook of the same things, that through death he might destroy the one who has the power of death, that is, the devil, and deliver all those who through fear of death were subject to lifelong slavery." (Hebrews 2:14-15 ESV)
Demons cannot possess me because Christ does, and His Spirit is within me. And he who is in me is greater than he who is in the world. When I walk my beat I look for them, and if I sense them, I call them out to do battle. They do not like it; they want the initiative. So I put on the whole armor of God and fight. That is what soldiers do.

For years I struggled with being somewhere else when war broke out. I trained the unit that defeated the Republican Guard at 73 Easting. But I was not there. I used to ask God, "Why did you make me a soldier and not use me?" now I know why. He has given me the gift of Discernment and allowed me to see partially into the spiritual realm.

Before I do my duty, I prepare my weapons, a Beretta PX Storm 9 mm and my Bible, and put on my body armor and the whole armor of God. And “though I walk through the valley of death, I fear no evil” for my 9mm and my Bible they comfort me. I train for that fight. If you lived in Colorado and worked at Schriever AFB, you might see as I walk along Curtis Road with 50lbs on my back regularly. And I live fire just about every two weeks. Ephesians 6:11-13New International Version (NIV)

[11] Put on the full armor of God, so that you can take your stand against the devil’s schemes. [12] For our struggle is not against flesh and blood, but against the rulers, against the authorities, against the powers of this dark world and against the spiritual forces of evil in the heavenly realms. [13] Therefore put on the full armor of God, so that when the day of evil comes, you may be able to stand your ground, and after you have done everything, to stand.

Now, theologically, God created angels before humans and they are a different order of created beings. They do not procreate, and they are a lower class of being than humankind. When Lucifer revolted he took a third with him. And the creative part of God, the Λόγος, also defeated death and Satan.

Ἐν ἀρχῇ ἦν ὁ Λόγος, καὶ ὁ Λόγος ἦν πρὸς τὸν Θεόν, καὶ Θεὸς ἦν ὁ Λόγος. (ΚΑΤΑ ΙΩΑΝΝΗΝ 1:1 NTPT)

The Λόγος created all things and all things hold together through Him. He made us in His image, not angels. They are a separate creation, created on the first day of Creation. When we die, we do not become angels, for we are a higher order, made a little while lower, when sin entered to world. If you are a Christian you command angels and can bid them to help you. I do that in a fight.
The first book I wrote, Quantum Phaith" has a lot of this stuff in it. Most people have an understanding of angels that is 97% wrong. I base my understanding holistically on Scripture and not along lines of church dogma. Sometime when people ask me what denomination I am, I tell them I am Batholic and that I worship all Gods that include Christ in the Godhead. In reality I am a Christ follower, only.

Quantum Phaith, where phaith is a science pun for faith, took 10 year to write. My exegesis in New Testament Greek was on John 1, which is why I know a lot about λόγος, the spoken word. Anytime in the Old Testament when God speaks, it is the λόγος, the reincarnate Christ. Moses met Him at the burning bush, for example. When He became incarnate, Christ dwelt among men. When He ascended into heaven, he sent the Spirit to dwell in us. Christians, because we are “possessed by the Spirit” cannot be possessed or harmed by demons. Demons, Satan including do not understand this because they are not omniscient. And without the Spirit, which they do not have, they cannot understand Scripture, although they could recite it from cover to cover.

When the angels were created on the first day of creation, they were perfect creatures, like all of God’s creation. But Lucifer, the archangel, allowed pride to consume him and revolted taking a third of the angels with him. And it was Lucifer who tempted Eve in the garden. Eve however, did not commit the “original sin”, for if you read the Hebrew carefully, you will see that Adam was right by her side. And it was Adam that had received God’s command not to eat from the tree in the center of the garden, and he said nothing to Eve when she was tempted. So yes, it was the man Adam who sinned and allowed Eve to sin, even though she did not know of the command. Prior to that time, humankind, made in the image of God, were superior to the angels.

So, when I speak of angels and demons, I speak of the spiritual realm. Psychopaths and sociopaths are probably demon possessed, but they are not demons, just people without the Spirit in them. When Christians die (bodily) they become saints. Sainthood is not something that the church awards to fallen believers who have performed great works or miracles, although they think they do. Sainthood is appointed by the λόγος. Some when you get to heaven, wherever that may be, you will find Saint Peter and Saint Julie standing together and worshipping the Lamb in the same manner. We are foolish to play God and elevate one above another.