Interface - The Journal of Wheel/Rail Interaction

WHEEL/RAIL INTERFACE

Applying Quality Concepts to the Wheel/Rail Interface
(Part 2 of 3)
By Stephen S. Woody • January, 2008

Part I of this article examined the results of a six-sigma project that Norfolk Southern initiated to improve the efficiency of its grinding program. Part 2 illustrates some the data analysis-related problems that NS experienced and learned from during other six-sigma projects.

One of the most common mistakes made in data analysis is failing to ensure that the data meets the underlying assumptions of a statistical test. When this happens, the analyst may draw a false conclusion from the result. Underlying assumptions to check include whether the:

• data is normal or non-normal.

• variations of each distribution being tested are equal.

• shapes of the distributions being tested are the same.

• number of samples is insufficient, correct, or excessive. (This will be explored in more detail.)

In order for a two-sample “t-test” to be valid, for example, both sample datasets must be normal and the variances must be tested for equality before running the t-test. For another example, the residuals must be normal, independent of each other, and have constant variance for a regression equation to be valid. It is possible to have an invalid regression equation even when the R² value is high.

Another common mistake made in data analysis is failing to define a practically significant test result as opposed to a statistically significant difference. A practically significant result is one where the difference between sample datasets is large enough to justify action from an economic or technical standpoint.

Take a scenario, for example, in which management wants to know if top-of-rail lubricators should be installed on a route. Assume that the decision will be based on a test of lateral forces on the high rail before and after installing a top-of-rail lubricator at a test site. Also assume that analysis indicates that there is a statistically significant 0.25-kip difference in lateral forces on the high rail before and after lubrication. This means that the distributions of samples taken before and after lubrication are different, and that the high rail forces did change. If, however, the average force reduction must be at least 2 kips in order to justify the cost of the top-of-rail lubricators, then there is no practical difference between the two sample datasets even though there is a statistical difference between them.

A closely related mistake is failing to use an appropriate number of samples for a statistical test. Many wheel/rail practitioners believe that there can never be enough data, and, consequently, take 1,000, 10,000 or even 100,000 samples. Some statistical tests, however, are extremely sensitive to the number of samples used.

For example, in order to compare the averages of two sample datasets, the statistical test constructs a theoretical distribution of means — called a t-distribution — for each dataset and then checks the amount of overlap of these t-distributions, not the overlap of the actual datasets themselves. If the t-distributions overlap more than a specified amount, the test result will show that there was no statistical difference between the sample datasets. However, the width of each t-distribution and thus the potential for overlap varies with the standard deviation of the actual sample dataset and the number of samples in the dataset.

Figure 1 shows the effect of sample size on statistical testing. The top graph in Figure 1 shows two actual sample datasets. Both distributions are normal and have the same standard deviation. The bottom left graph in Figure 1 shows the t-distribution for each dataset that would result if both sample datasets contained 30 samples. Note that the t-distributions overlap considerably, so the statistical test result should indicate that there is no difference between the two actual distributions. The bottom right graph in Figure 1 shows the t-distributions that would result if both sample datasets contained 200 samples. Note that the t-distributions do not overlap, so the statistical test result should indicate that there is a statistical difference between the two actual distributions. The surprising result is that you can get two different statistical test results for the same data solely due to the number of samples taken.

PAGE 1 OF 2 | NEXT PAGE >