From: Stan Hilliard
Date: 14 Feb 2000
Time: 00:21:29
If you have read this message prior to 2/14/00, note that I have replaced the old text with this new text, please neglect the old. This is the correction.
I agree with your strategy to target the specific groups of consumers where the problems are found. This strategy is similar to over-stress testing -- a common technique in reliability studies.
An independent two-sample t-test will determine if the difference between the two designs is significant. The data would follow the binomial distribution but the t-test of two means, based on the normal, will be valid if n*p' >=5 n=sample size, p=fraction defective. This same method will compare incident rates of designs, of countries, or of regions.
Using the two-sample t-test, the test statistic is:
t=[x1-x2]/[SE]
where:
x1=sample fraction defective of the old design.
x2=sample fraction defective of the new design.
SE=standard error of the difference: x1-x2.
SE=[sigma*sqr(1/n1+1/n2)]
sqr=square root
sigma=standard deviation of individuals (ie. for n=1) calculate at p1=population fraction defective of the old design.
sigma=sqr[(p1*(1-p1)]
sigma=sqr[0.001*0.999)]=.03162
(The reason for evaluating sigma and SE at p1 is that p1 is the hypotheses that acceptance of the new design is calculated from.)
You can make the development of this sampling plan easy with a sampling plan for the mean difference using TP414 for variables. It is described at:
www.samplingplans.com/software_oc.htm
To use TP414 for this problem, enter for sigma, the standard error, SE, of differences based on n=1.
SE input to program =sigma*sqr[1/1 + 1/1] =0.03162*sqr[2] =0.03162*1.414 =0.04471
In running the program you would choose, fixed-n for the mean with upper limit, SE=0.04471(the program labels it sigma), AQL=p2=0.0001, RQL=p1=0.001 (this will detect a 10X improvement), alpha=0.05, beta=0.05. The data to compare to the decision rules will be x1-x2.
If the sampling plan accepts, you will be accepting the new design and rejecting the old design as significantly high. After carrying out the sampling plan, check to assure that the n*p>=5 criterion was met.
Minimizing n: Hopefully, the failure rate p1 will be much higher within the target groups, because the above plan will have an n of many thousands if p1=0.001. I would design a series of plans based on a range of p1's going from 0.001 up to 0.5. Then, if you experience many failures with the old design, use a plan with high RQL to make and justify the decision. Since RQL=p1 effects SE, you will have to recalculate SE for each RQL.
Minimizing n further: TP414 can design the sample as either a fixed-n plan or a sequential sampling plan. The sequential plan is expected to reduce the sample size, and has the capability of large reduction in sample size if the new design is much different from the old. That is, if p2<>p2.
Large samples cause sequential tables to take up many pages - one line per sample item. I recommend generating the fixed-n plans before generating the sequentials. Then generate the classical sequential plan in graphic form. This will take only one page per graph. Generate the od curves. Last, generate the sequential tables on screen and print if needed.
You can see the output of a sequential sampling for the mean here:
www.samplingplans.com/outputvariablesmean.htm
Sincerely, Stan Hilliard