diff options
| author | Poul-Henning Kamp <phk@FreeBSD.org> | 2003-08-13 07:21:54 +0000 |
|---|---|---|
| committer | Poul-Henning Kamp <phk@FreeBSD.org> | 2003-08-13 07:21:54 +0000 |
| commit | 3b9b37bd542e3f9c268756138ae0f6427efc0baf (patch) | |
| tree | cd35fcf44bbfe92fddaa092733a1cf28b1490bf4 /usr.bin/ministat/README | |
| parent | f0ffd81bc20e775bcf027049638020ece507554d (diff) | |
| download | src-test2-3b9b37bd542e3f9c268756138ae0f6427efc0baf.tar.gz src-test2-3b9b37bd542e3f9c268756138ae0f6427efc0baf.zip | |
A small statistics tool for gauging the statistical significance
of data from benchmarks etc. Implements "Student's t" for various
confidence levels, defaults to 95%.
If your benchmarks are not significant at the 95% confidence level,
we don't want to hear about it.
Notes
Notes:
svn path=/head/; revision=118856
Diffstat (limited to 'usr.bin/ministat/README')
| -rw-r--r-- | usr.bin/ministat/README | 50 |
1 files changed, 50 insertions, 0 deletions
diff --git a/usr.bin/ministat/README b/usr.bin/ministat/README new file mode 100644 index 000000000000..cc5fe27df73e --- /dev/null +++ b/usr.bin/ministat/README @@ -0,0 +1,50 @@ +$FreeBSD$ + +A small tool to do the statistics legwork on benchmarks etc. + +Prepare your data into two files, one number per line +run + ./ministat data_before data_after + +and see what it says. + +You need at least three data points in each data set, but the more +you have the better your result generally gets. + +Here are two typical outputs: + +x _1 ++ _2 ++--------------------------------------------------------------------------+ +|x + x+ x x x + ++ | +| |_________|______AM_______________|__A___________M_______________|| ++--------------------------------------------------------------------------+ + N Min Max Median Avg Stddev +x 5 36060 36138 36107 36105.6 31.165686 ++ 5 36084 36187 36163 36142.6 49.952978 +No difference proven at 95.0% confidence + +Here nothing can be concluded from the numbers. It _may_ be possible to +prove something if many more measurements are made, but with only five +measurements, nothing is proven. + + +x _1 ++ _2 ++--------------------------------------------------------------------------+ +| + | +| x + +| +|x x x x + +| +| |_______________A_____M_________| |_M___A____| | ++--------------------------------------------------------------------------+ + N Min Max Median Avg Stddev +x 5 0.133 0.137 0.136 0.1354 0.0015165751 ++ 5 0.139 0.14 0.139 0.1394 0.00054772256 +Difference at 95.0% confidence + 0.004 +/- 0.00166288 + 2.95421% +/- 1.22812% + (Student's t, pooled s = 0.00114018) + +Here we have a clearcut difference, not very big, but clear and unambiguous. + + |