Created attachment 63034 [details] experimental spreadsheet to find error with formula BINOMDIST = (391, 462, 0.8, TRUE ()) I find abberrants results in open-office-calc on the binomial distribution B(n,p). Only certain values of n and p in a timely manner produce random errors. This calculation. BINOMDIST = (391, 462, 0.8, TRUE ()) should give the probability of obtaining x=391 success on n=462 random experiment with two issues and success probability p = 0.8. The real calculation in various formal-calculator Texas or Casio gives 0.995603391, libre-office gives 0,826 .... (Serious difference!) While a formal calculator gives a probability p with 1-p< 10 ^ {-30} as soon as x>420, libre-office gives results lower than 0.830 for all n <462. It's only for x = 462 that we found the expected result: 1. Most troubling is that the calculations of libre-office become again consistent for n <> 462 and p = 0.801 and p = 0799. I have noticed this inconsistency as rare as integer values n = 462, accidentally encountered by testing the binomial distribution under libre-office with this spreadsheet attached (error in cell J7 ). P, S,: functionality on the same sheet in excel gives good results!?
*** Bug 51087 has been marked as a duplicate of this bug. ***
Confirmed, also can verify using this calculator http://stattrek.com/online-calculator/binomial.aspx Marking as NEW and Prioritizing Normal: Can prevent high quality professional work for anyone doing stats Low: Not many people would be affected Thanks for reporting, trying to find someone to tackle it
This bug additionally affects the functions B, CritBinom and NegBinomDist. In these functions the precision is bad when pow(1-p,n) is a denormalized number. If pow(1-p,n) is zero an alternative using pow(p,n) is used. For double, numeric_limits<double>::min() = 2.22507e-308 numeric_limits<double>::denorm_min() = 4.94066e-324 0.2^462 = 9.88131e-324 so 0.2^462 has nearly no significant digits. Changed to select the calculation method depending on size of p, see https://gerrit.libreoffice.org/1213
Nice catch, I was going to take a shot at this one but I'm glad that it's solved.
This may have been already fixed by http://cgit.freedesktop.org/libreoffice/core/commit/?id=5cf55f5b7800e443c4f087e72ae05abc8b7fef45 The change submitted as https://gerrit.libreoffice.org/1213 clashes with those code changes. Please check in a recent daily build or 4.0.0.beta1 if the problem is solved.
The bug is fixed, tested with LibreOffice Version 4.0.0.0.beta1 (Build ID: 87906242e87d3ddb2ba9827818f2d1416d80cc7) on Windows XP and Version 4.1.0.0.alpha0 on Debian.
Migrating Whiteboard tags to Keywords: (ProposedEasyHack -> needsDevEval) [NinjaEdit]