• Tom Lane's avatar
    Improve type numeric's calculations for ln(), log(), exp(), pow(). · 7d9a4737
    Tom Lane authored
    Set the "rscales" for intermediate-result calculations to ensure that
    suitable numbers of significant digits are maintained throughout.  The
    previous coding hadn't thought this through in any detail, and as a result
    could deliver results with many inaccurate digits, or in the worst cases
    even fail with divide-by-zero errors as a result of losing all nonzero
    digits of intermediate results.
    
    In exp_var(), get rid entirely of the logic that separated the calculation
    into integer and fractional parts: that was neither accurate nor
    particularly fast.  The existing range-reduction method of dividing by 2^n
    can be applied across the full input range instead of only 0..1, as long as
    we are careful to set an appropriate rscale for each step.
    
    Also fix the logic in mul_var() for shortening the calculation when the
    caller asks for fewer output digits than an exact calculation would
    require.  This bug doesn't affect simple multiplications since that code
    path asks for an exact result, but it does contribute to accuracy issues
    in the transcendental math functions.
    
    In passing, improve performance of mul_var() a bit by forcing the shorter
    input to be on the left, thus reducing the number of iterations of the
    outer loop and probably also reducing the number of carry-propagation
    steps needed.
    
    This is arguably a bug fix, but in view of the lack of field complaints,
    it does not seem worth the risk of back-patching.
    
    Dean Rasheed
    7d9a4737
numeric_big.out 367 KB