• Dean Rasheed's avatar
    Fix corner-case loss of precision in numeric_power(). · 8e26b868
    Dean Rasheed authored
    This fixes a loss of precision that occurs when the first input is
    very close to 1, so that its logarithm is very small.
    
    Formerly, during the initial low-precision calculation to estimate the
    result weight, the logarithm was computed to a local rscale that was
    capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
    as close as 1e-16383 to 1, hence its logarithm may be as small as
    1e-16383, and so the local rscale needs to be allowed to exceed 16383,
    otherwise all precision is lost, leading to a poor choice of rscale
    for the full-precision calculation.
    
    Fix this by removing the cap on the local rscale during the initial
    low-precision calculation, as we already do in the full-precision
    calculation. This doesn't change the fact that the initial calculation
    is a low-precision approximation, computing the logarithm to around 8
    significant digits, which is very fast, especially when the base is
    very close to 1.
    
    Patch by me, reviewed by Alvaro Herrera.
    
    Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
    8e26b868
numeric.c 275 KB