Commit bbf4fdc2 authored by Neil Conway's avatar Neil Conway

Prevent corr() from returning the wrong results for negative correlation

values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.

Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
parent 4893eadc
...@@ -8,7 +8,7 @@ ...@@ -8,7 +8,7 @@
* *
* *
* IDENTIFICATION * IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.150 2007/06/05 21:31:06 tgl Exp $ * $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.151 2007/09/19 22:31:48 neilc Exp $
* *
*------------------------------------------------------------------------- *-------------------------------------------------------------------------
*/ */
...@@ -2274,8 +2274,7 @@ float8_corr(PG_FUNCTION_ARGS) ...@@ -2274,8 +2274,7 @@ float8_corr(PG_FUNCTION_ARGS)
if (numeratorX <= 0 || numeratorY <= 0) if (numeratorX <= 0 || numeratorY <= 0)
PG_RETURN_NULL(); PG_RETURN_NULL();
PG_RETURN_FLOAT8(sqrt((numeratorXY * numeratorXY) / PG_RETURN_FLOAT8(numeratorXY / sqrt(numeratorX * numeratorY));
(numeratorX * numeratorY)));
} }
Datum Datum
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment