Skip to main content
deleted 58 characters in body
Source Link
grg
  • 205.6k
  • 45
  • 364
  • 495

They are not storing π with unusual floating-point precision. They are using an incorrect value for π with double precision. To approximate 3.1415926536 in binary, at least 38 bits are required:

$$ 3.14159265359922… > 11.001001000011111101101010100010001001 $$

[something is wrong with this server's markdown]

3.14159265359922… > 11.001001000011111101101010100010001001

Notice that 2^-36 is about 1.5e-11, which coincides with the trailing 99. Double-precision floating-point has a 52-bit significand. To evaluate cos(pi/2) as -5e-12, the only other possible choice would be a 48-bit type, which would be very strange.

Near 0 and π, where the derivative is nearly zero, cos(θ) cannot be calculated very accurately:

cos(3.1415926536) ≈ -0.999999999999999999999947911

That differs from -1 by about 5.2e-23, which is smaller than ε for double, so cos(3.1415926536) is calculated as exactly -1... which is incorrect.

Near ±π/2, the derivative [-sin(θ)] is nearly ±1, so the error at the input becomes the output.

cos(1.57079632679961) ≈ -4.71338076867830836e-12
cos(1.57079632679962) ≈ -4.72338076867830836e-12
cos(1.57079632680000) ≈ -5.10338076867830836e-12

I happen to have a TI calculator that displays one less digit and calculates cos(π/2) as -5.2e-12. However, it is very different electronically and was designed to give an exact value for cos(90°).

I would guess that in Spotlight, cos(pi/2) is being calculated by retrieving a value for π, converting to a decimal string, storing that as the (exact, rational) binary value 11.00100100001111110110101010001000100100001101101111 (or 10000), dividing by 2, and then essentially subtracting that from the true value of π/2. You should find out whether cos(pi/2 + cos(pi/2)) is closer to zero (it might be -2.2e-35).

Multiplication by a power of two should affect only the exponent, not the significand. It might be possible to determine how rounding is applied by repeated halving or doubling.

They are not storing π with unusual floating-point precision. They are using an incorrect value for π with double precision. To approximate 3.1415926536 in binary, at least 38 bits are required:

$$ 3.14159265359922… > 11.001001000011111101101010100010001001 $$

[something is wrong with this server's markdown]

Notice that 2^-36 is about 1.5e-11, which coincides with the trailing 99. Double-precision floating-point has a 52-bit significand. To evaluate cos(pi/2) as -5e-12, the only other possible choice would be a 48-bit type, which would be very strange.

Near 0 and π, where the derivative is nearly zero, cos(θ) cannot be calculated very accurately:

cos(3.1415926536) ≈ -0.999999999999999999999947911

That differs from -1 by about 5.2e-23, which is smaller than ε for double, so cos(3.1415926536) is calculated as exactly -1... which is incorrect.

Near ±π/2, the derivative [-sin(θ)] is nearly ±1, so the error at the input becomes the output.

cos(1.57079632679961) ≈ -4.71338076867830836e-12
cos(1.57079632679962) ≈ -4.72338076867830836e-12
cos(1.57079632680000) ≈ -5.10338076867830836e-12

I happen to have a TI calculator that displays one less digit and calculates cos(π/2) as -5.2e-12. However, it is very different electronically and was designed to give an exact value for cos(90°).

I would guess that in Spotlight, cos(pi/2) is being calculated by retrieving a value for π, converting to a decimal string, storing that as the (exact, rational) binary value 11.00100100001111110110101010001000100100001101101111 (or 10000), dividing by 2, and then essentially subtracting that from the true value of π/2. You should find out whether cos(pi/2 + cos(pi/2)) is closer to zero (it might be -2.2e-35).

Multiplication by a power of two should affect only the exponent, not the significand. It might be possible to determine how rounding is applied by repeated halving or doubling.

They are not storing π with unusual floating-point precision. They are using an incorrect value for π with double precision. To approximate 3.1415926536 in binary, at least 38 bits are required:

3.14159265359922… > 11.001001000011111101101010100010001001

Notice that 2^-36 is about 1.5e-11, which coincides with the trailing 99. Double-precision floating-point has a 52-bit significand. To evaluate cos(pi/2) as -5e-12, the only other possible choice would be a 48-bit type, which would be very strange.

Near 0 and π, where the derivative is nearly zero, cos(θ) cannot be calculated very accurately:

cos(3.1415926536) ≈ -0.999999999999999999999947911

That differs from -1 by about 5.2e-23, which is smaller than ε for double, so cos(3.1415926536) is calculated as exactly -1... which is incorrect.

Near ±π/2, the derivative [-sin(θ)] is nearly ±1, so the error at the input becomes the output.

cos(1.57079632679961) ≈ -4.71338076867830836e-12
cos(1.57079632679962) ≈ -4.72338076867830836e-12
cos(1.57079632680000) ≈ -5.10338076867830836e-12

I happen to have a TI calculator that displays one less digit and calculates cos(π/2) as -5.2e-12. However, it is very different electronically and was designed to give an exact value for cos(90°).

I would guess that in Spotlight, cos(pi/2) is being calculated by retrieving a value for π, converting to a decimal string, storing that as the (exact, rational) binary value 11.00100100001111110110101010001000100100001101101111 (or 10000), dividing by 2, and then essentially subtracting that from the true value of π/2. You should find out whether cos(pi/2 + cos(pi/2)) is closer to zero (it might be -2.2e-35).

Multiplication by a power of two should affect only the exponent, not the significand. It might be possible to determine how rounding is applied by repeated halving or doubling.

Source Link

They are not storing π with unusual floating-point precision. They are using an incorrect value for π with double precision. To approximate 3.1415926536 in binary, at least 38 bits are required:

$$ 3.14159265359922… > 11.001001000011111101101010100010001001 $$

[something is wrong with this server's markdown]

Notice that 2^-36 is about 1.5e-11, which coincides with the trailing 99. Double-precision floating-point has a 52-bit significand. To evaluate cos(pi/2) as -5e-12, the only other possible choice would be a 48-bit type, which would be very strange.

Near 0 and π, where the derivative is nearly zero, cos(θ) cannot be calculated very accurately:

cos(3.1415926536) ≈ -0.999999999999999999999947911

That differs from -1 by about 5.2e-23, which is smaller than ε for double, so cos(3.1415926536) is calculated as exactly -1... which is incorrect.

Near ±π/2, the derivative [-sin(θ)] is nearly ±1, so the error at the input becomes the output.

cos(1.57079632679961) ≈ -4.71338076867830836e-12
cos(1.57079632679962) ≈ -4.72338076867830836e-12
cos(1.57079632680000) ≈ -5.10338076867830836e-12

I happen to have a TI calculator that displays one less digit and calculates cos(π/2) as -5.2e-12. However, it is very different electronically and was designed to give an exact value for cos(90°).

I would guess that in Spotlight, cos(pi/2) is being calculated by retrieving a value for π, converting to a decimal string, storing that as the (exact, rational) binary value 11.00100100001111110110101010001000100100001101101111 (or 10000), dividing by 2, and then essentially subtracting that from the true value of π/2. You should find out whether cos(pi/2 + cos(pi/2)) is closer to zero (it might be -2.2e-35).

Multiplication by a power of two should affect only the exponent, not the significand. It might be possible to determine how rounding is applied by repeated halving or doubling.