You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The realFrac_ generator uses the method realToFrac and a double scalar in order to smoothly interpolate between the two endpoints of the range. This causes types to round trip through Double, and may cause them to lose precision. While this is perhaps undesirable on the middle of the range by missing values, it can be particular problematic on the edges. There is no guarantee for RealFrac that this round-trip should preserve the value, and several standard library types such as Fixed do not preserve it.
On the edges, it can cause values to be generated which are below the lower bounds of the range, or above the upper bounds.
To fix this issue, the result value should be clamped to the range, and the function should include warning documentation about possible loss of precision on the inside of the range for high-precision types.
An example value which changes when converted to Double and back (generated randomly by Hedgehog)
357559679.487764797 :: Nano
becomes
357559679.487764775
So if your lower bound was 357559679.487764796 then this could have been incorrectly generated by a realFrac_ generator on Nano
The text was updated successfully, but these errors were encountered:
The realFrac_ generator uses the method
realToFrac
and a double scalar in order to smoothly interpolate between the two endpoints of the range. This causes types to round trip through Double, and may cause them to lose precision. While this is perhaps undesirable on the middle of the range by missing values, it can be particular problematic on the edges. There is no guarantee for RealFrac that this round-trip should preserve the value, and several standard library types such asFixed
do not preserve it.On the edges, it can cause values to be generated which are below the lower bounds of the range, or above the upper bounds.
To fix this issue, the result value should be clamped to the range, and the function should include warning documentation about possible loss of precision on the inside of the range for high-precision types.
An example value which changes when converted to Double and back (generated randomly by Hedgehog)
357559679.487764797 :: Nano
becomes
357559679.487764775
So if your lower bound was 357559679.487764796 then this could have been incorrectly generated by a realFrac_ generator on Nano
The text was updated successfully, but these errors were encountered: