## Let's look at the binary representation of floating point numbers¶

In :
function float32_as_binary(x::Float32)
bs = bitstring(x);
println(" ", bs[1:1], "    ", bs[2:9], "    ", bs[10:32]);
println("sgn   exponent           mantissa");
end

function binary_as_float32(sgn::String, exponent::String, mantissa::String)
@assert(length(sgn)==1);
@assert(length(exponent)==8);
@assert(length(mantissa)==23);
return reinterpret(Float32,parse(UInt32,sgn*exponent*mantissa;base=2));
end

Out:
binary_as_float32 (generic function with 1 method)
In :
x = Float32(3.0)

Out:
3.0f0

$3 = 11_b = 1.1_b \times 10_b^1 = 1.1_b \times 10_b^{128 - 127} = 1.1_b \times 10_b^{10000000_b - 127}$

(sign) (exponent) (mantissa) 0 10000000 100000000000...0

In :
float32_as_binary(x)

 0    10000000    10000000000000000000000
sgn   exponent           mantissa

In :
float32_as_binary(Float32(3.5))

 0    10000000    11000000000000000000000
sgn   exponent           mantissa

In :
binary_as_float32("0","00000000","00000000000000000000001")

Out:
1.0f-45
In :
reinterpret(Float64, UInt64(1))

Out:
5.0e-324
In :
Float32(1.0) / Float32(0.0)

Out:
Inf32
In :
Float32(0.0) / Float32(0.0)

Out:
NaN32
In :
sqrt(Float32(-1.0))

DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).

Stacktrace:
 throw_complex_domainerror(::Symbol, ::Float32) at ./math.jl:33
 sqrt(::Float32) at ./math.jl:557
 top-level scope at In:1
In :
Float32(1.0) / Float32(0.0) - Float32(1.0) / Float32(0.0)

Out:
NaN32
In :
neg0 = binary_as_float32("1","00000000","00000000000000000000000")

Out:
-0.0f0
In :
Float32(1.0) / neg0

Out:
-Inf32
In :
-Float32(0.0)

Out:
-0.0f0

If $x = y$, does that mean that $f(x) = f(y)$?

In :
x = -Float32(0.0);
y = Float32(0.0);
x == y

Out:
true
In :
x / x == x / x

Out:
false
In :
Float32(1.0) / Float32(3.0)

Out:
0.33333334f0
In :
float32_as_binary(0.33333334f0)

 0    01111101    01010101010101010101011
sgn   exponent           mantissa

In :
float32_as_binary(Float32(1.0) / Float32(3.0))

 0    01111101    01010101010101010101011
sgn   exponent           mantissa

In :
Float64(Float32(1.0) / Float32(3.0))

Out:
0.3333333432674408
In :
0.1f0

Out:
0.1f0
In :
Float64(0.1f0)

Out:
0.10000000149011612
In :
abs(Float32(1.0) / Float32(3.0) - Float64(1.0) / Float64(3.0)) / (Float64(1/3))

Out:
2.9802322443206464e-8

machine epsilion: $1.2 \times 10^{-7}$

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

In [ ]:



## Why randomized rounding?¶

Randomized rounding often does a better job of preserving the mean of a large vector than nearest-neighbor rounding.

Let's suppose we have a dataset, we're going to quantize it into an 8-bit integer, and then measure the resulting mean.

In [ ]:
using Statistics

In [ ]:
N = 1024;
X_original = 0.1 * randn(N) .+ 3.4;

In [ ]:
mean(X_original)

In [ ]:
X_8bit_nearest = Int8.(round.(X_original));

In [ ]:
mean(X_8bit_nearest)

In [ ]:
X_8bit_randomized = Int8.(floor.(X_original .+ rand(N)));

In [ ]:
mean(X_8bit_randomized)

In [ ]:
abs(mean(X_8bit_randomized) - mean(X_original))

In [ ]:
abs(mean(X_8bit_nearest) - mean(X_original))

In [ ]:


In [ ]:


In [ ]: