JoelMc JoelMc - 3 months ago 17
C# Question

Turn random bytes to .NET decimal 0..1 fractional range

I'm playing with a TRNG usb device, and successfully turning the random bytes into various usable data in C#.

I'd like to create a .NET decimal value between 0..1 (ex: 0.37327) directly from the bytes using binary reader (or other direct bytes -> decimal) method.

// assume: byte[] random_data of appropriate length, patterned for a 0..1 range
decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();


I was looking for the byte format of decimal, but it looks like it may not be a standard?

This is for hobby work, so it's acceptable to me to use something that might change in future.
I've looked at the bytes generated for sample input decimal values - I see negative and precision flags (1Ch max in last int32?), but the min/max data values dumping compiler constants are stumping me a bit, and I'm generate above zero values or invalid values:


  • fractional value nearest 1 (.99...9): FFFFFF0F 6102253E 5ECE4F20 00001C00

  • fractional value nearest 0 (.00...1): 01000000 00000000 00000000 00001C00


Can you help me onto the correct path for generating a full fractional 0..1 range?

Edit with "final" code, thanks to everyone!

Here's the C# code I ended up with to create an unbiased random decimal of range [0..1] from a suitable random byte stream (like a TRNG device, www.Random.org, or CSPRNG algorithm). Generated values look good to the eyeball, boundary tests pass, and as long as I avoided embarassing typos and copy/paste bugs, this should be usable as-is.

Thanks for the help and interesting discussions!

private decimal RandomDecimalRange01()
{
// 96 bits of random data; we'll use 94 bits to directly map decimal's max precision 0..1 range
byte[] data = new byte[12];
decimal value = 0;

// loop until valid value is generated, discarding invalids values. Mostly controlled by top 2 bits: 11 is always invalid, 00 or 01, is always valid, 10 has valid and invalid ranges. Odds make loop generally find value in one or a few iterations.
while (true)
{
// Acquire random bytes from random source (like TRNG device or CSPRNG api)
if (!trng.GetBytes(data))
{
throw new Exception("Failed to aquire random bytes from source");
}
else
{
// Read 94 random bits (pull 96 bits, discard 2)
BinaryReader reader = new BinaryReader(new MemoryStream(data));
int low = reader.ReadInt32();
int mid = reader.ReadInt32();
int high = reader.ReadInt32() & 0x3FFFFFFF; // don't consume upper 2 random bits - out of range

// Discard invalid values and reloop (interpret special invalid value as 1)
if (high > 542101086)
{
continue;
}
else if (high == 542101086)
{
if (mid > 1042612833)
{
continue;
}
else if (mid == 1042612833)
{
if (low > 536870910)
{
// Special override to generate 1 value for inclusive [0..1] range - interpret the smallest invalid value as 1. Remove code for exclusive range [0..1)
if (low == 536870911)
{
value = 1m; // return 1.0
break;
}
continue;
}
}
}

// return random decimal created from parts - positive, maximum precision 28 (1C) scale
value = new decimal(low, mid, high, false, 28);
break;
}
}
return value;
}


Sample generated values running TrueRNGPro TRNG device bytes through algorithm

0.8086691474438979082567747041
0.4268035919422123276460607186
0.7758625805098585303332549015
0.0701321080502462116399370731
0.3127190777525873850928167447
0.6022236739048965325585049764
0.1244605652187291191393036867


Tests around interesting boundary values

// test databyte values for max & min ranges
new byte[] { 0xFF, 0xFF, 0xFF, 0x2F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: 1 too large for algorithm, will be discarded
new byte[] { 0xFF, 0xFF, 0xFF, 0x1F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: special 1 more than largest valid .99999..., interpreted as valid 1 value
new byte[] { 0xFF, 0xFF, 0xFF, 0x0F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: largest valid value .9999...
new byte[] { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: smallest valid value, should be 0
new byte[] { 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: 1 more than smallest valid value, should be .000...1

Answer

From https://msdn.microsoft.com/en-us/library/system.decimal.getbits(v=vs.110).aspx

The binary representation of a Decimal number consists of a 1-bit sign, a 96-bit integer number, and a scaling factor used to divide the integer number and specify what portion of it is a decimal fraction. The scaling factor is implicitly the number 10, raised to an exponent ranging from 0 to 28.

The return value is a four-element array of 32-bit signed integers.

The first, second, and third elements of the returned array contain the low, middle, and high 32 bits of the 96-bit integer number.

The fourth element of the returned array contains the scale factor and sign. It consists of the following parts:

Bits 0 to 15, the lower word, are unused and must be zero.

Bits 16 to 23 must contain an exponent between 0 and 28, which indicates the power of 10 to divide the integer number.

Bits 24 to 30 are unused and must be zero.

Bit 31 contains the sign: 0 mean positive, and 1 means negative.

You are looking for 0 <= 96bitinteger / 10exponent <= 1

Multiplying through, this is the same as 0 <= 96bitinteger <= 10^exponent

This would enumerate every possibile pair of 96bitinteger and exponent that would produce a decimal value between 0 and 1 (assuming the sign bit is set to 0).

for (int exponent=0; exponent<=28; exponent++) {
    BigInteger max = BigInteger.Pow(10, exponent);
    for (int i = 0; i <= max; i++) {
        var fmt = "96bitinteger: {0}, exponent: {1}";
        Console.WriteLine(String.Format(fmt, i, exponent));
    }
}

Using exponent=28

1028 in hex is 204fce5e 3e250261 10000000. So once you place the 32-bit numbers according to the docs, and then create fhe final 32-bits accounting for the fact for some reason when they say bit 0 they mean the highest order bit, it isn't too hard to construct the decimal number 1 from bytes.

int[] data = new int[] { 0x10000000, 0x3e250261, 0x204fce5e, 0x1C0000 };
var random_data = data.SelectMany(BitConverter.GetBytes).ToArray();
decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();
Console.WriteLine(value);

Consider the more general new int[] { a, b, c, 0x1C0000 }. The constraints on a b and c, for creating a decimal number 0 <= d <= 1 are

if c < 0x204fce5e:
    a can be anything
    b can be anything
elif c = 0x204fce5e:
    if b < 0x3e250261:
        a can be anything
    elif b = 0x3e250261
        constrain a <= 10000000
    b can not be greater than 0x3e250261
c can not be greater than 0x204fce5e.