JoelMc - 1 year ago 69

C# Question

I'm playing with a TRNG usb device, and successfully turning the random bytes into various usable data in C#.

I'd like to create a .NET decimal value between 0..1 (ex: 0.37327) directly from the bytes using binary reader (or other direct bytes -> decimal) method.

`// assume: byte[] random_data of appropriate length, patterned for a 0..1 range`

decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();

I was looking for the byte format of decimal, but it looks like it may not be a standard?

This is for hobby work, so it's acceptable to me to use something that might change in future.

I've looked at the bytes generated for sample input decimal values - I see negative and precision flags (1Ch max in last int32?), but the min/max data values dumping compiler constants are stumping me a bit, and I'm generate above zero values or invalid values:

fractional value nearest 1 (.99...9): FFFFFF0F 6102253E 5ECE4F20 00001C00

fractional value nearest 0 (.00...1): 01000000 00000000 00000000 00001C00

Can you help me onto the correct path for generating a full fractional 0..1 range?

Here's the C# code I ended up with to create an unbiased random decimal of range [0..1] from a suitable random byte stream (like a TRNG device, www.Random.org, or CSPRNG algorithm). Generated values look good to the eyeball, boundary tests pass, and as long as I avoided embarassing typos and copy/paste bugs, this should be usable as-is.

Thanks for the help and interesting discussions!

`private decimal RandomDecimalRange01()`

{

// 96 bits of random data; we'll use 94 bits to directly map decimal's max precision 0..1 range

byte[] data = new byte[12];

decimal value = 0;

// loop until valid value is generated, discarding invalids values. Mostly controlled by top 2 bits: 11 is always invalid, 00 or 01, is always valid, 10 has valid and invalid ranges. Odds make loop generally find value in one or a few iterations.

while (true)

{

// Acquire random bytes from random source (like TRNG device or CSPRNG api)

if (!trng.GetBytes(data))

{

throw new Exception("Failed to aquire random bytes from source");

}

else

{

// Read 94 random bits (pull 96 bits, discard 2)

BinaryReader reader = new BinaryReader(new MemoryStream(data));

int low = reader.ReadInt32();

int mid = reader.ReadInt32();

int high = reader.ReadInt32() & 0x3FFFFFFF; // don't consume upper 2 random bits - out of range

// Discard invalid values and reloop (interpret special invalid value as 1)

if (high > 542101086)

{

continue;

}

else if (high == 542101086)

{

if (mid > 1042612833)

{

continue;

}

else if (mid == 1042612833)

{

if (low > 536870910)

{

// Special override to generate 1 value for inclusive [0..1] range - interpret the smallest invalid value as 1. Remove code for exclusive range [0..1)

if (low == 536870911)

{

value = 1m; // return 1.0

break;

}

continue;

}

}

}

// return random decimal created from parts - positive, maximum precision 28 (1C) scale

value = new decimal(low, mid, high, false, 28);

break;

}

}

return value;

}

Sample generated values running TrueRNGPro TRNG device bytes through algorithm

`0.8086691474438979082567747041`

0.4268035919422123276460607186

0.7758625805098585303332549015

0.0701321080502462116399370731

0.3127190777525873850928167447

0.6022236739048965325585049764

0.1244605652187291191393036867

Tests around interesting boundary values

`// test databyte values for max & min ranges`

new byte[] { 0xFF, 0xFF, 0xFF, 0x2F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: 1 too large for algorithm, will be discarded

new byte[] { 0xFF, 0xFF, 0xFF, 0x1F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: special 1 more than largest valid .99999..., interpreted as valid 1 value

new byte[] { 0xFF, 0xFF, 0xFF, 0x0F, 0x61, 0x02, 0x25, 0x3E, 0x5E, 0xCE, 0x4F, 0x20 }; // boundary: largest valid value .9999...

new byte[] { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: smallest valid value, should be 0

new byte[] { 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; // boundary: 1 more than smallest valid value, should be .000...1

Answer Source

From https://msdn.microsoft.com/en-us/library/system.decimal.getbits(v=vs.110).aspx

The binary representation of a Decimal number consists of a 1-bit sign, a 96-bit integer number, and a scaling factor used to divide the integer number and specify what portion of it is a decimal fraction. The scaling factor is implicitly the number 10, raised to an exponent ranging from 0 to 28.

The return value is a four-element array of 32-bit signed integers.

The first, second, and third elements of the returned array contain the low, middle, and high 32 bits of the 96-bit integer number.

The fourth element of the returned array contains the scale factor and sign. It consists of the following parts:

Bits 0 to 15, the lower word, are unused and must be zero.

Bits 16 to 23 must contain an exponent between 0 and 28, which indicates the power of 10 to divide the integer number.

Bits 24 to 30 are unused and must be zero.

Bit 31 contains the sign: 0 mean positive, and 1 means negative.

You are looking for 0 <= 96bitinteger / 10^{exponent} <= 1

Multiplying through, this is the same as 0 <= 96bitinteger <= 10^exponent

This would enumerate every possibile pair of 96bitinteger and exponent that would produce a `decimal`

value between 0 and 1 (assuming the sign bit is set to 0).

```
for (int exponent=0; exponent<=28; exponent++) {
BigInteger max = BigInteger.Pow(10, exponent);
for (int i = 0; i <= max; i++) {
var fmt = "96bitinteger: {0}, exponent: {1}";
Console.WriteLine(String.Format(fmt, i, exponent));
}
}
```

10^{28} in hex is `204fce5e 3e250261 10000000`

. So once you place the 32-bit numbers according to the docs, and then create fhe final 32-bits accounting for the fact *for some reason when they say bit 0 they mean the highest order bit*, it isn't too hard to construct the decimal number 1 from bytes.

```
int[] data = new int[] { 0x10000000, 0x3e250261, 0x204fce5e, 0x1C0000 };
var random_data = data.SelectMany(BitConverter.GetBytes).ToArray();
decimal value = new BinaryReader(new MemoryStream(random_data)).ReadDecimal();
Console.WriteLine(value);
```

Consider the more general `new int[] { a, b, c, 0x1C0000 }`

. The constraints on a b and c, for creating a decimal number 0 <= d <= 1 are

```
if c < 0x204fce5e:
a can be anything
b can be anything
elif c = 0x204fce5e:
if b < 0x3e250261:
a can be anything
elif b = 0x3e250261
constrain a <= 10000000
b can not be greater than 0x3e250261
c can not be greater than 0x204fce5e.
```