Article	60001057
Type	Wish
Product	Engine
Version	6016
Date Added	1/5/2010
Fixed	Yes [1/5/2010]
Submitted by	Adam White
Keywords

Subject

Precision of doubles saved in VDML/VDCL

Summary

Precision of doubles (like gPoint's X, Y, Z coordinates) saved in VDML/VDCL can be up to 10 decimal places, using DoublePrecision property. Can you extend this ?

Solution

Extended in 6017.

DoublePrecision property represents the maximum number of decimal digits used when saving double values in vdml format.Valid range 0 - 17
example:
 
The following code set the decimal precision to the maximum value:

   int prec = 17;

   doc.FileProperties.DoublePrecision = prec;//change the number of decimal digits when saved in vdml/vdcl
   Globals.DefaultPointEquality = 1.0d / Math.Pow(10, prec); // change the default point equality that is used to compare two gPoints
   Globals.DefaultLinearEquality = 1.0d / Math.Pow(10, prec);// change the default distance equality that is used to compare two double values.

   doc.lunits.Precision = (short)prec;//change the display of the decimal digits shown in VectorDraw property grid and command line prompts
   doc.lunits.ZerosSuppression = LUnits.LZeroSuppression.DecimalTrailing;// Suppresses trailing zeros in decimal dimensions (for example, 12.5000 becomes 12.5)
 

Note:
The .NET FrameWork is using maximum 17 digits (includes integer and decimal part) for double numbers.
For example, if we have a double like 1234567890.123456789 and we set the doc.FileProperties.DoublePrecision = 17 then the result will be 1234567890.1234568 (10 places for the "integer" part and 7 places for the "fractional" part).
Also : a double like 1234567890123456789 (it only have a integral part) will become 123456789012345680 and  a double like 0.1234567890123456789 will become 0.12345678901234568