hexagon logo

Decimal place rounding vs truncating

Our engineering manager asked me recently if we could change decimal place of certain characteristics on the report to either 3 places or 4 places, but not necessarily have the entire report be one or the other. Like some characteristics would be three places and some might be four places. So I told him that we can certainly do that. Then he asked if when we changed a characteristic from 4 places down to 3 places, does it round or truncate? I told him that I think that it rounds, but I'm not sure. Does anyone have any insight into this?
Parents
  • Might be better now, might have been specific to my machine at the time, but a while ago, I did this to a program, it was a longer one.

    I thought 3 places for ± .030" and 2 places for ±5° was more appropriate than 4 decimal places for everything. And it is.

    The program went from taking maybe 60 seconds to load onto the computer to taking 90 seconds, and editting could get painful.

    At that time, I decided it wasn't worth the effort and stopped doing what you suggest, and just leave it at one reporting resolution.
Reply
  • Might be better now, might have been specific to my machine at the time, but a while ago, I did this to a program, it was a longer one.

    I thought 3 places for ± .030" and 2 places for ±5° was more appropriate than 4 decimal places for everything. And it is.

    The program went from taking maybe 60 seconds to load onto the computer to taking 90 seconds, and editting could get painful.

    At that time, I decided it wasn't worth the effort and stopped doing what you suggest, and just leave it at one reporting resolution.
Children
No Data