The tostring( ) lua function introduces "roundoff" errors!

Moon said he has code that rounds numbers displayed to 2 digits. The results of your display were a number, a string, and a number. The two numbers were rounded to 2 digits, the string was not, because it is not a number.

Code:

---

perl -E '@n=(3,0.03000000026077,3.2); say sprintf "%.2g", $_ foreach @n;'
3
0.03
3.2

---

Quote:

---

Originally Posted by Andraax

Code:

---

perl ......

---

---

Thanks for the perl, Andraax. :confused:

Quote:

---

Originally Posted by Minty23185Fresh

Thanks for the perl, Andraax. :confused:

---

Perl uses C / C++ printf formats, as does LUA. Just demonstrating that some formats don't pad with 0s or spaces, as you said in your post.

Quote:

---

Originally Posted by Minty23185Fresh

Thanks, Moon Wizard. I'd like to know the secret of Debug.console(). As far as I recall, back from the C days most format(%w.pf, ..) results in pre or post padding with either 0's or spaces. Something that isn't occurring with Debug.console() output.

---

Working on my own routine to do the rounding I need to my desired number of places...

One thing that I have noticed, every time I Debug.console() a number, if there are a large number of decimal places, Debug performs rounding before display. It appears to top out at 4 decimal places. It is as if there is an environment setting for it (set by Smiteworks? or internal to lua itself?).

Examples:
n = 15.1999999999999999; Debug.console(n); --> 15.2
n = 26.2300000000009879; Debug.console(n); --> 26.23
n = 37.7349999990005329; Debug.console(n); --> 37.735
n = 48.1111000000000812; Debug.console(n); --> 48.1111
n = 51.5432799999999210; Debug.console(n); --> 51.5433 not 51.54328 !!

Quote:

---

Originally Posted by Andraax

Perl uses C / C++ printf formats, as does LUA. Just demonstrating that some formats don't pad with 0s or spaces, as you said in your post.

---

Ah. Thank you. Is that what the "g" is in the format "%.2g", it prevents trailing 0's ?

Yes.

Try:

Code:

---

Debug.console(string.format("%.5g",n))

---

Thanks Andraax, I will.
I've just about finished the function I'm writing, but that little pearl might be exactly what I need.

@Andraax:
Not quite it. It prints 5 digits, including the integer part. So 123.456781. ---> 123.45. I want 123.45678. Five fractional digits regardless of the magnitude of the integer part.

I tried adding a width, as in %10.5g, but that pre-pads with spaces and still prints a total of 5 digits.
I remember this with C/C++. How a language as powerful as these could be so aggravating in this respect!

I don't need to waste your time as well as my own with such a trivial problem. Do you have an Internet resource for regular expressions that you like? The Lua manual is lacking. Other sites I have visited in the last hour have a nibble here, a byte there, and maybe a couple words. But most are incomplete.

Thanks in advance.

There are no regular expressions built into lua for FG. If you are talking about Lua search expressions, I have looked far and wide for a lua search expression tester but it just doesn’t exist.
As for formatting, the lua 5.1 manual says the formatting instructions are identical to C language print formatting. Away from my computer but you should be able to find plenty on C print formatting.
The string manager in FG has a trim function you can use to strip off those padding blanks (StringManager.trim)