constant folding percision issue

Bugs will be moved here once resolved.

Post » Tue Jan 14, 2014 8:19 pm

Link to .capx file (required!):
https://dl.dropboxusercontent.com/u/5426011/bug/bug_constant_folding.capx

Steps to reproduce:
1. Run capx.

Observed result:
Global variables a and b have different values.
"a" is set to 64999966 / 1000000 and later reads as 65.
"b" is set to 64999966*one / 1000000 and reads as 64.999966.
"one" is a global set to 1 to bypass "constant folding".

Expected result:
I would expect both expressions to evaluate to 64.999966 and not just b.

Browsers affected:
Chrome: yes
Firefox: yes
Internet Explorer: untested

Operating system & service pack:
Windows vista sp2
Construct 2 version:
r155
B
92
S
32
G
109
Posts: 5,291
Reputation: 70,993

Post » Wed Jan 15, 2014 7:39 am

JFYI, I was testing this on IE11 yesterday with the same results
B
9
S
3
G
1
Posts: 112
Reputation: 1,827

Post » Wed Jan 15, 2014 4:23 pm

Thanks, should be fixed in the next build.
Scirra Founder
B
397
S
236
G
88
Posts: 24,423
Reputation: 194,560

Post » Wed Jan 15, 2014 4:51 pm

@Ashley - would you mind sharing what was wrong and the fix? It seems like it was a rounding error and could be useful to keep in mind as a javascript programmerFimbul2014-01-15 16:52:11
B
36
S
8
G
8
Posts: 532
Reputation: 6,903

Post » Wed Jan 15, 2014 6:37 pm

@Fimbul - it wasn't actually the constant folder or JS. It was the stringstream (in C++) that writes the project data to javascript - by default the stream double-to-text converter had a precision of 6 decimal places, which would round some values. Upping it to 16 decimal places fixed the problem.Ashley2014-01-15 18:37:34
Scirra Founder
B
397
S
236
G
88
Posts: 24,423
Reputation: 194,560


Return to Closed bugs

Who is online

Users browsing this forum: No registered users and 5 guests