Javascript rounding problem

Nov 11, 2011 at 11:56 AM
Edited Nov 11, 2011 at 11:56 AM

Hi guys,

I've got a perculiar problem with the compressor...

I have the following array in a js file (before compression)

var serverResolutions = [         
156543.03390625, 
78271.516953125,  
39135.7584765625,         
19567.87923828125, 
9783.939619140625,  
4891.9698095703125,         
2445.9849047851562, 
1222.9924523925781,  
611.4962261962891,         
305.74811309814453, 
152.87405654907226,  
76.43702827453613,         
38.218514137268066, 
19.109257068634033,  
9.554628534317017,         
4.777314267158508, 
2.388657133579254,  
1.194328566789627,         
0.5971642833948135,  
0.29858214169740677,  
0.14929107084870338,         
0.07464553542435169     
];
So far, all well and good but, I'm encountering an issue whereby once the compression is done, the array looks like:
var serverResolutions = [156543.03390625,
78271.516953125,39135.7584765625,
19567.8792382813,
9783.93961914062,
4891.96980957031,
2445.98490478516,
1222.99245239258,
611.496226196289,
305.748113098145,
152.874056549072,
76.4370282745361,
38.2185141372681,
19.109257068634,
9.55462853431702,
4.77731426715851,
2.38865713357925,
1.19432856678963,
0.597164283394814,
0.298582141697407,
0.149291070848703,
0.0746455354243517]
At first glance, it all looks okay but, notice the decimal precision on the array items - they've been rounded.
This is causing big problems for me as nothing is working.
What can I do to prevent this rounding from occurring?
I hope to hear back soon as it's quite a pressing matter.
Cheers,
Andy
Coordinator
Jan 27, 2012 at 7:47 AM

Hmm...the problem is actually in the EcmaScript library - the ScriptConvert.ToString method takes a double and does this:

                string ret = d.ToString ("g");
                // TODO: This is plain wrong, but as close as we can get
                // without converting DtoA to C#.

(note the comment!), which defaults to 15 digit precision, (http://msdn.microsoft.com/en-us/library/dwhawy9k.aspx#GFormatString) so your values of 16+ are being rounded.

I guess, since we already have a modified version of this library, we could fix this... purekrome, what are your thoughts?

Coordinator
Jan 27, 2012 at 10:46 AM

@freeranger - totally correct there mate! I believe we can tweak the EcmaScript library if that's code is from a file i've ported across.

So far, i've done a 1-to-1 conversion from java to c# -> including (on purpose/by design) all bugs/errors/bad formatting, etc. If the error exists in the java YUICompressor code, then it will exist here (again, on purpose).

But, I'm totally open to changing this stance -> starting with this issue.

Coordinator
Jan 27, 2012 at 10:55 AM

I think porting projects often come across this dilemma - are we true to the original, warts and all, or do we try to improve upon it?

The case for staying true is often for ease of switching between the Foo and Bar language versions of the libraries, so there is consistency, but I'm not sure how much of an issue this is in the real world TBH....

I don't think there's any real benefit in keeping bugs ;)  Also, the aim of the library is to compress css and javascript and have it remain functional - whether it results in exactly the same output as the Java version is probably not really an issue (if there was a decompress option then maybe...) so I say lets improve/de-bug where we can!

For this issue, we could easily change it to ToString("g99"), which would give a lot more precision...but I wouldn't want to go rewriting the library entirely to e.g. not use doubles at all.

If you're happy with this, then turn this into an issue and assign to me and I will sort it out.

Coordinator
Jan 27, 2012 at 11:04 AM

Well said :) 

Lets turn this into an issue can fix it up.

Coordinator
Jan 27, 2012 at 11:16 AM
Tnx ;)

Okay well if I do this one and turn the optional parameter into an overload then we should have enough changed to do a point release?

The other issue to sort is the dlls in the main distro - how is a distro built?



On 27 Jan 2012, at 12:04, purekrome <notifications@codeplex.com> wrote:

From: purekrome

Well said :)

Lets turn this into an issue can fix it up.

Coordinator
Jan 27, 2012 at 11:18 AM

Yeah. we have a point release, if we add a new overload IMO.

 

the distro is a manually painful thing :( i'm thinking of actually dropping it. I'll start a new thread about that.

Coordinator
Jan 27, 2012 at 11:30 AM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.
Coordinator
Jan 27, 2012 at 2:37 PM

Hmmm...... it's not as simple as it first appeared :/

Using Andy's input, I get these results for a Double.ToString()

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125     78271.516953124999*
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125       19567.8792382813*     19567.87923828125      19567.87923828125
     9783.939619140625       9783.93961914062*     9783.939619140625     9783.9396191406249*
    4891.9698095703125       4891.96980957031*     4891.969809570313*    4891.9698095703125
    2445.9849047851562       2445.98490478516*     2445.984904785156*    2445.9849047851562
    1222.9924523925781       1222.99245239258*     1222.992452392578*    1222.9924523925781
     611.4962261962891       611.496226196289*     611.4962261962891     611.49622619628906*
    305.74811309814453       305.748113098145*     305.7481130981445*    305.74811309814453
    152.87405654907226       152.874056549072*     152.8740565490723*    152.87405654907226
     76.43702827453613       76.4370282745361*     76.43702827453613     76.437028274536132*
    38.218514137268066       38.2185141372681*     38.21851413726807*    38.218514137268066
    19.109257068634033        19.109257068634*     19.10925706863403*    19.109257068634033
     9.554628534317017       9.55462853431702*     9.554628534317017     9.5546285343170165*
     4.777314267158508       4.77731426715851*     4.777314267158508     4.7773142671585083*
     2.388657133579254       2.38865713357925*     2.388657133579254     2.3886571335792541*
     1.194328566789627       1.19432856678963*     1.194328566789627     1.1943285667896271*
    0.5971642833948135      0.597164283394814*    0.5971642833948135    0.59716428339481353*
   0.29858214169740677      0.298582141697407*    0.2985821416974068*   0.29858214169740677
   0.14929107084870338      0.149291070848703*    0.1492910708487034*   0.14929107084870338
   0.07464553542435169     0.0746455354243517*   0.07464553542435169   0.074645535424351692*

The ones with a * differ from the original input.  

If I use Decimal.ToString instead, I get:

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125        78271.516953125
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125      19567.87923828125      19567.87923828125      19567.87923828125
     9783.939619140625      9783.939619140625      9783.939619140625      9783.939619140625
    4891.9698095703125     4891.9698095703125      4891.969809570313*    4891.9698095703125
    2445.9849047851562     2445.9849047851562      2445.984904785156*    2445.9849047851562
    1222.9924523925781     1222.9924523925781      1222.992452392578*    1222.9924523925781
     611.4962261962891      611.4962261962891      611.4962261962891      611.4962261962891
    305.74811309814453     305.74811309814453      305.7481130981445*    305.74811309814453
    152.87405654907226     152.87405654907226      152.8740565490723*    152.87405654907226
     76.43702827453613      76.43702827453613      76.43702827453613      76.43702827453613
    38.218514137268066     38.218514137268066      38.21851413726807*    38.218514137268066
    19.109257068634033     19.109257068634033      19.10925706863403*    19.109257068634033
     9.554628534317017      9.554628534317017      9.554628534317017      9.554628534317017
     4.777314267158508      4.777314267158508      4.777314267158508      4.777314267158508
     2.388657133579254      2.388657133579254      2.388657133579254      2.388657133579254
     1.194328566789627      1.194328566789627      1.194328566789627      1.194328566789627
    0.5971642833948135     0.5971642833948135     0.5971642833948135     0.5971642833948135
   0.29858214169740677    0.29858214169740677     0.2985821416974068*   0.29858214169740677
   0.14929107084870338    0.14929107084870338     0.1492910708487034*   0.14929107084870338
   0.07464553542435169    0.07464553542435169    0.07464553542435169    0.07464553542435169

Which is great, but the problem now is that this is all deep in the bowels of the EcmaScript library, which looks like it would need major surgery to fix it, to replace all the Double fruitness with Decimal fruitiness...and who knows what knock-on impact that might have.

Sticking with double and going to G16 offers us the better results than we have currently, but obviously it's still not perfect.  I can go up to G99, but that doesn't solve the problem and results in a different set of inaccurate values.

Annoyingly, I can see the correct value i the debugger if I break into the ScriptConvert method, but I can't seem to ToString that to what I can see for all these values.... Anyone got any brainwaves?

Jan 27, 2012 at 2:51 PM

Hi Guys,

Sorry for this bug - it's seeming like a bit of a pain to fix.

Whilst not ideal, would it be more "versatile" to evaluate the length of the decimal part of each string it is trying to compress and set the format pattern based on that?

Andy


From: freeranger [notifications@codeplex.com]
Sent: 27 January 2012 15:37
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Hmmm...... it's not as simple as it first appeared :/

Using Andy's input, I get these results for a Double.ToString()

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125     78271.516953124999*
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125       19567.8792382813*     19567.87923828125      19567.87923828125
     9783.939619140625       9783.93961914062*     9783.939619140625     9783.9396191406249*
    4891.9698095703125       4891.96980957031*     4891.969809570313*    4891.9698095703125
    2445.9849047851562       2445.98490478516*     2445.984904785156*    2445.9849047851562
    1222.9924523925781       1222.99245239258*     1222.992452392578*    1222.9924523925781
     611.4962261962891       611.496226196289*     611.4962261962891     611.49622619628906*
    305.74811309814453       305.748113098145*     305.7481130981445*    305.74811309814453
    152.87405654907226       152.874056549072*     152.8740565490723*    152.87405654907226
     76.43702827453613       76.4370282745361*     76.43702827453613     76.437028274536132*
    38.218514137268066       38.2185141372681*     38.21851413726807*    38.218514137268066
    19.109257068634033        19.109257068634*     19.10925706863403*    19.109257068634033
     9.554628534317017       9.55462853431702*     9.554628534317017     9.5546285343170165*
     4.777314267158508       4.77731426715851*     4.777314267158508     4.7773142671585083*
     2.388657133579254       2.38865713357925*     2.388657133579254     2.3886571335792541*
     1.194328566789627       1.19432856678963*     1.194328566789627     1.1943285667896271*
    0.5971642833948135      0.597164283394814*    0.5971642833948135    0.59716428339481353*
   0.29858214169740677      0.298582141697407*    0.2985821416974068*   0.29858214169740677
   0.14929107084870338      0.149291070848703*    0.1492910708487034*   0.14929107084870338
   0.07464553542435169     0.0746455354243517*   0.07464553542435169   0.074645535424351692*

The ones with a * differ from the original input.

If I use Decimal.ToString instead, I get:

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125        78271.516953125
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125      19567.87923828125      19567.87923828125      19567.87923828125
     9783.939619140625      9783.939619140625      9783.939619140625      9783.939619140625
    4891.9698095703125     4891.9698095703125      4891.969809570313*    4891.9698095703125
    2445.9849047851562     2445.9849047851562      2445.984904785156*    2445.9849047851562
    1222.9924523925781     1222.9924523925781      1222.992452392578*    1222.9924523925781
     611.4962261962891      611.4962261962891      611.4962261962891      611.4962261962891
    305.74811309814453     305.74811309814453      305.7481130981445*    305.74811309814453
    152.87405654907226     152.87405654907226      152.8740565490723*    152.87405654907226
     76.43702827453613      76.43702827453613      76.43702827453613      76.43702827453613
    38.218514137268066     38.218514137268066      38.21851413726807*    38.218514137268066
    19.109257068634033     19.109257068634033      19.10925706863403*    19.109257068634033
     9.554628534317017      9.554628534317017      9.554628534317017      9.554628534317017
     4.777314267158508      4.777314267158508      4.777314267158508      4.777314267158508
     2.388657133579254      2.388657133579254      2.388657133579254      2.388657133579254
     1.194328566789627      1.194328566789627      1.194328566789627      1.194328566789627
    0.5971642833948135     0.5971642833948135     0.5971642833948135     0.5971642833948135
   0.29858214169740677    0.29858214169740677     0.2985821416974068*   0.29858214169740677
   0.14929107084870338    0.14929107084870338     0.1492910708487034*   0.14929107084870338
   0.07464553542435169    0.07464553542435169    0.07464553542435169    0.07464553542435169

Which is great, but the problem now is that this is all deep in the bowels of the EcmaScript library, which looks like it would need major surgery to fix it, to replace all the Double fruitness with Decimal fruitiness...and who knows what knock-on impact that might have.

Sticking with double and going to G16 offers us the better results than we have currently, but obviously it's still not perfect. I can go up to G99, but that doesn't solve the problem and results in a different set of inaccurate values.

Annoyingly, I can see the correct value i the debugger if I break into the ScriptConvert method, but I can't seem to ToString that to what I can see for all these values.... Anyone got any brainwaves?

Coordinator
Jan 27, 2012 at 4:38 PM
Hi Andy,

It might be worth you downloading the source code and taking a look at that to get an idea of what is going on.
The EcmaScript library is tokenising the input, it is not something YUICompressor does itself.

This library would need some serious surgery in order to deal with this edge case.
I don't know the reasoning behind the choice of double for that library - it may be simply for performance, or it may be some other consideration.

One way around this problem for you would be to compress everything except the array of values.
The functions manipulating them would still be fine, but the actual values would be set up in a separate script.

Would that work for you?

On 27 Jan 2012, at 15:52, andyofengland <notifications@codeplex.com> wrote:

From: andyofengland

Hi Guys,

Sorry for this bug - it's seeming like a bit of a pain to fix.

Whilst not ideal, would it be more "versatile" to evaluate the length of the decimal part of each string it is trying to compress and set the format pattern based on that?

Andy


From: freeranger [notifications@codeplex.com]
Sent: 27 January 2012 15:37
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Hmmm...... it's not as simple as it first appeared :/

Using Andy's input, I get these results for a Double.ToString()

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125     78271.516953124999*
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125       19567.8792382813*     19567.87923828125      19567.87923828125
     9783.939619140625       9783.93961914062*     9783.939619140625     9783.9396191406249*
    4891.9698095703125       4891.96980957031*     4891.969809570313*    4891.9698095703125
    2445.9849047851562       2445.98490478516*     2445.984904785156*    2445.9849047851562
    1222.9924523925781       1222.99245239258*     1222.992452392578*    1222.9924523925781
     611.4962261962891       611.496226196289*     611.4962261962891     611.49622619628906*
    305.74811309814453       305.748113098145*     305.7481130981445*    305.74811309814453
    152.87405654907226       152.874056549072*     152.8740565490723*    152.87405654907226
     76.43702827453613       76.4370282745361*     76.43702827453613     76.437028274536132*
    38.218514137268066       38.2185141372681*     38.21851413726807*    38.218514137268066
    19.109257068634033        19.109257068634*     19.10925706863403*    19.109257068634033
     9.554628534317017       9.55462853431702*     9.554628534317017     9.5546285343170165*
     4.777314267158508       4.77731426715851*     4.777314267158508     4.7773142671585083*
     2.388657133579254       2.38865713357925*     2.388657133579254     2.3886571335792541*
     1.194328566789627       1.19432856678963*     1.194328566789627     1.1943285667896271*
    0.5971642833948135      0.597164283394814*    0.5971642833948135    0.59716428339481353*
   0.29858214169740677      0.298582141697407*    0.2985821416974068*   0.29858214169740677
   0.14929107084870338      0.149291070848703*    0.1492910708487034*   0.14929107084870338
   0.07464553542435169     0.0746455354243517*   0.07464553542435169   0.074645535424351692*

The ones with a * differ from the original input.

If I use Decimal.ToString instead, I get:

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125        78271.516953125
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125      19567.87923828125      19567.87923828125      19567.87923828125
     9783.939619140625      9783.939619140625      9783.939619140625      9783.939619140625
    4891.9698095703125     4891.9698095703125      4891.969809570313*    4891.9698095703125
    2445.9849047851562     2445.9849047851562      2445.984904785156*    2445.9849047851562
    1222.9924523925781     1222.9924523925781      1222.992452392578*    1222.9924523925781
     611.4962261962891      611.4962261962891      611.4962261962891      611.4962261962891
    305.74811309814453     305.74811309814453      305.7481130981445*    305.74811309814453
    152.87405654907226     152.87405654907226      152.8740565490723*    152.87405654907226
     76.43702827453613      76.43702827453613      76.43702827453613      76.43702827453613
    38.218514137268066     38.218514137268066      38.21851413726807*    38.218514137268066
    19.109257068634033     19.109257068634033      19.10925706863403*    19.109257068634033
     9.554628534317017      9.554628534317017      9.554628534317017      9.554628534317017
     4.777314267158508      4.777314267158508      4.777314267158508      4.777314267158508
     2.388657133579254      2.388657133579254      2.388657133579254      2.388657133579254
     1.194328566789627      1.194328566789627      1.194328566789627      1.194328566789627
    0.5971642833948135     0.5971642833948135     0.5971642833948135     0.5971642833948135
   0.29858214169740677    0.29858214169740677     0.2985821416974068*   0.29858214169740677
   0.14929107084870338    0.14929107084870338     0.1492910708487034*   0.14929107084870338
   0.07464553542435169    0.07464553542435169    0.07464553542435169    0.07464553542435169

Which is great, but the problem now is that this is all deep in the bowels of the EcmaScript library, which looks like it would need major surgery to fix it, to replace all the Double fruitness with Decimal fruitiness...and who knows what knock-on impact that might have.

Sticking with double and going to G16 offers us the better results than we have currently, but obviously it's still not perfect. I can go up to G99, but that doesn't solve the problem and results in a different set of inaccurate values.

Annoyingly, I can see the correct value i the debugger if I break into the ScriptConvert method, but I can't seem to ToString that to what I can see for all these values.... Anyone got any brainwaves?

Jan 28, 2012 at 9:29 PM

Hi,

Thanks for the guidance J

I’ve managed to do a workaround by comparing the numeric versions against a string version of the same values – if they match, I use the numeric version, if not, I use the string version but parseFloat it. Not ideal by any means, but it does mean I can keep the file compressed and still keep the geographical accuracy.

In terms of compressing everything except the array, this won’t really work as we’re programmatically compressing all script and style files – in a similar vein to the way MVC4.5 seems to. Otherwise, yea, it would have been a great suggestion.

Are you going to keep this bug open or, is it such a fringe case (except for any usages with Latitude/longitude values) that it’s not worth spending too much time on?

Thanks again,

Andy

From: freeranger [email removed]
Sent: 27 January 2012 17:39
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Hi Andy,

It might be worth you downloading the source code and taking a look at that to get an idea of what is going on.

The EcmaScript library is tokenising the input, it is not something YUICompressor does itself.

This library would need some serious surgery in order to deal with this edge case.

I don't know the reasoning behind the choice of double for that library - it may be simply for performance, or it may be some other consideration.

One way around this problem for you would be to compress everything except the array of values.

The functions manipulating them would still be fine, but the actual values would be set up in a separate script.

Would that work for you?


On 27 Jan 2012, at 15:52, andyofengland <notifications@codeplex.com> wrote:

From: andyofengland

Hi Guys,

Sorry for this bug - it's seeming like a bit of a pain to fix.

Whilst not ideal, would it be more "versatile" to evaluate the length of the decimal part of each string it is trying to compress and set the format pattern based on that?

Andy


From: freeranger [notifications@codeplex.com]
Sent: 27 January 2012 15:37
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Hmmm...... it's not as simple as it first appeared :/

Using Andy's input, I get these results for a Double.ToString()

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125     78271.516953124999*
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125       19567.8792382813*     19567.87923828125      19567.87923828125
     9783.939619140625       9783.93961914062*     9783.939619140625     9783.9396191406249*
    4891.9698095703125       4891.96980957031*     4891.969809570313*    4891.9698095703125
    2445.9849047851562       2445.98490478516*     2445.984904785156*    2445.9849047851562
    1222.9924523925781       1222.99245239258*     1222.992452392578*    1222.9924523925781
     611.4962261962891       611.496226196289*     611.4962261962891     611.49622619628906*
    305.74811309814453       305.748113098145*     305.7481130981445*    305.74811309814453
    152.87405654907226       152.874056549072*     152.8740565490723*    152.87405654907226
     76.43702827453613       76.4370282745361*     76.43702827453613     76.437028274536132*
    38.218514137268066       38.2185141372681*     38.21851413726807*    38.218514137268066
    19.109257068634033        19.109257068634*     19.10925706863403*    19.109257068634033
     9.554628534317017       9.55462853431702*     9.554628534317017     9.5546285343170165*
     4.777314267158508       4.77731426715851*     4.777314267158508     4.7773142671585083*
     2.388657133579254       2.38865713357925*     2.388657133579254     2.3886571335792541*
     1.194328566789627       1.19432856678963*     1.194328566789627     1.1943285667896271*
    0.5971642833948135      0.597164283394814*    0.5971642833948135    0.59716428339481353*
   0.29858214169740677      0.298582141697407*    0.2985821416974068*   0.29858214169740677
   0.14929107084870338      0.149291070848703*    0.1492910708487034*   0.14929107084870338
   0.07464553542435169     0.0746455354243517*   0.07464553542435169   0.074645535424351692*

The ones with a * differ from the original input.

If I use Decimal.ToString instead, I get:

              Original          ToString("G")        ToString("G16")        ToString("G17")
   ===================    ===================    ===================    ===================
       156543.03390625        156543.03390625        156543.03390625        156543.03390625
       78271.516953125        78271.516953125        78271.516953125        78271.516953125
      39135.7584765625       39135.7584765625       39135.7584765625       39135.7584765625
     19567.87923828125      19567.87923828125      19567.87923828125      19567.87923828125
     9783.939619140625      9783.939619140625      9783.939619140625      9783.939619140625
    4891.9698095703125     4891.9698095703125      4891.969809570313*    4891.9698095703125
    2445.9849047851562     2445.9849047851562      2445.984904785156*    2445.9849047851562
    1222.9924523925781     1222.9924523925781      1222.992452392578*    1222.9924523925781
     611.4962261962891      611.4962261962891      611.4962261962891      611.4962261962891
    305.74811309814453     305.74811309814453      305.7481130981445*    305.74811309814453
    152.87405654907226     152.87405654907226      152.8740565490723*    152.87405654907226
     76.43702827453613      76.43702827453613      76.43702827453613      76.43702827453613
    38.218514137268066     38.218514137268066      38.21851413726807*    38.218514137268066
    19.109257068634033     19.109257068634033      19.10925706863403*    19.109257068634033
     9.554628534317017      9.554628534317017      9.554628534317017      9.554628534317017
     4.777314267158508      4.777314267158508      4.777314267158508      4.777314267158508
     2.388657133579254      2.388657133579254      2.388657133579254      2.388657133579254
     1.194328566789627      1.194328566789627      1.194328566789627      1.194328566789627
    0.5971642833948135     0.5971642833948135     0.5971642833948135     0.5971642833948135
   0.29858214169740677    0.29858214169740677     0.2985821416974068*   0.29858214169740677
   0.14929107084870338    0.14929107084870338     0.1492910708487034*   0.14929107084870338
   0.07464553542435169    0.07464553542435169    0.07464553542435169    0.07464553542435169

Which is great, but the problem now is that this is all deep in the bowels of the EcmaScript library, which looks like it would need major surgery to fix it, to replace all the Double fruitness with Decimal fruitiness...and who knows what knock-on impact that might have.

Sticking with double and going to G16 offers us the better results than we have currently, but obviously it's still not perfect. I can go up to G99, but that doesn't solve the problem and results in a different set of inaccurate values.

Annoyingly, I can see the correct value i the debugger if I break into the ScriptConvert method, but I can't seem to ToString that to what I can see for all these values.... Anyone got any brainwaves?

Coordinator
Jan 29, 2012 at 11:14 AM
andyofengland wrote:

 In terms of compressing everything except the array, this won’t really work as we’re programmatically compressing all script and style files – in a similar vein to the way MVC4.5 seems to. Otherwise, yea, it would have been a great suggestion.

I meant that the js with these long decimals would be included in the html page, with the scripts accessing the values still being compressed.

andyofengland wrote:

Are you going to keep this bug open or, is it such a fringe case (except for any usages with Latitude/longitude values) that it’s not worth spending too much time on? 

Well it is pretty much an edge case, and the problem lies in a library we are using, not one of our own making.....but I don't like to see issues unresolved, so will look into it a bit more, but no promises....

Coordinator
Jan 30, 2012 at 3:55 AM

Just out of curiosity Andy, what are those numbers? (with such high precision). 

Jan 30, 2012 at 10:37 AM

Hi,

They are fairly precise latitude and longitude zoom levels that are needed for Open Layers/Bing mapping.

Cheers,

Andy


From: purekrome [notifications@codeplex.com]
Sent: 30 January 2012 04:55
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: purekrome

Just out of curiosity Andy, what are those numbers? (with such high precision).

Coordinator
Feb 2, 2012 at 2:54 PM

BTW Andy, the EcmaScript library we use is a "warts and all" port of the Rhino java library: http://www.mozilla.org/rhino/, so if the problem exists there, it should be reported to the rhino guys and we will implement the same fix.

Would you be able to test with the rhino library and see if you get the same rounding issue?

Feb 2, 2012 at 4:03 PM

Hi,

I'm not familiar with that library and, am going away on holiday for a couple of weeks so I won't have any time to try.

I can see about giving it a go on my return though.

Cheers

Andy


From: freeranger [notifications@codeplex.com]
Sent: 02 February 2012 15:54
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

BTW Andy, the EcmaScript library we use is a "warts and all" port of the Rhino java library: http://www.mozilla.org/rhino/, so if the problem exists there, it should be reported to the rhino guys and we will implement the same fix.

Would you be able to test with the rhino library and see if you get the same rounding issue?

Coordinator
Feb 3, 2012 at 7:41 PM

Rhino is used by the yahoo YUI Compressor....so I just tried your input with that...and it worked :}

We are a couple of versions behind the latest yahoo code, so lets bring it up to date and see what happens....

Feb 4, 2012 at 8:17 AM

Hi,

That’ good news (as it must mean there’s less work for the fix) J

Thanks for taking a look.

Andy

From: freeranger [email removed]
Sent: 03 February 2012 20:42
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Rhino is used by the yahoo YUI Compressor....so I just tried your input with that...and it worked :}

We are a couple of versions behind the latest yahoo code, so lets bring it up to date and see what happens....

Coordinator
Feb 4, 2012 at 8:40 AM
Sady that's not the case :/
I also tried the version that we are on...and it worked there too - so its not a defect that has been fixed...so I suspect the java version of a double is different to the c# one and therefore some major surgery may be required :(

Further investigation needed...

On 4 Feb 2012, at 09:18, andyofengland <notifications@codeplex.com> wrote:

From: andyofengland

Hi,

That’ good news (as it must mean there’s less work for the fix) J

Thanks for taking a look.

Andy

From: freeranger [email removed]
Sent: 03 February 2012 20:42
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Rhino is used by the yahoo YUI Compressor....so I just tried your input with that...and it worked :}

We are a couple of versions behind the latest yahoo code, so lets bring it up to date and see what happens....

Feb 4, 2012 at 12:28 PM

Oh stink L

From: freeranger [email removed]
Sent: 04 February 2012 09:41
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Sady that's not the case :/

I also tried the version that we are on...and it worked there too - so its not a defect that has been fixed...so I suspect the java version of a double is different to the c# one and therefore some major surgery may be required :(

Further investigation needed...


On 4 Feb 2012, at 09:18, andyofengland <notifications@codeplex.com> wrote:

From: andyofengland

Hi,

That’ good news (as it must mean there’s less work for the fix) J

Thanks for taking a look.

Andy

From: freeranger [email removed]
Sent: 03 February 2012 20:42
To: Andy Nightingale
Subject: Re: Javascript rounding problem [YUICompressor:279118]

From: freeranger

Rhino is used by the yahoo YUI Compressor....so I just tried your input with that...and it worked :}

We are a couple of versions behind the latest yahoo code, so lets bring it up to date and see what happens....