Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
iminore
Posting Yak Master
141 Posts |
 Posted - 2005-08-17 : 08:42:30
|
| I'm reading a recordset from a SQL Server procedure. One of the columns is type float but when I get it back into my ASP code I'm losing precision of the last few digits.An example is 1125899906863598.0 which becomes 1.1258999068636E+15Using FormatNumber doesn't help - the last few digits have somehow been lost.Using CDbl gives 1125899906863600Any ideas? |
|
|
Kristen
Test
22859 Posts |
Posted - 2005-08-17 : 09:02:01
|
| Perhaps CAST it as a DECIMAL so SQL ships it in that format, rather than a Float?Kristen |
 |
|
|
iminore
Posting Yak Master
141 Posts |
Posted - 2005-08-17 : 09:17:21
|
| Just tried that - same result.I'm setting the variable in ASP as double but it seems determined to drop the last two digits. |
|
|
|
madhivanan
Premature Yak Congratulator
22864 Posts |
Posted - 2005-08-17 : 09:18:57
|
| What happens if you use decimal or float type variable in ASP?MadhivananFailing to plan is Planning to fail |
|
|
|
iminore
Posting Yak Master
141 Posts |
Posted - 2005-08-17 : 09:27:39
|
| No such types in ASP vbscript. |
|
|
|
Kristen
Test
22859 Posts |
Posted - 2005-08-17 : 09:38:36
|
| CAST it as a varchar(50) and then convert to number in ASP?If that works then its the worst Hack I've ever come up with!Kristen |
|
|
|
iminore
Posting Yak Master
141 Posts |
Posted - 2005-08-17 : 09:41:35
|
| Tried that, but thanks for the effort. |
|
|
|
Kristen
Test
22859 Posts |
|
|
iminore
Posting Yak Master
141 Posts |
Posted - 2005-08-17 : 11:47:39
|
| Think I may have cracked it.Defined column as bigint (instead of float) and casting now preserves lower digits. |
|
|
|
|
|
|
Floating number