Xojo data corruption (Database Framework)

If it is so toxic here, why are you discussing here on this platform if I am allowed to ask. I want to say - and that politely - we are not all negative here and we have not all “attitudes” here. We are discussing
also the critical stuffs and we are speaking about critical stuffs. That is on TOF impossible. Here we are more free. We can say what we want to. Politely.

It must, as it ends with the same effect.

And… here it is, again.

The current issue seems isolated to DatabaseRow.Column().CurrencyValue

If Xojo have more internationalization clashes at more places, they need to further investigate.

well crap

what type affinity do you get if you add the query I indicated above ?
ie make the code do


Var lDouble As Double = 5.2
Var db As New SQLiteDatabase
db.DatabaseFile = SpecialFolder.Desktop.Child("test-delete-it.sqlite")
db.CreateDatabase

Const createIt As String = "DROP TABLE IF EXISTS test; CREATE TABLE test (""kk6000"" real); "
db.ExecuteSQL(CreateIt)
db.ExecuteSQL("INSERT INTO test (""kk6000"") VALUES (?);", lDouble)
db.ExecuteSQL(CreateIt + “INSERT INTO test (”“kk6000"”) VALUES (5.2);")


Var rs As RowSet = db.SelectSQL("SELECT typeof(kk6000) as whatType FROM test")

For Each row As DatabaseRow In rs
  
 system.debuglog row.Column("whatType").stringvalue

next

Var rs As RowSet = db.SelectSQL("SELECT * FROM test")

For Each row As DatabaseRow In rs
  
  Var d1 As Double = row.Column("kk6000").DoubleValue
  Var c1 As Currency = row.Column("kk6000").CurrencyValue
  Var s1 As String = row.Column("kk6000").StringValue
  Var sc1 As String = c1.ToString
  Var sd1 As String = d1.ToString
  
  Break // c1 is DESTROYED !
  
Next row

I’d expect it to show REAL both times
If it does then I’d say this falls on xojo and how they convert REAL to CURRENCY

Yes, it’s a crap. As already stated, It occurs in more databases than SQLite.

If you DON’T USE iteration, avoiding DatabaseRow and ask for RowSet.Column().CurrencyValue directly, it works.

The problem arises if you use a For Each DatabaseRow

That definitely makes it Xojo’s issue since DatabaseRow IS their thing not sqlite’s (or any other DB vendors)

Yes. I said that since the beginning. And Robin classified it as “Framework Bug” in that case.

So, if you are using a For Each from a DB and collecting Currency values, in most European countries and Americas, you may be surprised with insane large errors, that if written back to the DBs, will destroy your data.

That’s one place we found it, Xojo should fix it ASAP, and investigate if they committed such kind of error in more places in the framework.

Again, it was tested and proven DAYS ago that the problem is NOT when the data is stored.

And even if the answer was there much before, I aswer you again those times, same others, the problem IS NOT the insert.

The problem being the insert was the first thing descarded.

well, insisting in something even when it was explained why that is not the problem then being mad for getting an explanation of why that is wrog, THAT is toxic.

this currency problem is not recent…
I made my own currency conversion routines from string in 2016
mainly because the tests I had at that time showed they were not handled correctly for france…
it is also not related to sqlite
this happens with postgres and mysql too, at different level though.
I did not claim too much at that time because I knew it wouldn’t be fixed by non french people to handle french currency problems ! I made my own methods, and they still works.

You are not alone…#forJava #forXojo #forC++

I skipped entire generations of Xojo due to too many bugs. I elected the 2016r3 as a “good enough” version, because in 2016r4 the graphics engine has changed to directx and things got unstable for “generations”, 64 bit codes too, math bugs related to currency, printing bugs, bugs, bugs, … So I decided to keep on the more stable 32 bit only compiler while possible and fixed at 2016r3. Every analyzed edition had its round of “still bugged, or got new bugs”, so I never used any 2017, 2018, 2019 releases for production, and waited anxiously for the 2020 series to stabilize, but it showed as another lost year, but now I needed Catalina+ support and finally 64 bit. So I finally started a migration at 2021 r1.1 as my “stable enough”, removed some patches for Currency bugs from 2016, mixed api1 with the minimum api2, modernized my HTML controls using the better Xojo<->Control javascript intercomm that I used invented hacks for the 2016 edition, etc and started a new cycle of “looking for another better version” to end the full api2 conversion. And that I thought it was 2021r3.1… Thank God we didn’t. Maybe we could introduce a modernized combination of Xojo “For Each” and bad math/conversion bugs at some point and destroyed our entire user base.

Xojo really needs data types that CAN better map to native DB types better
If I define a decimal(18,6) in postgresql there’s no suitable type for Xojo
Currency wont support 6 decimal places, and doubles lose precision beyond a certain point and cant represent this value with the same precision
Maybe by incorporating a set of data types like bob delaney’s infinite precision plugin
but it would expand database flexibility a ton and possibly help with reliability

There’s a request to support Quad (128 bit floating point) that soon will be present in the hardware (Currently I only know RISC-V CPUs with the Q extension with it) and will have unmatched speed then.
But the compilers already have it ready to be used in the software format. It provides 33 precise digits with a floating point wherever you want in those 33 digits
https://gcc.gnu.org/onlinedocs/libquadmath

Yeah Xojo wont lift it from gcc
Can you say “legal land mine” ?
They wont touch most GPL code for that reason
The mysql plugin is a good example

But they dont have to go that route
The could use code like bob delaney’s for arbitrary precision
Or LLVM’s built in 128 bit support like Rust already does

:slight_smile: http://feedback.xojo.com/case/59669

  1. http://feedback.xojo.com/case/38472

Not sure LLVM had decent 128 bit support then

Might have

Definitely does now

They implemented the 128 bit int math from LLVM into their std lib (primitive types) but they have 2 separated crates (libs you need to import) to handle f128 and d128, and the math lib used by f128 is from libquadmath, and d128 is based on the DecNumber library decNumber Library - 3.68 (that also has an IEEE 754 binary128 implementation in its internals if I’m not wrong). The Julia language also implemented LLVM int128, seems that all them are expecting changes when the CPUs come up with 128 bit fp instructions natively. Rust in their early days experienced the fp128 into the kernel (framework) and later decided to keep it as crates (language add ons) until CPUs start to support IEEE 754 binary128, and only then they will intend to deprecated things in crates and make some of those fp 128 types primitives. Julia also have some optional imports like those maintained by 3rd parties, and a primitive BigFloat, that uses internaly the GNU MPFR lib https://www.mpfr.org/

is the bug Windows only ?

No, as a framework bug it is affecting multiple platforms and databases.

This is a serious bug but if you want anything done about it then you not only need to show exactly where that bug is but you also need to raise a stink on TOF.

But so far it is buried in a thread with the non-offensive title “Currency in sqlite” that does not tell you anything about what is happening, instead of (like here) " Xojo data corruption (Database Framework)"

Is this a case of self-censorship???