I have created a thread, and in the thread’s Run event, it calls a method that opens a large text file and reads the data and loads it into a JSONItem.
When I call Thread.Run, the whole program stops until the thread finishes. It’s like it’s not running on a thread at all. Changing it’s priority even to 1 makes no difference.
I put a breakpoint after calling Thread.Run, and it doesn’t hit the breakpoint until the file is completely loaded.
Is there a limitation on what can be done in the Thread (no UI stuff is in there)? I read that any methods it calls will run within that thread too, so that shouldn’t be blocking the main thread, should it?
Xojo uses co-operative threading, which avoids complications from pre-emptive threading, but at the end of the day means that the application can only do one thing at one time.
You may want to consider braking it up, so it does it in smaller increments (if that’s possible), which will mean that the thread will operate in the way you expect.
In simple words, you only have ONE single “real” Thread. Any xojo thread, just means that the single real thread STOPS running the main thread, do a little of another thread, little of another one, go back tho the main and so on.
Some actions cant be “paused” to go to another thread, looks like some file related ones have this problem.
Solution, use a tool with real multithreading.
Xojo solution, use “helper apps”, Examples, Multiprocessing. You need a “pro” license.
I know about how Xojo threads are just splitting time between them and the main thread, and in this instance, that would work fine, except that it’s not splitting at all. It never splits to the main thread until the file is loaded and parsed.
I guess as Christian says, file operations maybe can’t be split? Would be helpful if there were a list of operations that can (or can’t) be split.
I do miss real threading from .NET…
I haven’t checked into it, but I thought helper apps were just console apps running the other thread. You need a different license to spawn a console app from a main thread?
Some additional observations I made writing my last app which did a similar task:
In comparison, the reading of the file itself did not take that long. Reading a file in one piece or divided up into smaller pieces with Binarystream into a Memoryblock was quite fast. The processing of the data took most of the time, especially when the data files were large.
I got the worst performance when I read in the data in small chunks, processed the data, read in the next small chunk, processed the data and so on.
I got the fastest performance when I read the file in one piece into a Memoryblock then processed the data an loop.
right but if you want to read a file in a thread and NOT have that thread possibly block reading in “chunks” may be the only alternative to get the reading to yield time to other threads
chunk size would definitely be a factor in overall performance
experimentation would be needed to determine an optimal chunk size but I’d guess 512K to 1Mb is in the ballpark based on current drives etc
Of course. What I often see is that people read in a file with 150000000 records/rows line be line. Each line read is immediately processed, then the next line es read in etc. This process takes several minutes and eventually (depending how your code looks) blocks the UI, even when the method gets called from a thread. Reading the same file into memory in one chunk only takes 2 seconds (so blocking the UI is very short). What I try to say: When to UI is blocked for a long time, the blocking is probably caused mostly by the processing of the data and maybe additionally because the data is read in too many small chunks (row by row).
there is definitely a balance in how things are handle that can make the UI block - or not
sometimes you control it since its all in Xojo code you write so you can inject thread yields or loop boundaries where threads can switch
and sometimes not like a call to a plugin that doesnt yield or a framework function that also doesnt yield and so no matter what you do you get a blockage
when framework functions that do not yield in threads are encountered a feedback report is in order
plugins an email to the author can be useful
