Split(x,x)

General Programming
1

Hi All,

Mac Desktop, API1 only, 19r1.

I have a large (to me at least) up to 20Mb text file to search through for several differing patterns of data.

it is a file of data generated by one of my Amateur Radio programs and it includes callsign and location (public QRA data published by the user with the expectation it is used by anyone to determine an approximate location).

This file is updated every 60 seconds and is intended by the app to be parsed by third parties to add features to the application use.

I am using Split(x,x) which works ok and surprized me how quickly it can split a 10Mb file into an array of 15 elements following an EndOfLine.

But I had thought it might be simple to add to the Split(x,x) a secondary hack or two to tweak it to remove more of the unwanted stuff in a single pass.

The question is, what is the raw code for Split(x,x) or anything that anyone has which has a similar function, or is there different way to remove expected data from a string of data that is not a built in Xojo method and will at least not be any slower than Xojo inbuilt stuff might be.

I am not wanting super speed like less than 100ms to parse out all the stuff I want, but I might like to keep it below 10 seconds on its first pass at boot up, subsequent passes will use the time stamp generated by the file (at 1 minute intervals) to work on only a very tiny amount of data, maybe 2k each time.

Thanks for looking.

Mark

2

Guess it depends on what it is you need to do
Replace and ReplaceAll can be useful - but on huge strings they take in one, process it and produce another so at one point you have the “old” data and “new” data in memory at the same time
That may, or may not, be an issue

3

I am not thinking replace is what I would like to do, remove is more like it.

in looking at replace I noticed trim which might help in one of the instances I have which is great.

trying to explain in a note what the data needs to be parsed into and from what, is a lot more complex than I am sure any one needs to see.

trim will now get a testing out and I will come back if there is anything I might need to optimize.

Thanks Norman.

4

Replace will remove if you replace with an empty string.

For very powerful (but super complicated) there’s the RegEx stuff.

5

I was going to mention RegEx.Replace and then I remembered how I hate Regex LOL. I will be working on unpdating an app I worked on some years ago and I need to work with some large .edi files, I re-read the RegEx docs and decided I’m keeping my Split() solution :slight_smile:

6

Thanks both,
still to get to test replace, if the empty string removes rather than replace with an empty string it will work fine.

I also forgot to mention regex is not something I want to spend time learning, it seems to the uninitiated (me) like linux command line without any of the charm or well written instructions that linux has.

I am glad Hector that I am not the only person to want to persevere with the less complicated inbuilt functions!
BUT we are both fools to dismiss the absolute power of RegEx, I know I just can not be bothered to study it, my bad for sure.

7

True. In fact RegEx is in my list of things I want to learn in the next couple of months. I don’t like it but I’m well aware of it’s power.

I’ve been following this guy’s youtube channel for a while and I like the way he explains things:

https://www.youtube.com/watch?v=7DG3kCDx53c

I guess I’ll watch the series on RegEx a couple of times, read about it and then practice practice practice. Perhaps I’ll even manage to not hate it in the future.

8

Hahahahahahahahahahahahahaha … have to admit this is the best line of humor I’ve seen in a while :slight_smile:

1 Like
9

regex, Perl, TECO - all write only.

As somebody said: there’s no obfuscated Perl contest because there’s no point.

Same applies to the others I mentioned.

1 Like
11

I thank you Norman!

12

to get back to one of the original questions, is there any code in Xojo than anyone has that mimics the Split(x,x) function.
I wonder how this function, or others such as Left, Right etc are formulated, is it a C function of some sort or what? (forgive my utter ignorance of how my random typing in the IDE ends up as an application)

13

It is part of the framework that Xojo provides that is written in C++/Objective-C (depending on platform the mix varies)