I need to tokenize a simple text string, the consists of “words” and “punctuation”
I need to split by the punctioan (including spaces) and return an array of all bits. In such a way that JOIN would return the original string
This.Is/a=Test
would return
["This", "." , "Is", "/", "a" , "=" , "Test"]
most Split functions toss out the delimiters, or only allow one
let s: String = "Lorem Ipsum is simply dummy text "
print(stringWidth(s))
let v = s.split(separator: "")
print(v)
["L", "o", "r", "e", "m", " ", "I", "p", "s", "u", "m", " ", "i", "s", " ", "s", "i", "m", "p", "l", "y", " ", "d", "u", "m", "m", "y", " ", "t", "e", "x", "t", " "]
which splits it into individual characters… which I can use, just slower
as I need to measure the width of each