Be careful with encoding & adding "strings"

If True Then
  Dim s1 As String = "hello"
  
  System.DebugLog "before appending chrb(&h09) s1.Encoding is " + s1.Encoding.internetName
  
  s1 = s1 + ChrB(&h09) // this one WILL give you a NIL encoding !
  
  If s1.Encoding Is Nil Then
    System.DebugLog "after appending chrb(&h09) s1.Encoding is nil"
  Else
    System.DebugLog "after appending chrb(&h09) s1.Encoding is " + s1.Encoding.internetName
  End If
  
End If

If True Then
  Dim s1 As String = "hello"
  
  System.DebugLog "before appending Chr(&h09) s1.Encoding is " +  s1.Encoding.internetName
  
  s1 = s1 + Chr(&h09)
  
  If s1.Encoding Is Nil Then
    System.DebugLog "after appending Chr(&h09) s1.Encoding is nil"
  Else
    System.DebugLog "after appending chrb(&h09) s1.Encoding is " + s1.Encoding.internetName
  End If
  
End If

If True Then
  Dim s1 As String = "hello"
  
  System.DebugLog "before appending Encodings.ASCII.Chr(&h09) s1.Encoding is " + s1.Encoding.internetName
  
  s1 = s1 + Encodings.ASCII.Chr(&h09)
  
  If s1.Encoding Is Nil Then
    System.DebugLog "after appending Encodings.ASCII.Chr(&h09) s1.Encoding is nil"
  Else
    System.DebugLog "after appending chrb(&h09) s1.Encoding is " + s1.Encoding.internetName
  End If
  
End If
1 Like
           before appending chrb(&h09) s1.Encoding is UTF-8
           after appending chrb(&h09) s1.Encoding is nil
           before appending Chr(&h09) s1.Encoding is UTF-8
           after appending Chr(&h09) s1.Encoding is UTF-8
           before appending Encodings.ASCII.Chr(&h09) s1.Encoding is UTF-8
           after appending Encodings.ASCII.Chr(&h09) s1.Encoding is UTF-8

Fascinating. On Windows, apparently a string doesn’t have an encoding until, well, I don’t know when. This is what your code returns on Windows 10 and both Xojo 2020r2.1 and 2021r1.1.

10:17:20 AM : before appending chrb(&h09) s1.Encoding is 
                      after appending chrb(&h09) s1.Encoding is nil
                      before appending Chr(&h09) s1.Encoding is 
                      after appending chrb(&h09) s1.Encoding is 
                      before appending Encodings.ASCII.Chr(&h09) s1.Encoding is 
                      after appending chrb(&h09) s1.Encoding is

Hmm… I’d rather think the InternetName is empty/not found. A string must have either an encoding or a nil encoding, and Norman’s tests check for nil encodings.

Windows sometimes sets nil but also sometimes sets an empty encoding what is really a problem cause you’ve to detect iv there is valid encoding or not

An empty encoding… :thinking:
Is that a real encoding which has no known/displayable name? A nil encoding (I guess no, as you mentioned it in the same sentence)? Or the Windows’ default/ASCII encoding?
I can’t imagine an “empty encoding”. Kind of having an object which has no colour and is not transparent either. :thinking:

The problem is: nobody knows. I had to write a software patch for that

exactly
Chrb appends a NIL encoded BYTE (use Chrb & the other B functions when you use a string as “bag of bytes” )
everything else makes sense

ugh - that _should _ be reported as a bug since string literals _should _ be UTF-8

Well, they do have a encoding. looking at the debugger says it is UTF8 BUT the encoding itself has no name:

image

Adding a ConvertEncoding;

Dim s1 As String = "hello"
s1 = ConvertEncoding(s1, Encodings.UTF8)

The encoding changes to:

image

:rofl:

That too se should be reported as a bug :frowning:

http://feedback.xojo.com/case/64905

Is this new with the recent test release? Or has it been around for a while?

I havent tested old versions to know for sure
I’d suspect it’s been around for a while