Fast BitCounting

it was a common technique in 8 bit writing. I used to write adventure games where object properties were stored in bits…

  • carryable
  • wearable
  • worn
  • edible
  • sourceoflight
  • lit

end so on.
It saved a ton of memory

2 Likes

Back in the days when 640kb of RAM was considered enough for anybody I can understand why such space-saving techniques were required but nowadays when even £70 phones have 2GB of RAM? I’d give it a miss.

1 Like

One of the things that I miss from the PL/I programming language is bit strings. You can do a lot more with bits compared to simply using bytes. Let’s take an Sudoku solver as an example. When bit presentation is used for grid numbers, where bit position is used to mark the number inside the grid cell. It only takes just a few lookup tables and bitwise operations to test all the possible number candidates for every cell inside the Sudoku grid! You can then easily pick first the grid cells with singles and hidden singles to minimize the search space for back tracking solver.

For what that all is needed. We have really fast computers today. In Java I would make a class for the booleans and an ArrayList and that’s it. Can be big as it wants to be. No problems. Is flexible in it’s size and it is fast enough for a^ll needs on todays computers. For what it is needed to do something really memory saving and using Swift for it? It becomes big in memory while loads a bunch of libraries. I dan’t say that a kilobyte will be a decision taker for this question.

1 Like

And that there, is why phones NEED 2Gb ram.
If a boolean = an integer, and an integer = 64 bits, it’s using 64 times as much memory as it (strictly) needs, because “It’s easy” and “Everybody has loadsa RAM”

(rant mode start… dont mind me…)
Then all languages are built on other things… libraries, calls to sub libraries and so on.
Nothing we code these days is ‘just do the thing’.
Its all ‘prepare a wish list, fill a class, call a routine, which checks and then calls another and another, all the way down until something somewhere does the thing’
Bloat.

(rant mode cancel)

1 Like

That’s the plan… because my time is expensive but hardware is cheap.

Unless you’re trying to upgrade RAM in an Apple PJ made Mac, then an extra 8GB of RAM can end up costing over a $1,000!

Everything Apple looks inexplicably expensive to me :money_mouth_face:

1 Like

I took the occasion to get a better understanding of Swift’s memory management and found this helpful thread:
why-is-a-bool-taking-7-bytes-of-memory.

Based on one answer I tested Bool arrays of different size in order to see their memory footprint:

let boolArray1 = Array(repeating: true, count: 1)
let totalBytes1 = boolArray1.capacity * MemoryLayout<Bool>.stride
let usedBytes1 = boolArray1.count * MemoryLayout<Bool>.stride

let boolArray8 = Array(repeating: true, count: 8)
let totalBytes8 = boolArray8.capacity * MemoryLayout<Bool>.stride
let usedBytes8 = boolArray8.count * MemoryLayout<Bool>.stride

let boolArray50 = Array(repeating: true, count: 50)
let totalBytes50 = boolArray50.capacity * MemoryLayout<Bool>.stride
let usedBytes50 = boolArray50.count * MemoryLayout<Bool>.stride

print("This array has a \(totalBytes1) byte buffer, of which \(usedBytes1) bytes are used.")
print("This array has a \(totalBytes8) byte buffer, of which \(usedBytes8) bytes are used.")
print("This array has a \(totalBytes50) byte buffer, of which \(usedBytes50) bytes are used.")

Results:
This array has a 16 byte buffer, of which 1 bytes are used.
This array has a 16 byte buffer, of which 8 bytes are used.
This array has a 64 byte buffer, of which 50 bytes are used.

Looks like Swift happily uses 1 byte per boolean value stored in an array + alignment.

Still used today, now with more bits :smile:, this technique won’t go away never. Many data structs carry lots of flags (0/1) or microvalues in bit fields (like 0 to 3 using 2 bits somewhere in a bit structure).

Windows carries such kind of structs in their API, macOS and Linux too.

If you do low level programming, your life is full of those kind of values and flags. Hardware always pack a lot of info in bits and send them to you to decode and act on them. And you do the other way around.

1 Like

yeah. Bit masking is quick and powerful.
The only odd part is counting the number of bits.
Sam has found a one-line-of-code answer.

1 Like

Can also let the CPU count it…

The following built-in functions are changed to generate new SSE4.2 instructions when -msse4.2 is used.

int __builtin_popcount (unsigned int)
Generates the popcntl machine instruction.

int __builtin_popcountl (unsigned long)
Generates the popcntl or popcntq machine instruction, depending on the size of unsigned long.

int __builtin_popcountll (unsigned long long)
Generates the popcntq machine instruction.

Its likely Arm has something similar maybe…

Documentation for them is like:

__builtin_popcount() is a built-in function of GCC compiler. This function is used to count the number of set bits in an unsigned integer. In other words, it counts the number of 1’s in the binary form of a positive integer.

And usage is like:


// C++ code to demonstrate the
// __builtin_popcount function
#include <bits/stdc++.h>
 
using namespace std;
 
int main()
{
 
    int n = 4;
 
    // Printing the number of set bits in n
    cout << __builtin_popcount(n);
 
    return 0;
}

Edit: The arm cpu instruction for same seems to be neon_cnt

1 Like

Which is how I found the Swift function for counting bits… I stumbled across the processor instruction, then figured out which Swift API used said instruction.

The information I found said the instruction in the ARM chipset is VCNT.