My idea for secure communication is to make client send it’s public key to the server. The server creates a shared secred (servers private, client public) which it uses to encrypt a session-key to the client.
If the client is sending an invalid public key, no communication is possible and the server drops the connection.
If the client is sending a valid public key, it knows the shared secred (servers public, client private) and uses that to decrypt the session key.
Now, session key can be used for encryption/decryption. Session key will always be random, because it’s not possible for anyone to guess what that is.
Each end knows its own private key
Any one can know the public keys
So if the client encrypts a message with its private key it can be verified that it IS the correct client by decrypting the message with the public key. That public key can only decrypt messages encrypted with the matching private key.
Then you do the reverse
WHAT gets encrypted can be anything - even a known shared “secret” because the private / public key pairs prevent anyone from using a different private / public pair that would encrypt & decrypt in the same way
I would suggest handling encryption and identity verification separately.
e.g. Here’s how SSH handles public key authentication:
- Insecure connection established (TCP, websocket, etc.)
- Each side generates a random encryption key pair, exchanging the public halves
- Each side generates the session encryption key using the exchanged public keys
- The session is now encrypted.
- Server sends a block of random data to the client.
- Client SIGNS the data with their long-term signing key.
- The server uses the public half of the client’s signing key to authenticate the signature.
- The client has proven its identity.