When you log in, the website sends a "challenge." Your device signs it with the private key, and the server verifies it with the public key. Since the private key never leaves your device, hackers cannot steal it from a server.