If the ECDSA key pair is still derived using an hmac keyed by the private key, I can do that with the key loaded in the tpm. Otherwise it is unlikely to work with my current setup.

Once the key is loaded into the TPM as an HMAC key, you can no longer get the plaintext key back out. You have to use the TPM to perform any HMAC operations using the key.