Navigating to an answer from the user's page only "shadow votes" it for you, it doesn't get counted.