There is a possibility that the hacker is using the "Forgot your password" feature to retrieve your password, so be sure to change your secret questions and check the backup e-mail address (if present) in your Yahoo account options.
You're very lucky in this case that you can still use your Yahoo account, as I've seen a few cases where people had their account compromised and the hacker had changed their password and the recovery options (secret questions & recovery e-mail.)
Finally, be sure to notify your Yahoo contacts (e.g. quick e-mail cc'd to the lot) that your account has been compromised, as it's very likely that this person is impersonating you asking your contacts to visit spam links or worse still, sending 419 scams to them faking that your are stuck in London, etc. and need cash.