Thanks for the extra details. I created a similar form on one of my test sites and submitted a few forms. The replyto email address was always properly set up.
I wonder if the problem could be caused by your server's email configuration.
Could you try to use this plugin to bypass your server's mail functions, and use an external mail provider like your Gmail account?
http://wordpress.org/plugins/wp-mail-smtp/
Let me know if that helps.