It seems that I can't add any extra pages. I think that might be the problem, not the plug in. I have tried to add a FAQ page and I get the same error as with the contact form page.
EDIT: Something was obviously wrong with my page.php file. I downloaded from WordPress the original one for the Twenty Twelve theme that I was using and saw that it was different from the one I was using. After using the original file the problem is now gone.