You'll need to get shorter bolts to put back into the cover so that it seals. However I would recommend pulling the lower cover and using RTV to reseal it. I had the same issues. But since your gasket got all oily, that is why you'll want to reseal rather than just run bolts back in. If you just throw in bolts, you'll probably continue to leak. You'll only need two bolts and unfortunately I can't remember the size at the moment.
Sorry I didn't explain this before and caused you some shit. I just figured you would have noticed that.