I'm sorry, but I'll be out of town on the day of your party. I'll have to beg off on your invitation.
I have an important meeting, so I'll have to beg off.