Plan when you are going to enjoy your free time,and treat it in the same way as anything else on your calendar.If something else more important comes along you can choose whether or not to postpone it,but never cancel it! Do what you want. Many people worry too ...