It depends on many things: your age, diet, metabolism, exercise routine, sleep schedule, how often you eat, etc.
Then, whats the magic number?
Again, it depends. But most people with a comfortable routine and diet will go 1-2 times a day, usually around the same time of day each time.
In the end, you should just go every time your body wants you to go, trying not to miss any bowel movements that are ready to leave your system.
A change in routine or diet can effect how often you go. Not to worry, this is normal. For instance, if you go on a vacation or are away for a weekend trip that is outside of your normal routine your body can have trouble adjusting to the quick new change. You might not be able to poop for a couple of days, especially since there is a good chance your diet has changed while on your trip. Your body will adjust, and start getting back on track. Or, I am sure you will have to poop as soon as you get back from your weekend trip, and it will be glorious.
Logically, if you go uncontrollably and multiple times a day, this may be cause for concern. On the other end, if you don’t pass your bowels for weeks at a time, this could be concerning too.
In the end, you just need to listen to your body. No matter how often you go, if it feels natural and there is no cause for bleeding or pain, then you’re probably ok.
However, it is healthy to try and find a nice consistent routine of eating, sleeping, and exercising that will give your bowels consistent movements to give your body the nutrients it requires. Ideally, it should take around 20 hours for the food you eat to be passed through your bowels.