Being on the field is only one reason. The other is that baseball uniforms aren’t that unusual to wear around. There is no reason for a basketball coach to wear a tank top and shorts or for a football or hockey coach to put on pads.
Not that I like to walk around in stirrup socks.
I assume you’re talking about a conference on the pitcher’s mound.
When I played football in junior high, the coach, reserve players, and other assorted hangers-on were restricted to a specific portion of the sidelines (in the middle of the field; I can’t remember what yard lines the marks were at, but I know it wasn’t all that close to the goal).
I remember being told that our team could be penalized if we left the designated area. I think it was the same penalty for having an extra player on the field itself (i.e. in bounds) and not a distinct term and penalty for being on the sidelines but too near the goal, but I’m not positive about that.
Baseball is the only team sport I know of where the manager/coach routinely goes into the actual playing area during the game. I have seen football coaches go onto the field as a result of a player injury, though.
Football closely regulates where players and coaches can stand. Generally players have to stay between the 25 yard lines and 3 yards off of the field except for a certain number of coaches and one player (usually the one who brings in the play).
Basketball coaches have something referred to as the “coaches’ box” which they are not supposed to cross without the referee’s permission. In the NBA, I believe it’s the halfcourt line. In the NCAA, it’s much closer to the bench.
Hockey coaches don’t wear skates so they can’t do much. Besides they need to be on the bench to tell players when to substitute.
Baseball does regulate the number of trips a manager may make to the mound (2 before having to remove the pitcher unless there is an injury.) There are even more restrictions on the amateur level.