I think maybe I should clarify in smaller/newer words my point above:
If by “choose an element of non-empty set X”, you mean “Give me an actual description which isolates a single particular element from X”, then of course I can’t do that for you; you haven’t even told me what the set X is. But, then, the axiom of choice won’t help you here either; it just tells you that certain choice functions exist, without giving any description that actually singles them out. And, if all you want is an existence proof without a description, you won’t need to invoke the axiom of choice in this case; by stipulation, X is non-empty (i.e., there exists an element within it), so such a proof is trivial.
What the axiom of choice becomes necessary for is something like “Each of X_0, X_1, X_2, …, is non-empty. Thus, let us define a sequence of elements a_0 in X_0, a_1 in X_1, a_2 in X_2, …” In proposing the existence of such a sequence, we are asking not just to make one choice, but to make a whole bunch of them, simultaneously. I.e., we want a proof not just that there are elements in each X_i (which is trivial to show; it was our stipulation), but, furthermore, that there is actually a function selecting a particular such element a_i for each index i. Demonstrating the existence of such a function is tricky if one has no means at hand for giving an explicit rule defining a particular such function; the axiom of choice is the principle that guarantees the existence of such a function, absolving us of the need to think up such a rule to “manually” prove its existence.
Note, however, that if we only had to make finitely many choices, then we could prove the existence of such a function “manually” anyway. The easiest way to see this is by considering the case where we already know there exists a sequence a_0, …, a_n chosen from X_0, …, X_n, respectively, and want to extend this to a sequence chosen from X_0, …, X_{n+1}. By stipulation, we will take X_{n+1} to have at least one element; it follows that there is at least one way of extending the old sequence a_0, …, a_n to a new one a_0, …, a_{n+1} doing what we want (in fact, there will be one such way for each element of X_{n+1}), and so, of course, such a sequence exists. Arguing inductively using this result, we find, even without assuming the axiom of choice, that finite “choice sequences” always exist. The axiom of choice only becomes necessary when we want to make an infinity of choices.
Also, another note of clarification: Plenty of mathematicians, though (the convention or majority or some such term) are happy to use the axiom of choice liberally, seeing no particular need to give arguments which don’t assume it or even to pay attention to whether or not they are invoking it implicitly.