First: by a “hypotenuse number”, I assume you mean any number of the form a[sup]2[/sup] + b[sup]2[/sup]. If that’s correct, then I think you can show that you can’t form squares of an arbitrary side length, and in particular, 3 is impossible. But I don’t know whether this generalizes to all non-hypotentuse numbers.
Consider the two lines that form the opposite sides of this square. We can assume that one of them passes through (0,0), while the second line has an equation of ax + by + c = 0. (The first line will obey a similar equation, but with c = 0.) Since the second line connects two points with integer coordinates, it can be shown that we can put this equation in a form where a, b, and c are all relatively prime integers. (“Relatively prime” means that they have no common divisor other than 1.) We can then use known results from plane geometry to find that the distance between these two lines is
D = c/√(a[sup]2[/sup] + b[sup]2[/sup]).
If these two lines are to form opposite sides of a square of area 3, then the distance between them must be D = √3, which implies that
3 (a[sup]2[/sup] + b[sup]2[/sup]) = c[sup]2[/sup].
This implies that c is a multiple of 3, or c = 3d for some d; which means that
a[sup]2[/sup] + b[sup]2[/sup] = 3 d[sup]2[/sup].
Thus, a[sup]2[/sup] + b[sup]2[/sup] is also a multiple of 3. But square numbers are always either multiples of 3 or 1 more than multiples of 3. (In fancy language, they’re either 0 or 1 mod 3.) The only way for a[sup]2[/sup] + b[sup]2[/sup] to be a multiple of 3 is for a and b to both be divisible by 3. But we said above that a, b, and c had no common divisors greater than 1. This is a contradiction, and so we can conclude that it’s impossible to form a square of area 3 in this way.
This proof relies critically on the fact that a[sup]2[/sup] + b[sup]2[/sup] can’t be a multiple of D[sup]2[/sup] = 3 without a and b also both being multiples of D[sup]2[/sup] = 3. If D was some other non-hypotenuse number, this proof technique might not be possible. It still holds for D[sup]2[/sup] = 7 (the next non-hypotenuse number), but I don’t want to work out D[sup]2[/sup]= 11 (the following one) before I’ve had my coffee. More later, if I have further thoughts.
ETA: oh, and please don’t hesitate to ask for clarification if any of the above is unclear; I’ve used a bunch of jargon and glossed over a few results, but I’m happy to fill those gaps in if need be.