Apparently the courts have looked at it and it is legal.
Public Citizen v US District Court for DC (2005 Circuit Court case)
FIELD et al. v. CLARK, Collector (1892 US Supreme Court case)
Summary can be found at the link below: