I’m not a lawyer, but it seems like you’re saying that by putting up the sign, he’s somehow granting permanent, irrevocable consent for all the future? I don’t see how that can be the case. The sign indicates you need his consent to drive on the roadway. Consent can be withdrawn at any time. If he withdraws consent, the lumber mill owner can’t drive on it anymore.
Having said that, I think Mama Zappa’s idea to consult a lawyer has merit. Specifically, I’d be very concerned about liability issues. If a truck overturns because the roadway was rutted, or a tree falls onto a vehicle, you could wind up looking at a lawsuit. Even if you trust him 100% (which I hope you’re not foolish enough to do), his insurance company could sue you to try to recover their costs.