Because that’s not what Father’s Day is, as Thudlow Boink has had to say twice now. Mother’s Day and Father’s Day are written as singular possessives because they are personal celebrations of the unique qualities of our own mothers and fathers. They are not celebrations of the common nature of a group, which would seem almost insulting, as if your father was an interchangeable commodity. The singular possessive connotes the thought: “Father, this is your special day”.
This is as distinct from something like Veterans Day, distinctly plural, which honors the service and sacrifice of veterans as a group. It’s sometimes written as “Veterans’ Day” but unlike the case where it’s important to emphasize the singular, as above, the plural possessive is often dropped because of the somewhat awkward trailing apostrophe, and simply because it’s acceptable in some contexts for nouns to function as adjectives (“telephone call”, “winter jacket”).
Same for something like “Writers Guild”; the plural possessive would not be wrong, but neither is its omission. But “Father’s Day” without the apostrophe just mangles its meaning. Likewise I’ve seen Veterans Day (aka Veterans’ Day) sometimes written as “Veteran’s Day” which is stupid, as it prompts me to ask, “which one?” For “Father’s Day” the answer is: yours.