Am I the only one to think of Judy Blume when I opened the thread?
Damn and blast. Batch number 5 turned out a thick chocolate sludge, and it has the texture of grit. Time to do batch number six, which will be made with cocoa.
I am about ready to hang up my spatula in defeat.
o/` Kiss the goop goodbye,
the sweetness and the sorrow.
Wish me luck, for I’ll try too…
What I did for fudge,
what I did for fudge…
That batch was too dry,
Another one tomorrow.
Why is it turning to goo?
What I did for fudge,
what I did for fudge…
Wrooooooooooooooooong,
candy’s never wrooooooooooooooooong.
As we try again,
the failures we’ll remember.
Kiss that batch goodbye,
instructions I did follow.
Why won’t this crap set up true?
What I did for fudge,
what I did for fudge… o/`
I wonder, why skim milk? Maybe that’s messing it up. It just doesn’t sound right, to me. It’s fudge; don’t worry about the milk fat. It ain’t diet food. 
I’ve never had a problem with it being grainy because of the stirring after it comes off the heat (that I know of). Occasionally (and I rarely make the stuff as it is) I’ll cook it a little too long and it will be a little grainier than I want, but usually I get a nice fudge from it.
I would give this one a try before giving up completely:
http://www.foodnetwork.com/food/cooking/recipe/0,,FOOD_9936_26073,00.html
Alton Brown can usually be counted on for complete and accurate recipes - he’s also very clear about when to stir and when not to. Have you considered that maybe you haven’t been stirring enough at the end of the recipe?
Hope this helps -