It meant, according to authorized history book I own a copy of, that they were an Association of Young Christian Men.
I see no sign of that here in Australia: it is noticably different in some other countries.
Their assocation with physical fitness came from the theology known as “muscular Christianty”. Wikipedia tells me that “The movement came into vogue during the Victorian era and stressed the need for energetic Christian evangelism in combination with an ideal of vigorous masculinity.”
I see no sign of energetic Christian evangelism in their community work now.