Either way seems fine to me, except that there is probably some subtle difference I can’t put my finger on at the moment.
Or putting it another way, I would be more likely to say “in the second grade”, but still, “in second grade” isn’t going to raise my innate Chomskyan grammatical hackles either.
To shed some light on this, I’ve always been interested in little grammatical questions of this type, and it so happens that I’ve spent much of my working life around Oracle databases. I’ve noticed that when we talk about database tables, the word “the” is omitted, or not, depending on the structure of the statement. One could say that they have “inserted some records into CUSTOMER_TRANSACTIONS”, or one could say they have "inserted some records into the CUSTOMER_TRANSACTIONS table. But if one says “inserted some records into CUSTOMER_TRANSACTIONS table”, then you know that the speaker is not a native speaker of English. For me, I think the “in second grade”/“in the second grade” dichotomy may be similar, but, as I said, I can’t nail it down.