Céad mile fáilte!*
Which hotel? O’Connell street is wide and majestic, stretching for about a quarter mile north of the Liffey at O’Connell Bridge, the dead centre of the city, with some significant historical buildings (as well as the Dublin Spire), but it can be a bit seedy after dark.
Trains can be a bit pricey, and they’re pretty slow, so a day-trip to somewhere like Galway or Cork may be prohibitive, as you’d be travelling for 6-10 hours in one day. However, the east coast is well served by rail, and you can get to Wicklow or Wexford to the south of Dublin, and Drogheda or Belfast to the north. Belfast is a pretty nice city, and worth a visit. There are also coach tours - I believe you can get a coach to the Hill of Tara, Newgrange (both neolithic monuments north of Dublin), and the Boyne Valley, all in the same area and dripping with history.
The open-top bus tour of Dublin is pretty good, and allows you to jump on and jump off at various interesting things. Kilmainham Jail is on the route, and is one of the most interesting historical attractions I’ve been to here. I third the sentiments about the Guinness Storehouse tour. Very pretentious and expensive, though the advertising section was cool, as was the thing about the artisans who made the barrels, and the bar at the top does give some fantastic views and a free pint, but I’m not sure if it’s worth it.
Can’t think of anything. Hallowe’en is coming up, but that seems just to be kids blowing themselves up with fireworks. That said, every night in Temple Bar, which is five minutes from your hotel, feels like some kind of festival (to do with drink!).
Dublin Castle is vaguely interesting, but it’s not a real (ancient) castle. Can’t think of any other genuine castles near Dublin that you can visit, though. Powerscourt House in Co. Wicklow is do-able, but it’s a stately home rather than a castle.
Bushmills is on the northern tip of Northern Ireland, so a visit might not be realistic. However, Jameson’s is in easy walking distance of your hotel (or a short ride on the Luas, the groovy new tram that only opened last week).
By the way, it’s just become rather cold here - above freezing, but windy - and it rains at least once every day, so bring coats and umbrellas.
Since you’re on your honeymoon, I won’t suggest that you meet up with us DubDoper reprobates. However, if you want to, I’m sure we could hook up for a couple of pints. 
*(OK I’m pretending; I’m English but I live here.)