There are two oaths of office administered. The constitutional oath and the judicial oath. Traditionally the Chief Justice administers the constitutional oath. I believe that the Chief Justice also typically administers the judicial oath (although Kennedy administered it to Gorsuch who had been his law clerk).
After the Senate confirms a judge, the President must issue a “commission” which he signs and then is countersigned by the Attorney General. See here. The commission is then presented to and accepted by the Court. (Although, I don’t think it can be rejected, the judge becomes a judge when the commission is signed). And then the judge can be sworn in.
I don’t know what timeframe is standard. But Gorsuch was confirmed on April 7. His commission was dated April 8 and he was sworn in on April 10. So, he could probably be on the bench by Tuesday.
ETA: Kagan was confirmed on August 5 and sworn in on August 7.