There are two ways to Oscarsborg from Oslo city:
You can get the ferry on the Oslo fjord slowly for an hour and 88 crowns. There are awesome views from the upper deck of the ferry during it's route in Oslofjord. The ferry leaves from the berth Rådhusbrugge 4 on square Rådhusplassen in the center of Oslo.
Also you can get the bus #541 from the Central Bus Station of Oslo to Drøbak town. Price is the same, but bus goes a little faster than ferry. In the little town of Drøbak local ferry will get you to Oscarsborg within 5 minutes.