I used to dock my sailboat at nearby Stock Island, near Mile Marker 5. You'll notice that US 1 Overseas Highway is navigated by Mile Marker (MM), from MM 127 in Florida City to MM 0 in Key West.
5 hours isn't much time. If you want snorkeling, the closest decent spot to Key West is Western Sambo Ecological Reserve. I don't know if any of the boat operators' schedules would match your itinerary, or if any of them are still operating due to the pandemic.
At least get some photos at the Southernmost Point. Check out the Ernest Hemingway home with its 6-toed cats.
Then, you can do the Duval Crawl. The tradition is that you start out at the north end of Duval Street. Go to a bar, have a drink. Walk to the next bar, have a drink. I think I made it to 4, being the lightweight that I am, but my recollection may not be accurate. Eventually, you'll still be trying to walk, but you'll actually be crawling. Check with the bus service first, to make sure they'll let you board, assuming you can still find the bus.