You have three nice official launch areas with good fishin' nearby. Heckscher State Park, Captree State Park and the Babylon Town Park where the old OBI was. Heckscher flats have good fishing as do the marshes to the north and east of Captree. There is a nice cove to the west of the OBI but both this and the Captree launch (not the boat ramp) are on Fire Island Inlet which means boat traffic and very strong currents at times. I would suggest that you make an exploration trip without fishing the first time out. There is strong current in this inlet way after the time of high/low tide.
Getting involved with other folks is a great way to acquire local knowledge of conditions on the water and fishing opportunities. Nothing like being out with help if there's a problem. You need to get to know your boat and develop some kayak skills before you concentrate on the fish. There's a lot going on and the water won't cut you any slack if you make a mistake.
You have to dress for the water temps and wear your PFD at all times. You should also learn to reenter your boat if you get dumped. My advice is to seek out some professional instruction and get involved with folks who can show you the ropes.
May 17 NYS Parks has a Kayak Safety day at Captree with the Coast Guard and others where you can get a boat safety inspection, see some on the water demonstrations, meet some people and get a chance to use your boat in these very same waters. I'll post details when I get the specifics.