I sail out of Atlantic City. This summer I would like to spend a few days
sailing around the NYC area. Are there safe anchorages? Cheap tieups?

Horseshoe Cove and Atlantic Highlands inside Sandy Hook. 79th Street Boat
Basin in Manhattan. City Island in LIS, Port Washington, Oyster Bay.

Most people avoid Chelsea Piers about 20 Street Manhattan Hudson River.

Several marinas accross from Manhattan in New Jersey, all with a rep of being
pricy.

I am not aware of any dinghy docks in Manhattan.