We spent a night here during a college outdoor program's spring break trip. This was our third night out and our first night inland on the island. We came ashore at Plum Orchard and tied our boats up on the shore. It's a bit of a haul down an sandy road that then turns into the Duck House Trail. Depending on how your pack your gear, this could be a pleasant walk or it could be a buggy slog. So either way, be prepared and if you're paddling pack wisely.
The site itself is great. It's wooded and the ground is a dirt/sand combo which is nicer than all sand. The campsite is also a short walk to the ocean side of the island. Be aware that this is not any potable water at Yankee Paradise. You will either need to fill your water up from a spigot at Plum Orchard or walk a bit down the parallel trail (see the NPS map) to find a pump.
These sites are in the middle of the island and relatively wooded. So you may not experience enough breeze to keep the bugs at bay so be prepared. This is a great option if you're paddling or walking around the island.