If you can get a site, do it!
I really love Idyllwild and Fern Basin is one of the few campgrounds there. This is the definition of small mountain town so make sure to stop and check out the little coffee shops in "town". The campground is fairly small and off a pretty rugged dirt road, but it's worth it. I've stayed here a few times and it's always peaceful and quiet. This is a tenter's campground if you ask me. There's lots of hikes around the area - some steep, some like a dirt road. There's an area off the far end of the campground that has big boulders that you can climb up on for a perfect view of the sunset.
Site14 & 18
Month of VisitJune