Just don't go on a holiday weekend
If you go during the winter months (that is, if roads are open), you may likely be the only one there! The lake is not very easily accessible for camping but it's an easy hike down for fishing, photography, and simply enjoying nature. Keep your eyes peeled, you may spot a bald eagle. The actual campground is usually crowded with no privacy. Perhaps suitable for a large group. Restrooms, never mind, just bring a shovel and use the woods; it's cleaner. Just don't be disgusting and leave it all out. Leave No Trace, PLEASE. Also, you'll need to carry out all your trash. On a holiday weekend (Labor Day especially) there are so many visitors but otherwise only a few venture out that way. Your best bet for solitude is to drive 2-4 miles passed the campground which will be on your right and find a good dispersed spot. Just bring ribbon or rope to mark a tree from the road in case you need to travel down to Woods Canyon Lake store or out to a Rim viewpoint for cell service.