Baker's Hole is located very close to West Yellowstone. This is both a pro and a con -- pro, as it allows you to zip into town if you need some groceries, want to grab a pizza, etc. You can literally be in town in 3 minutes. 
The "con" side is that the main road into West is VERY close to many sites within the campground, and the road noise is pretty constant. We had a last minute trip and scored Site #16 (Electric) and while the site itself is great (timbered shade, roomy parking spot for the camper, hammock-friendly trees, food lock box, etc.) it is literally 40-50 yards from the road. 

At one point, about 30 Harley's roared by, and I couldn't hear my wife talking to me at the picnic table. Not good.....But, that is the deal if you land in one of the spots nearest the road. 

My advice (and plan next time) is to get there early (it is first come/first serve, no reservations) and head directly to the spots closest to the river. Not only does that put you within 10 seconds of fishing, they were a fair bit quieter than those near the road. 

The camp hosts check in and drive around in their golf cart frequently, taking care of the place. They do a great job. 

All in all, I would recommend this campground, as you have easy access to fishing the Madison, a quick trip to West Yellowstone if needed, or to start a trip in Yellowstone Park. 

A few tips:

- Bring bug spray and a citronella candle/Thermacell, etc. anytime in the summer. The mosquitoes can be really bad. 

- Bring a hammock; great trees to sling one up can be found at almost every site

- Bring your flyrod! 

- Bring bear spray, and be diligent about food prep and storage. This is grizzly country.