Scrub the potatoes under cold water to remove any dirt. Then, completely dry the potatoes.
Heat your grill to 375°F.
Wrap each potato in foil.
Place on hot grill over indirect heat. Cook for 1 hour, turning occasionally.
Remove from grill and allow to cool enough to handle.
Lay the potatoes flat on the counter or cutting board. Cut off the top of each potato.
Using a spoon, scoop out all but about a 1/4-inch of the flesh of each potato (creating a "potato boat").
Place removed flesh from all 4 potatoes into a large bowl.
Add butter, sour cream, sliced green onion and buttermilk. Mix until thoroughly combined and a mashed-like consistency forms.
Season with salt, pepper, and garlic powder, to taste.
Brush the outer skins of each potato with olive oil.
Sprinkle with salt.
Place the oiled and seasoned potato shells on a sheet pan.
Evenly fill each potato with the mashed-potato mixture.
Then, evenly top each potato with shredded cheddar and bacon crumbles.
Place the filled potatoes directly on the grates of the grill over direct heat. Slightly press down on each potato to get a flat bottom for extra contact. Cook until skins are crisp and grill marks form, about 8 - 10 minutes. The insides of the potatoes should be warm and the cheese fully melted.
Remove from grill.
Serve hot.