Yes if they're incandescent bulbs.
Sorta if they're LED's. LED's have to be connected through a constant resistor, because they can't regulate their own current and will just pop if connected directly to power and ground (which would likely be the case for full brightness on the rheostat). So, if they're LED's ,you'd want to connect all the grounds to a resistor, then to your rheostat, then to ground. If the rheostat has a non-zero minimum resistance, then it takes care of this already.