It depends on what kind of system you are talking about.
In electronics or electrical engineering, an "open" circuit means there is no path for the electricity to flow through -- it is "turned off". A "closed" circuit means there is a full path for the electricity to flow through, so that it may flow from one side of the Power supply through the circuit and out to the other side of the power supply, thus doing work -- the circuit is "turned on."
In servo systems engineering (usually a discipline within electrical and mechanical engineering) there are "open-loop" or "open-circuit" systems and "closed-loop" or "closed-circuit" systems. This means the system either does not have feedback (open-loop) or does have feedback (closed-loop) to allow the controller to correct the signals it is sending to the motor or other thing being controlled, in order to correct for unexpected changes in the load or power supply etc. In a closed-loop system there will be some sort of input sensor that senses what the system is SUPPOSED to do, a controller to generate the signals needed to make this happen, a motor or other actuator to perform the work, and a sensor to sense the result and feed this back to the controller so that it can make corrections to the control system:
In mechanical engineering, particularly in hydraulics and pneumatics, an "open circuit" is a system which does not feed the working fluid back to the pump in a continuous cycle after being used, but instead exhausts it to the outside world. A "closed circuit" channels the used working fluid back from the motor or actuator to the compressor, where it is re-used. Almost all pneumatic (compressed air) systems are open-circuit systems, since it is simpler and doesn't hurt anything to release the used air back into the atmosphere. Almost all hydraulic (pressurized liquid) systems are closed-circuit systems because the hydraulic fluid is usually an expensive and often times toxic oil.
In SCUBA diving, there is also "open circuit" and "closed circuit" air systems. In an open-circuit system the air comes from the compressed air cylinder, through a set of hoses and regulators to the diver's mouthpiece, the diver inhales the air and when the diver exhales, the air just bubbles out into the water and is lost. Thus it is an "open circuit" in the pneumatic sense, because the air is only used once and then exhausted to the outside world. In a closed-circuit SCUBA system or "re-breather," a very small amount of oxygen comes from a compressed air or oxygen cylinder and is added to a reservoir of air from which the diver breathes. The diver's body uses some of the oxygen in the air and replaces it with carbon dioxide. As the diver exhales the breath goes through a chemical "scrubber" which takes the carbon dioxide out, and the compressed gas cylinder adds in a little bit of oxygen to replace it, and the mix is then sent back to the reservoir for the diver to breathe from again. Thus this is a "closed circuit" in the pneumatic sense because the air after being used by the diver is brought back to the reservoir and used again and again and again: