You should leave the fan on. Just because the lights are off doesn't mean they aren't growing. As long as the fan is blowing on them it will keep fresh air moving over the plants and also help strengthen the stalks. Plus the fan cost pennies to run.
Unless you are running a sealed room with CO2 let the fans run all the time. Increased air exchange and circulation is never a bad thing.
Keep the fan on. They need the air to move constantly. If you turn your fans off the humidity will get to levels the plants wont like.