You want to keep the LED current about the same as it is with the batteries.
You could either reduce the voltage to be the same as the battery voltage, or, increase the current-limiting resistance that is used in series with the LEDs.
First, you need to determine what voltage is actually being used, from the batteries. There are four 1.5-volt batteries but we don't know how they are connected together. The voltage used for the LED circuits could be anywhere from 1.5 Volts 6 Volts, depending on how the batteries are connected.