Hi, you can use simple comparator, like a LM393, all is depending from this do you have some available part in your circuit that can be used. The diagrams from IanP is good, but have one big disadvantage, the LED is on when the voltage is in nominal range. There are two problems:
1- Via the led you will have a current that will be extra load, and discharge your battery
2 - The human is more sensitive to light on, but in this case you will have light off on event.
How IanP say, you have many different ways, if you can be a more specific, we can help you.