Don't forget the hands-on part of electronics. Just like you'd write up a simple function to test out a piece of code, you should also start dabbling with small circuits.
Get a small-medium
solderless breadboard to do your prototyping on, a cheap soldering iron (about US$20), some small gauge solid core wire (to use as interconnects on the breadboard), a wire stripper, a simple volt/ammeter (US$20-40), and a low voltage DC power source. You can start with 6V, 9V batteries initially. If you can find a small, old (functional) ATX power supply, use a wire or switch to connect the PS_ON line to ground to fire it up... then you'll have some loosely regulated +12, -12, +5, -5, +3.3 VDC supplies that could be useful on later projects.
As for parts, you can find resistor kits and capacitor grab-bags from vendors like Newark, Digikey, etc, if you don't have a local place to source small electronic components. Start with leaded components, then move up to surface mount when you start messing around with making your own circuit boards.