This is due to the physical behavior.
It is the same as with a mass. When the force acting on the mass is finite, its speed can't change instantaneously.
From the math you can see it also: Speed = (1/m)*integral(F*dt).
A finite value force will never result in an instantaneously changing speed, or position.
A mechanical spring is a counterpart component. You first have to displace on end of the spring before a force develops.
The electric capacitor you can see as a jar/pail that is filled with water. V = heigth of water, I (electrical current) = liters/s, m^3/s, etc.
d(height)/dt is propoportional to the volume/s that enters the jar/pail. As the input flow is practically spoken finite (think of filling a pail with tap water), the rate of change of heigth is finite also. So the heigth can't change instantaneously