I don't think it will make any difference between 2 and 4 layers. When you get up to over 8 or 10 layer, then you have to worry about it. It is all about cost of pcb, some cheap place specify the minimum to be bigger. Call the place you fab your board.
I believe the main reason for bigger pads and vias for many layer boards is because they worry about the layers slide and have a slight offset when they press on it to form the board. I designed a 26 layer board, and I had to make the vias pad size bigger to ensure the drill land on all the pads in different layers. But that's an extreme. Talk to the fab house, we can't tell you the lower limit.
For high volume production and reliability, I would use more relaxed size to make sure the boards are reliable.