It's not the encoder that you'll need, it's a decoder (isn't that what I wrote).
You can use a lookup table to do that job, but don't forget to debounce your keypad.
Depending on the strength of your encryption - you can make it as simple or as difficult as you like.
In the beginning, I would start with comparing the password with a preprogrammed register (during configuration). But start with decoding your keypad, that's a good point to start with the basics of VHDL.
Have you considered taking some sort of course, or take a book about VHDL (there are numerous examples in this forum of good books to start with)?
If you don't understand the basics of the language, you won't get very far.
With respect to the LCD - you will need the datasheet of it. Most LCDs can operate in 4-bit and 8-bit mode.
Of course I can give examples - I could even do the job for you, but that's not the point. You asked for basic help how to start - not that someone made the design for you, did you?