PSP
PSP is a shortcut for Portable Sony Playstation - a sweet gadget by Sony that lets you listen to music, watch movies and videos, play games, see photos, listen to the radio, browse internet, and do many other things. And it fits in your pocket.

Firmware
Firmware is what people call the operating system of the PSP. All the features, the "possibilities" of your PSP, such as watching movies and listening to music - are applications (programs) that are written on a Flash drive. This is like a package, the core, the brains of your PSP.

Since the firmware is stored on a Flash drive, it may easily be replaced. Usually, upgraded to a new version.
A couple of years ago, when PSP started, the firmware version was 1.0. By the time I'm writing it now, the latest version is 3.90.

Custom Firmware
Besides the official firmwares created by Sony, there are firmwares by 3rd-parties.
Main reason for creating Custom Firmwares is to enable your PSP for homebrew.
First official firmwares were open for people to download and install many games from the Internet, which was very great. But after version 1.5, Sony seems to have decided to stop the luck, and restricted people from download and install games, which became known as Homebrew.

Homebrew Enable
Homebrew is basically the common name for non-official things. Basically, for games. Since Sony originally restricts you from share games by copying discs or downloading them from the Internet, a lot of people take off the limitation by installing Custom Firmwares.
And once done, you are open to thousands of games to download, absolutely free of charge (despite original $20/UMD).
Also, many applications, since they are treated as games, may be run under custom firmwares only.

Your firmware
It is easy to find out what version of PSP firmware you are running.
Do the following:
- Turn on your PSP
- Make sure you are in the PSP's main menu
- Navigate to Settings → System Settings → System Information
- Press "X"

You will see a line saying: System Software Version x.xx

x.xx is your firmware version.

To learn more about homebrew games and downgrading, take a look at this article.