Drivers are a part of operating systems (like GNU/Linux) that help communicate with the hardware. 

People often draw operating systems as part of a pyramid:

 

A pyramid with four layers: apps, utilities, kernel, hardware.
The only pyramid scheme you can rely on...

  • Hardware Your computer and things you've plugged into it.
  • Kernel The part of the operating system which, among other things, dishes out the computer's resources and gives other programs an easy way to work with the hardware. This is the 'Linux' in 'GNU/Linux'.
  • Utilities The part of the operating system which helps you use the computer. This includes stuff like the programs used to browse and make changes to your files, connect to the internet, draw windows on the screen, and build programs from human-readable source code. Many GNU programs used with the Linux kernel fit in this category, which is why we call the whole operating system 'GNU/Linux'.
  • Applications The programs you use to actually do stuff with your computer. This includes web browsers, media players, image editing programs and games. Usually, people don't treat these as part of the operating system. 

(It's actually a bit more complicated than this, but this works for our purposes.)

The kernel is the part we're interested in, since drivers are usually part of it.

Drivers help the kernel convert instructions from utilities and applications to ones the hardware understands. 

Some drivers have an extra job. Some pieces of hardware, like Wi-Fi adaptors or video cards, need a program loaded onto them so they know how to work. These programs run on small processors separate from the main one in your computer. This firmware is usually included with the driver.

Because these programs don't run on your main processor, they don't really count as hardware or software. Since they're almost in-between, people call them firmware — neither hard nor soft. In our pyramid, firmware kind of fits in a fifth layer between the kernel and hardware.

A pyramid with five layers: apps, utilities, kernel, firmware and hardware. The firmware layer overlaps the ones for hardware and kernel
...unless you count this one too.

Just like other programs, firmware should be free software. Unfortunately, many otherwise-free drivers contain proprietary firmware. Usually this firmware is supplied as a bunch of unreadable numbers: a 'binary blob'. 

Drivers that aren't free — even in part — make us dependent on the companies that wrote them. Without source code, it's very difficult to investigate malfunctions and almost impossible to fix them. Only the company that owns the driver has the ability to do it for us and they may well decide that it's not in their interest to do so. If a driver is entirely non-free, it can't even be independently updated to work with newer kernel versions or rebuilt for new architectures.

When the inner workings of non-free drivers and firmware are secret, we're forced to take companies at their word that they operate as intended and don't contain malicious features. Useful research is also stymied. Fully-free drivers like ath9k have been the subject of dozens of academic papers investigating how wireless networking can be improved, all because their free license gives people the right to tinker and learn. What if the same could be true for even more hardware?

We should use hardware that works with fully-free drivers. Our independence is worth it.