Essentials:

Cycling is a very equipment heavy recreational activity. If you don't have them already, these are items that you should probably start with. The second set of items, although nice to have, are probably better to get after you've gone a few rounds on your bike.

A Bike

How would you bike without one?

A Helmet

Probably the second most important thing to buy. It saves lives.

Hydration

Keep yourself well hydrated, even if you feel like you might not need it.

A Bike Pump

It's always good to have your tires inflated just right.

A Tool Kit

You never know when you're going to need to make a small repair.

Nice to Have:

Gloves

After long rides, these will help prevent blisters from forming.

Bike Clothing

Gym clothes works to start. Many of these items will improve comfort.

A Lock

Very useful in metropolitan areas.

Bike Lights

Useful and, in some cases, required to ride your bike at night.

A Storage Rack or Bag

Great, especially if you're using the bike to commute.

A Better Seat

Your rear will thank you later.

Maintenance Gear

Regular maintenance of your bike will help keep it going.