It is a good idea to avoid connecting LEDs, resistors and other electronic components directly to the Arduino because you can easily damage the headers on the Arduino and the circuit quickly turn into an unorganized mess. When prototyping, it is a lot easier to connect the components together using a solderless breadboard.
A solderless breadboard enables us to connect electronic components together without the need to solder. You can get a good solderless MB-102 breadboard for under ten US dollars. They are also reusable, which makes them perfect for creating prototypes and experimenting with circuit designs.
I would recommend not skimping too much on your breadboards because you will be using them so much. You can find some very cheap and small breadboards like the one in the following photograph; however, it is a lot harder to organize your circuit...