To delete a folder, simply remove all campaigns from the folder, and it will automatically be deleted. 

To remove a campaign from a folder, open the folder, then hover over the campaign until the 2 small option icons appear. The left-hand option is "ungroup from this folder". Click it. You will not prompted for confirmation. The campaign will immediately be removed from the folder.

Note: It is not possible to change the name of a folder. If you need to change a folder name, you will need to delete the folder and create a new one.