If you go to the bottle depot, you can buy bottles there real cheap.
As for starter equipment (assuming festa-brew, brewhouse or similar kit), you could make do with the bottles, a 23L carboy, a racking cane (with attached hose), bottle-wand, and whatever sanitiser you want to use. You'd later need a bottling bucket or carboy.