Controlling the self-assembly of supramolecular structures is vital for living cells, and a central challenge for engineering at the nano- and microscales. Nevertheless, even particles without optimized shapes can robustly form well-defined morphologies. This is the case in numerous medical conditions where normally soluble proteins aggregate into fibers. Beyond the diversity of molecular mechanisms involved, we propose that fibers generically arise from the aggregation of irregular particles with short-range interactions. Using minimal models of frustrated aggregating particles, we demonstrate robust fiber formation for a variety of particle shapes and aggregation conditions. Geometrical frustration plays a crucial role in this process, and accounts for the range of parameters in which fibers form as well as for their metastable, yet long-lived character.