The recently reported evidence for Gravitational Waves (GWs) in PTA datasets leaves the question of whether the signal is astrophysical or coming from the Early Universe, in the latter case originating from physics beyond the Standard Model. In this talk, I discuss several aspects of cosmological GW backgrounds that can help in addressing this question, and present results of dedicated searches in the datasets. In particular, I describe the imprint of the QCD crossover on GW spectra from transient early Universe processes. I then focus on two such sources: large curvature perturbations and cosmic domain walls. I also highlight the relevance of complementary constraints from cosmological observations on such scenarios for model comparison using PTA data.