The onset of psychosis is thought to involve interactions between environmental stressors and the brain, with cortisol as a putative mediator. We examined the relationship between the cortisol stress response and brain structure in subjects at ultra-high risk (UHR) for psychosis. Waking salivary cortisol was measured in 22 individuals at UHR for psychosis and 17 healthy controls. Grey matter volume was assessed using magnetic resonance imaging at 3 T. The relationship between the stress response and grey matter volume was investigated using voxel-based analyses. Our predictions of the topography of cortisol action as a structural brain modulator were informed by measures of brain glucocorticoid and mineralcorticoid receptor distribution obtained from the multimodal neuroanatomical and genetic Allen Brain Atlas. Across all subjects, reduced responsivity of the hypothalamus-pituitary-adrenal (HPA) axis was correlated with smaller grey matter volumes in the frontal, parietal and temporal cortex and in the hippocampus. This relationship was particularly marked in the UHR subjects in the right prefrontal, left parahippocampal/fusiform and parietal cortices. The subgroup that subsequently developed psychosis showed a significant blunting of HPA stress response, observed at trend level also in the whole UHR sample. Altered responses to stress in people at high risk of psychosis are related to reductions in grey matter volume in areas implicated in the vulnerability to psychotic disorders. These areas may represent the neural components of a stress vulnerability model.