In this talk, I will review key ingredients used to construct off-shell higher-derivative invariants in (gauged) supergravity theories. The construction is based on an interplay between the superconformal tensor calculus, the superform approach to construct supersymmetric invariants, and novel off-shell superspace techniques. I will discuss how to obtain curvature-squared (four-derivative) invariants for (gauged) minimal supergravities in five (5D). These are linked to the description of a universal sector alpha'-corrections to the low-energy limit of compactified string theory used, for example, for precision tests in holography. Time permitting, I will review some older results on how to obtain the two 6D N=(1,0) and single 6D N=(2,0) (six-derivative) conformal supergravity actions that describe type B anomalies of six-dimensional conformal field theories.