Epyonwing is correct. To get a cape, "the physical thing on your back" as you put it, Your guild leader must take 2000 gold (2 platinum) to a guild registrar (pre-searing) or guild emblemer (post-searing). Then your guild leader will be able to design a cape to appear on all guild members.