The Australian Taxation Office will spend at least $73.4 million to relocate its offices into "the best building in Adelaide", four streets away.