It's not the best way, but have you tried parent constraining the objects instead, preserving offsets?