That is exactly what the scaling the 3D texture node does so I don't think there is a way to stop that from happening.