So, perhaps I was too flowery with my wording here.

Galaxies are unfathomably huge and distant in terms of a human trying to grasp their size as anything relatable, anything other than an abstract number with a huge exponent.

We can and have and still do measure the actual distances to far away galaxies. Likewise with their size, and likewise with atomic and subatomic particles.

We have observed, measured, and calculated that the father away galaxies are from us, the faster they are moving away from us. This concept is generally encapsulated as Hubble’s Law.

but it’s not absurd to say we don’t know much about the nature of the universe

This is in fact absurd to say, unless you take ‘we’ to mean something approximating 5th graders.

Let me try another angle here:

If we, as in, our entire galaxy, and our sense of scale and distance, were for some reason shrinking, and the rest of the universe was static…

Why would we not observe everything outside our galaxy, or solar system, or planet, or whatever the boundary of your proposed ‘shrinking zone’ is… why would we not observe everything outside of that becoming larger?

If this were actually happening, we could very easily measure and observe this… but we don’t.