Well, technically for Gödel's theorem to apply to a formal system it has to satisfy a few properties. One of those is that it has a computable list of axioms. Infinite sets can be computable too - the only requirement is that determining whether an axiom is contained in the set can be performed by an algorithm.

For instance the set of even numbers is infinite but computable, just check whether the number is divisible by 2.

The algorithm itself is finite, even if the set it determines is not.