The set-theoretical argument for Goodstein’s Theorem meets William Gasarch’s criteria  of an argument that prima facie defies belief.
We shall show that the disbelief is justified, since Goodstein’s argument can be carried out completely over the structure of the natural numbers—without appealing to any properties of transfinite ordinal sequences—to yield the following:
Theorem 1: If is a Goodstein Sequence over the natural numbers then, for any natural number , the first terms of the Goodstein Functional Sequence formed by replacing the base syntactically in the hereditary representation of each term of by form a monotonically decreasing sequence of natural numbers.
Proof: Follows immediately from (iii) below.
Now, we shall argue that since we can show that the first order Peano Arithmetic PA cannot admit a model that contains a non-successor element such that, for any given natural number , we can have , we cannot conclude from the above argument that every Goodstein sequence over the natural numbers must terminate finitely.
However Goodstein’s argument is that since any model of a first order Ordinal Arithmetic would admit a non-successor element such that, for any given finite ordinal , we have , we can conclude from the above argument—when interpreted in the Arithmetic—that every Goodstein sequence over the finite ordinals must terminate finitely.
Moreover, since we can set up a 1-1 correspondence between the natural numbers and the finite ordinals, Goodstein concludes that every Goodstein sequence over the natural numbers must also terminate finitely.
We shall argue that Goodstein’s argument is a curious case of proving a Theorem involving the set-theoretical membership-based relation `‘ over the structure of the ordinals below and—ignoring Thoraf Skolem’s cautionary remarks  about unrestrictedly corresponding putative mathematical relations and entities across domains of different axiom systems—invalidly postulating that a corresponding theorem involving the natural number inequality relation `‘ must therefore hold over the structure of the natural numbers.
Some properties of Goodstein’s sequence over the natural numbers
We note that, for any natural number , R. L. Goodstein  uses the properties of the hereditary representation of to construct a sequence of natural numbers by an unusual, but valid, algorithm.
Hereditary representation: The representation of a number as a sum of powers of a base , followed by expression of each of the exponents as a sum of powers of , etc., until the process stops.
For example, we may express the hereditary representations of in base and base as follows:
For the moment we shall ignore the peculiar manner of constructing the individual members of the Goodstein sequence, since these are not germane to understanding the essence of Goodstein’s argument. We need simply accept for now that is well-defined over the structure of the natural numbers, and has the following properties:
(i) For any given natural number we can construct a hereditary representation—denoted by —of the term of the Goodstein sequence in the base ;
Example: The hereditary representations of the first two terms of are :
(ii) We can define a Goodstein Functional Sequence  over by replacing the base in with the variable for each ;
(iii) We can show that some member of Goodstein’s sequence terminates finitely (i.e., evaluates to ) if, and only if, there is some natural number such that for any given natural number :
If in , then .
Terminate finitely: By Goodstein’s algorithm, after a all subsequent members of the sequence necessarily remain , and the sequence is said in such a case to terminate finitely at its first value.
We shall show that the proof of (iii)—which depends, of course, on the peculiar nature of Goodstein’s algorithm—is merely tedious, but straightforward. The main point to note is that the proof appeals only to the usual properties of the natural numbers.
The question arises: Are we free to postulate the existence of such a natural number , and conclude—as Goodstein’s argument does—that some member of must evaluate to in ?
Though it sounds absurd, the following theorem shows that this is precisely the freedom that the ordinal-based argument for Goodstein’s Theorem claims (albeit implicitly)!
Theorem 2: Goodstein’s ordinal sequence over the finite ordinals terminates with respect to the ordinal inequality `‘ in any sound interpretation of a first order Ordinal Arithmetic even if Goodstein’s natural number sequence over the natural numbers does not terminate with respect to the natural number inequality `‘ in any sound interpretation of PA.
Proof: Assume that Goodstein’s natural number sequence over the natural numbers does not terminate with respect to the natural number inequality `‘ in any sound interpretation of PA.
Let be the largest term amongst the first terms of . It is tedious but straightforward to show that, by our assumption, is a monotonically increasing function of . Hence there is no natural number such that for all .
Consider next Goodstein’s ordinal number sequence over the finite ordinals. There is now the axiomatically postulated transfinite ordinal such that for all .
Since there are no infinite descending sequences of ordinals under the ordinal inequality `‘, Goodstein’s ordinal number sequence must terminate finitely with respect to the ordinal inequality `‘ in any sound interpretation of a first order Ordinal Arithmetic.
Since the finite ordinals can be meta-mathematically put into a 1-1 correspondence with the natural numbers, it follows that:
Corollary: The relationship of `terminating finitely’ with respect to the ordinal inequality `‘ over an infinite set of ordinals containing a transfinite ordinal in any sound interpretation of a first order Ordinal Arithmetic cannot be corresponded to the relationship of `terminating finitely’ with respect to the natural number inequality `‘ over an infinite set of natural numbers in any sound interpretation of PA.
Return to 1: William Gasarch. 2010. Theorems that you simply don’t believe. Computational Complexity blog, Thursday, March 11, 2010.
Return to 2: Thoralf Skolem. 1922. Some remarks on axiomatized set theory. Text of an address delivered in Helsinki before the Fifth Congress of Scandinavian Mathematicians, 4-7 August 1922. In Jean van Heijenoort. 1967. Ed. From Frege to Gödel: A source book in Mathematical Logic, 1878 – 1931. Harvard University Press, Cambridge, Massachusetts.
Return to 3: R. L. Goodstein. 1944. On the Restricted Ordinal Theorem. Journal of Symbolic Logic 9, 33-41.
Return to 4: Strictly speaking the denotation should be: .
Return to 5: For ease of expression we express `‘ as `‘, and `‘ as `‘ unless indicated to the contrary.
Return to 6: We prefer this notation to that of the usual `base bumping’ function as it makes the argument more transparent.