Professor Chris Green kindly posted a response to my blog post about Congress's alleged power to define what "privileges" and "immunities" are covered by the Fourteenth Amendment. My main point was that Congress generally has no definitional power under the Privileges or Immunities Clause with regard to either enumerated or unenumerated rights; Congress simply has no role with regard to either. Perhaps Professor Green believes Congress has such definitional power with respect to unenumerated rights, but if so then I respectfully disagree.
Professor Green is entirely correct that it's very important to ask whether the Privileges or Immunities Clause involves unenumerated rights, and I am happy to engage on that point, and to argue that it does not. It seems to me that none of the five examples that Professor Green has given include the key phrase “privileges [and/or] immunities of citizens of the United States.” The closest of those five examples (by far) is Green's lead example, where Congressman John Bingham said something very similar. That remark by Bingham in 1867 (which incidentally was not widely published like President Andrew Johnson's veto message was in 1866) referred not merely to rights related to prevalent rates of taxation, but rather referred explicitly to the Comity Clause right of equal treatment which is emphatically an enumerated rather than unenumerated right.
No one disputes that various statesmen in 1866 believed United States citizenship implied various unenumerated natural rights, but whether they believed any of those unenumerated rights were judicially enforceable against Congress is quite another question, as is whether they believed those unenumerated rights were “privileges [and/or] immunities of citizens of the United States.” The phrase "privileges and immunities" typically referred in the Comity Clause not to all natural rights possessed by native citizens but only those rights that native citizens could actually vindicate against their own state; that is one important reason why political leaders of that era could sometimes distinguish “privileges [and/or] immunities of citizens of the United States” from the broader set of rights belonging to U.S. citizens.
UPDATE (3/23): Just to be crystal clear, I am absolutely not drawing any sharp distinction between rights on the one hand versus "privileges and immunities" on the other hand. All I am saying is that the latter are a subset of the former. This has been established Comity Clause doctrine for well over a century.