Good question, messy answer.
Within the space of one opinion, the Supreme Court managed to say all this:
A pardon reaches both the punishment prescribed for the offence and the guilt of the offender, and when the pardon is full, it releases the punishment and blots out of existence the guilt, so that, in the eye of the law, the offender is as innocent as if he had never committed the offence. If granted before conviction, it prevents any of the penalties and disabilities consequent upon conviction from attaching; if granted after conviction, it removes the penalties and disabilities and restores him to all his civil rights; it makes him, as it were, a new man, and gives him a new credit and capacity.
There is only this limitation to its operation: it does not restore offices forfeited or property or interests vested in others in consequence of the conviction and judgment.
That was in 1866.
In a couple of decisions shortly afterwards, the Supreme Court came down on both sides of the issue, according to which part of the government controlled the forfeited money at the time of the decision (this distinction being a direct consequence of the “vested in others” language that’s bolded above).
More recent opinions in Federal court have been … mixed. What’s safe to say is that the facts of each case can make a real difference.