The new PS3 will provide backwards compatibility with the PS2 and PSOne by including the PS2’s main processors inside the PS3’s box. Even then some known incompatibilities have already been found. These will likely be ironed out in forthcoming Firmware upgrades.
Sony took this same approach with the PS2 to provide backwards compatibility with the PSOne, by taking the CPU of the original PSOne and using it as the I/O controller inside the PS2. Not only would this processor control all the I/O operations in the machine, but PSOne games could be run directly on this chip to provide backwards compatibility.