We examine ultrafast nonlinear birefringence of femtosecond optical pulses propagating through a semiconductor optical amplifier whose active element is composed of InAs/InP quantum dots at about 1.55 μm. The polarization- sensitive carrier dynamics in quantum dots are modeled for nonlinear gain bleaching that is employed for nonlinear birefringence for all-optical- wavelength converters. It is revealed that quantum dots support ultrafast nonlinear gain bleaching for ultrafast birefringence used in ultrafast all-optical-wavelength converters. The pump-pulse energy is one of the key ingredients for a well-shaped converted signal. In addition, nonlinear gain and index dynamics are discussed for ultra-high-speed operation with reduced pattern-effects.