Photoacoustic imaging using exogenous contrast agents has emerged as a hybrid technique that enables the deep imaging of optical properties of tissues with high spatial resolution. The power of this imaging technique can be greatly enhanced by the use of contrast agents that absorb at near-infrared wavelengths and whose optical properties can be modulated in response to the local environment. We have designed contrast agents consisting of gold nanoparticles coated with anisotropic silica nanoshells. The tunable aggregation of these Janus particles in cell culture media resulted in a dramatic amplification of photoacoustic signals in the near-infrared region. We also demonstrated imaging using these contrast agents in mammalian cells, including macrophages and breast cancer cells as well as in vivo. The ability to modulate Janus particle aggregation in response to a range of stimuli in combination with the high resolution and deep penetration of multiwavelength photoacoustic imaging are attractive for a broad range of applications in diagnostic imaging and theranostics.