An experimental computer software "erases" human pedestrians in images from Google Street View, replacing them with background buildings, foliage and sidewalks. The technique could help preserve privacy in public environments in the digital age.
The as-yet unnamed system removes pedestrians from urban scenes pulled from Google Street View, which provides panoramic views of cities, towns and rural areas across the world.
Street views are constructed by stitching together overlapping images taken from a moving vehicle.
Google Street View currently blurs faces and license plates from its images. Nevertheless, clothes, body shape, and height combined with geographical location can be enough to make some pedestrians personally identifiable even if the face is blurred out, according to the eraser system's creators.
The pedestrian removal is relatively "ghost free" — meaning that the artifacts caused by the pixel swapping are usually not distracting. But the pedestrian remover does occasionally produce strange results — like dogs on leashes with no owners, and shoes with feet but nothing else.
The computer vision system replaces holes in the images with an approximation of the actual background behind each pedestrian. These corresponding background pixels are pulled from the image taken right before or right after the image in question.
Computer science graduate student Arturo Flores from the University of California, San Diego (UCSD) developed this proof-of-concept system. Flores and UCSD computer science professor Serge Belongie presented the work in June 2010 at the IEEE International Workshop on Mobile Vision.
One next step, according to Flores, is to remove groups of pedestrians from single images.
In addition, the system struggles to generate background pixels when the pedestrian happens to be walking in the same direction as the vehicle at just the right speed. In these cases, the pedestrian may cover up the same spot in multiple frames, foiling the computer scientists’ pixel-swapping approach to removing pedestrians.
The pedestrian remover only works in urban settings so far – where the pixels blocked by people are often "on a dominant planar surface" – which makes them simpler to replace.
The system, for example, can replace the pixels blocked by a person walking by a mural of horses grazing in a pasture. But the system cannot replace the pixels behind a person on a country road walking by actual horses grazing in a pasture, because this background is not predominately flat.
"This is a cute idea that, as far as we know, has not been explored," said Belongie.