Stanford Researchers Develop AI Model PIGEON that Excels in Location Extraction from Google Street View Imagery


Published on:

Researchers at Stanford University have unveiled a breakthrough in deep learning with their creation of PIGEON, an AI model that can extract location information from Google Street View imagery. 

By analyzing photos, PIGEON can accurately identify the general country and infer the location within a 40km radius of where the image was taken. The model’s impressive performance surpasses that of human players in the popular online game GeoGuessr.

Developed as an image geolocation model derived from their pre-trained CLIP model called StreetCLIP, PIGEON relies on a set of semantic geocells. These geocells represent bounded areas of land, accounting for region-specific details like road signs and infrastructure quality. The model recently participated in a DeepMind competition, triumphing over GeoGuessr’s top-ranked player, Trevor Rainbolt.

One key factor contributing to PIGEON’s performance is its reliance on OpenAI’s CLIP. By leveraging CLIP, which provides exposure to a wide range of images, PIGEON can perceive finer details and achieve greater effectiveness compared to geolocation models trained from scratch or based on ImageNet.

To optimize the accuracy of the model, the research team devoted significant effort to refining the semantic geocells. This approach resolved issues where previous models tended to incorrectly identify locations at sea, even when utilizing CLIP. The geocells were scaled proportionally to population density and respected different levels of administrative boundaries.

Additionally, the researchers developed a loss function that minimizes prediction penalties when the predicted geocell is close to the actual location. They also applied a meta-learning algorithm to enhance position prediction within a given geocell, further improving accuracy.

The PIGEON model demonstrates impressive performance, correctly identifying 92% of countries with a median kilometre error of 44km. This achievement translates to a GeoGuessr score of 4,525. Notably, around 40% of PIGEON’s guesses are within a 25km radius of the target, as highlighted in the research paper.

With its remarkable capabilities in location extraction, PIGEON has attracted interest from open-source intelligence platforms intrigued by its geolocation technology. The Stanford researchers foresee broader applications for the model, enabling easier localization of diverse outdoor imagery.

Vishak is a skilled Editor-in-chief at Code and Hack with a passion for AI and coding. He has a deep understanding of the latest trends and advancements in the fields of AI and Coding. He creates engaging and informative content on various topics related to AI, including machine learning, natural language processing, and coding. He stays up to date with the latest news and breakthroughs in these areas and delivers insightful articles and blog posts that help his readers stay informed and engaged.

Related Posts:

Leave a Reply

Please enter your comment!
Please enter your name here