Property Finder, a digital real estate platform operating across the Middle East and North Africa, discovered that 96.8 percent of its evaluation labels for vision AI were inherited directly from one model’s predictions rather than verified by human experts, according to an AWS blog post. The systematically skewed ground truth made every alternative model appear to underperform against the incumbent, blocking objective comparison.
The company processes tens of thousands of property images daily, evaluating 11 quality attributes per image including watermark detection, room classification, furnishing status, photo quality and construction indicators. Property Finder needed a more cost-effective option when migrating its vision workload to Amazon Bedrock, but any alternative model was at a disadvantage when measured against the biased labels.
According to Property Finder and the AWS Generative AI Innovation Center team, the company built an unbiased ground truth dataset and applied iterative prompt optimization to select a new model on Amazon Bedrock. The post states this approach achieved higher accuracy at 86 percent lower cost, though the exact accuracy figure and comparison basis are not specified.
The team used a 256-image expert-annotated development set to score predictions with Micro F1 metrics. The optimization loop ran through 27 prompt versions, with error analysis inspecting failed images to identify visual patterns that informed subsequent prompt revisions. The post describes the methodology as progressing from manual refinement with AI assistance to automated optimization in later versions.
The blog post is co-authored with Khaled Thabit, Meriem Boughrara and Mohamed Desouky from Property Finder.