Perth's Duplicate Image Problem: The Numbers Exposing a Digital Record-Keeping Crisis

Perth's rapid expansion — Metronet construction corridors, AUKUS-linked defence infrastructure around Henderson and Garden Island, and a housing approvals pipeline running at near-record pace — has generated an avalanche of digital documentation. Buried inside that avalanche is a problem that IT managers across the city have quietly flagged for two years: duplicate images are consuming storage budgets, slowing project workflows and muddying official records. The scale of the waste, when measured, is striking.

A review of publicly available procurement and ICT strategy documents from several Australian state governments shows that duplicate or near-duplicate image files can account for between 20 and 40 per cent of total unstructured digital storage in large agencies. For a mid-sized Perth organisation managing ongoing construction photography, planning submissions and aerial survey data, that translates directly into unnecessary expenditure on both hardware and cloud licences.

Why It Matters Here, Right Now

Perth is not an abstract case study. The City of Stirling — which covers suburbs from Scarborough to Dianella and sits adjacent to HMAS Stirling on Garden Island — has been processing a significantly higher volume of planning and development imagery since 2024, driven by infill housing targets set under the state government's Housing and Homelessness Action Plan. The City of Swan, covering the Ellenbrook corridor where Metronet's extension is under construction, faces similar data pressure. Both councils rely on document management systems that were not designed with automated deduplication in mind.

When duplicate images proliferate across shared drives or content management platforms, the downstream consequences are concrete. Staff retrieve the wrong version of a site photograph. Heritage consultants working on properties in Fremantle's West End submit assessments based on outdated aerial captures. Engineers cross-referencing as-built documentation for the Thornlie-Cockburn Link find multiple near-identical files with different timestamps and no clear version hierarchy. Each scenario adds time — and time, on a government hourly labour rate that typically sits above $80 for specialist roles, costs money.

The Western Australian Government's own Digital Strategy, updated in 2023, explicitly identifies data quality and redundancy as a priority risk area for agencies handling capital works documentation. The Department of Finance, which oversees whole-of-government ICT procurement from its offices on the corner of Havelock and Hay Streets in the CBD, has encouraged agencies to assess deduplication tooling as part of broader cloud migration programs. Progress has been uneven.

What the Data Actually Shows

Storage is cheap until it isn't. Enterprise-grade cloud storage in Australian data centres — used by most WA government agencies under the whole-of-government Microsoft and AWS panel arrangements — runs at roughly $25 to $35 per terabyte per month at the lower consumption tiers, rising steeply once agencies cross contracted thresholds. An agency holding 10 terabytes of image data where 30 per cent is redundant is paying for approximately 3 unnecessary terabytes every month. Over a financial year, that is between $900 and $1,260 in storage alone, before factoring in backup replication, egress fees and the labour cost of manual file management.

Scale that figure across a large department running multiple concurrent capital projects — say, Main Roads WA coordinating works along Leach Highway, Tonkin Highway and the Stephenson Avenue extension simultaneously — and the numbers compound quickly. Industry benchmarks cited in Gartner research on unstructured data management suggest organisations that deploy automated deduplication tools reduce image storage footprints by an average of 25 per cent within 12 months of implementation.

The technology to fix the problem is not exotic. Perceptual hashing tools, which identify visually similar images even when file metadata differs, have been commercially available since the early 2010s. Several are available under open-source licences. The barrier is not cost — it is the absence of a mandated policy and a clear agency champion to push implementation through procurement committees.

For Perth organisations navigating this now, the practical path forward starts with a storage audit. Identify which shared drives or content platforms hold the largest image repositories, run a deduplication analysis tool against a sample dataset, and quantify the redundancy rate before taking any findings to a budget committee. The State Records Office of Western Australia, based in Alexander Library Building on Francis Street, publishes guidance on digital asset management that can frame the business case. The numbers, once properly counted, tend to make the argument themselves.