Mirakle Research · September 4, 2026
A Shopify review is not a permanent record
Two of our reports this summer were about three days in July when a great many reviews left Shopify App Store listings, and a third about the week in September when most of them came back. Each time the question was what made those days different. Watching the same listings for 67 days straight puts the question the other way round. In that window 201,591 reviews left a listing and 56,965 came back to one; 57,673 did both, and one moved 20 times. The number printed on a listing is a reading taken at a moment. We had been treating it as a total.
The thing that does not fit
A review can leave and come back
Start with the detail that broke the assumption the rest was built on. Of the 204,902 reviews that moved at all, 57,673 — 28.1% of them — moved more than once. 1,802 moved three times or more, and 442 moved four times or more inside a single two-month window.
We had assumed a deletion was terminal, and built a change log to match: departures recorded carefully, returns not recorded at all until 4 August. The assumption was not a conclusion we reached, which is what made it hard to notice — it was the shape of the table. It cuts one way, and only one: every number on this page is a floor, because a review that left and returned between two of our scans left no trace for us to count.
The reviews that move repeatedly are slightly more critical than the ones that move once — 6.75% are rated one or two stars against 4.36% — and slightly longer, at 137 characters against 127. Both gaps are small. We would not build anything on them.
The check
How much of this is a handful of large apps
A total that large across a store this size could still be a local event: a few very large listings churning while everything else sits still. Quite a lot of it is. The ten busiest apps hold 12.4% of all presence events and the busiest hundred hold 40.1%, across 7,696 apps in total. A large listing has more reviews to lose, so some of that is arithmetic rather than behaviour.
Measured against each app's own shelf the picture changes. 7,696 apps — 61.4% of every app we hold a review for — saw presence churn, and the median one among them saw 33.3% of its shelf move. 2,591 apps saw events equal to half their shelf or more.
Take both incidents out and that app-level picture collapses: on ordinary days the median affected app sees 0.0% of its shelf move. So the two measurements answer different questions and neither cancels the other. Corpus-wide the churn is continuous; at the scale of one listing it is rare. Both of those are true and the second one is the limit on the first.
The other direction
New to us is not new
Reviews leaving a listing is only half of what a presence log records, and the other half is stranger. 139,069 reviews were recorded by our scraper for the first time in the window. 39.4% of them were already more than a year old, and the oldest was posted in 2013. The median arrival was 77 days old.
A listing does not accumulate reviews in one direction. Old ones surface on it, having been somewhere we could not see, and no public signal distinguishes a review posted yesterday from one that has just reappeared after three years. The consequence is not that our count is too low. It is that arrival and authorship are different events, and a listing only ever shows the first.
The rest of it
The other changes
Presence is the loudest thing that happens to a review and the easiest to count, which is why it has taken three reports to get here. It is not the only one. In the same window, reviews gained developer replies, had their text edited by the person who wrote them, had their star rating changed, and came back out of Shopify's archive. Each is a listing changing under a reader who has no way of knowing it changed, and each is a smaller version of the same fact: the page is a current state, and it is presented as a history.
The gift-card listing the first report follows spent forty-four days displaying a rating two tenths of a star above the one it holds now, on a fifth fewer reviews. Nothing on the page marked those days as different from any other, and every shopper who read it during them read a current state believing it was a history.
What survived
What this changes about reading a review count
A review count is a measurement of a page on a day. Over two months 21.5% of the reviews we held changed presence at least once, and even with both incidents removed the corpus loses 739 reviews a day and regains 118.
For most individual listings, on most days, nothing happens. The churn is real at the scale of the store and rare at the scale of a page. Both of those matter, to different people: it means a developer will usually see a stable count, and it means anyone comparing two snapshots of the whole store a month apart is comparing two different populations without being told.
Our own reporting is part of what this corrects, and the correction runs backwards through all three pages. We read a drop in a review count as an event; one of them was an event, and the other has since largely undone itself. What let us tell the two apart was a balanced panel and a second parser reading a different page — neither of which is available to a developer looking at their own listing, which is the whole difficulty.
There is no mechanism at the end of this. We can say that the corpus moves, how fast, in which direction and to whom; we cannot say what moves it, because the only surface we can read is the one that has already been changed. That is not a gap in the evidence with a fact missing from it. It is the form the evidence takes when the thing you are studying is a page that other people rewrite: the statistics are not a proxy for the cause, they are the cause's only visible signature, and three reports is what it took to learn to read it.
Method
How this was measured
The window is 5 July 2026 to extraction. A presence change is a review we had recorded on a listing that a later completed scan did not find, or the reverse; both come from our own change log, and a review can contribute more than one. Two Shopify-side incidents fall inside the window — the removals of 16-18 July and the returns of 3-7 September, each the subject of its own report — and every headline figure is given with and without them, because they are large enough to carry a conclusion on their own. Returns were only written to the change log from 4 August, so return counts before that date are understated and the true churn is higher than what is reported here. The denominator for corpus shares is the reviews our scraper had already recorded before the window opened. Apps are counted only where we hold at least one unarchived review.
Limits. Everything here is our observation of a public page, not of Shopify's systems. A review that leaves a listing may have been removed, or its store may have closed, or it may have been hidden for a reason we cannot see; we do not distinguish these and no number here should be read as a count of moderation decisions. Our own scanning is the other limit: an app is scanned every few days rather than continuously, so a review that left and returned between two scans is invisible to us, and the churn reported here is a floor. Returns were not logged before 4 August, which pushes the same way.
Every number comes from the snapshot behind this page, published as raw JSON. The two incident reports are the September returns and what stayed off.