Mirakle Research · September 4, 2026
The reviews Shopify didn't put back
143,831 reviews left Shopify App Store listings over three days in July, and 82.2% of them came back in September. The companion report establishes that the return was selective, and ends where the selection does: at a boundary it can describe but not explain. 25,592 reviews sit on the far side of it. This page is about their shape, because a boundary is legible from what it leaves out. They are 1.94 times longer, they carry low ratings at 1.73 times the rate, and they concentrate in categories that make up a small corner of the store.
The first cut · length
Whether the length gap is really the empty reviews
Take the widest of those differences first, and try to destroy it. A median of 252 characters against 130 has an obvious dull explanation: Shopify listings carry a lot of reviews with no text at all, and if those came back first the survivors would look wordy by default.
They did come back first — 10.3% of the returns have no text against 4.8% of the remainder. So we cut the removals into length bands and measured the kept-off rate inside each.
It is not a gradient. It is a step. A review with no text at all, one under sixty characters, and one up to a hundred and fifty all stay off at about 9.3%. Cross a hundred and fifty characters and the rate is 24.4%; past four hundred it is 33.8%. Whatever separated these two piles did not care how short a short review was, and cared a great deal once someone had written a paragraph.
A step is a more particular object than a slope. A slope would say the two piles differ in degree; a step says there is a place where the behaviour changes, and puts that place at about a hundred and fifty characters — two sentences. We cannot see what made the line fall there. We can say that it fell somewhere, which is more than we knew when the difference was only a pair of medians.
The gift-card listing in the companion report is one review's worth of this. Its longest review by a wide margin — 1,090 characters, a merchant of two years describing money lost to fraudulent orders — was the first thing to go and among the last to return. Everything else it lost was short praise, and the short praise was back within the week.
The second cut · rating
The gradient runs down the whole scale
Cut the same pile a second way and it separates again. 27.9% of the removed one-star reviews are still off a listing, against 17.0% of the five-star ones — 1.64 times the rate. Every step of the scale sits between them in order.
In absolute terms this is still mostly a five-star story, because the removal was: 143,831 reviews went and the overwhelming majority of them were five stars. The gradient is about which of them came back, not about who was targeted.
Two cuts now, and they are not independent of each other. Unhappy people write at greater length, so a rule acting on length would produce a rating gradient for free, and a rule acting on rating would produce a length step for free. Nothing so far says which of the two we are looking at, or whether both are the shadow of some third thing. So a third cut, chosen because it is not made of the review's text at all.
The third cut · category
Two corners of the store carry most of it
Across 68 categories with at least 300 removals, the kept-off rate runs from 49.4% in Dropshipping down to 5.9% in Workflow automation. Print on demand and Dropshipping sit at the top, and together they hold 3,829 of the remainder at 49.4% — against 16.0% across every other category.
The reviews those two categories kept off are also the angriest in the remainder: 14.99% are one or two stars, against 7.66% elsewhere, with a median length of 318 characters.
Which folds the three cuts into one. The reviews these two categories kept off are both longer than the remainder and angrier than it, so length, rating and category are not three findings but one population described three times. That is a smaller claim than three would have been and a sturdier one, because the same group answers to every instrument we have pointed at it. Why these two categories and not others is not something our data can answer.
The fourth cut · the words
What the remainder says
Three cuts made from what we know about a review from the outside: how long, how many stars, which shelf. There is a fourth available, made from the inside, and it should agree with them if the population is really one thing. Counting words as substrings of the review text, and reporting rates per 1,000 reviews in each half, dropshipping appears 13.36 times more often in what stayed off than in what came back — 36.73 mentions per 1,000 against a far smaller rate on the other side. aliexpress follows at 8.03 times.
The word list was fixed before the rates were looked at, and support is in it as a check: it is the most common word in Shopify review text generally, and it comes out almost flat. A word that appears everywhere should show no lift, and it doesn't.
So the fourth cut agrees with the other three, and it agrees in a specific way: the vocabulary that separates the two halves is the vocabulary of sourcing, of suppliers and shipping and refunds, which is also the subject matter of the two categories the third cut found. Four instruments, one population. What none of the four supplies is the reason it was treated differently, and the words are the closest we get: they say what these reviews are about, not what was done with them.
Who carries it
How much each app is still missing
A population described four times is still an abstraction, and the reviews in it are attached to particular listings. 3,913 apps are still missing something. The median one is missing 2 reviews, which for a large listing is invisible. 1,527 of them are missing a tenth or more of their whole shelf, and the worst is missing 100% of it.
Small listings wear it hardest, which is the same shape the removal had and the same shape the return had: 32.5% of removals from apps holding under 10 reviews stayed off, against 12.4% for apps holding 5,000+. Three events in a row have sorted the store the same way, which is what makes the last one worth stating plainly: the cost of all of this fell hardest on the listings least able to absorb it, and it did so without anyone choosing that.
What survived
What the evidence supports
The remainder of a removal that has otherwise been reversed is longer, more critical, newer, and concentrated in two categories. Those are four descriptions of one group rather than four independent facts about it. Each holds up inside the controls we could run.
That is where the chain stops, and it stops short of a mechanism. A removal Shopify announced on 6 July is still 67.7% off the shelf, so deliberate removals do persist and some of this remainder may be that. Reviews attached to stores that have since closed also drop off listings on their own. Both would concentrate in the same categories for reasons that have nothing to do with what the review said. We can rank the possibilities; we cannot choose between them from a public listing page.
It is worth being exact about what that leaves. We have a boundary measured four ways and no rule behind it, but a boundary measured four ways is not nothing, because whatever drew it had to be consistent enough to draw the same line each time we looked. The statistics are not standing in for a cause we failed to find. They are the only form in which this particular cause is visible from outside.
For a small developer the size of this is not abstract. A listing with twenty reviews that is still four short, seven weeks after a correction that gave most developers theirs back, is carrying a visible cost from something nobody told them about.
Method
How this was measured
The population is every review our scraper recorded as gone from a listing between 16 and 18 July 2026, split by whether it is on a listing today. "Kept off" is a statement about the present, not about intent: we observe that a review has not returned, and nothing about why. Every marker is computed identically over both halves of that same removal, so a difference between them cannot be category mix, app mix or season. Categories are the app's primary category as Shopify lists it, reported only where a category's apps lost at least 300 reviews. The lexicon counts a word as a substring of the lowercased review text and reports its rate per 1,000 reviews in each half; no review text leaves the database. The removals Shopify announced on 6 July are carried as a second control. The returns ran from 3 to 7 September and were over when this ran, so the remainder measured here is what settled rather than a figure still in motion.
Limits. This is a measurement of listings, not of moderation. We cannot see a decision, an appeal, or a store closing. The category finding is a rate over each category's own removals, so it is not driven by category size, but a category is a coarse instrument and apps are only assigned their primary one. The lexicon counts substrings, so never matches inside other words; it is a rough instrument used for ranking, not for counting. Returns were only written to our change log from 4 August, so the population here is defined by the state a review is in today rather than by log entries.
Every number comes from the snapshot behind this page, published as raw JSON. The earlier reports are the week of 13 July, what came back in August, and the September returns.