Mirakle Research · September 21, 2026
Which is worth more reviews: a free plan, a free trial, or a lower price?
A Shopify app listing can be made cheaper in three ways: drop the price, add a plan that stays free, or lengthen the free trial. Our pricing log caught all three between 30 April and 21 September. Following each app's reviews for 8 weeks before and 5 after, against 5 apps in the same category that changed nothing, the three come apart. Taking a free plan away is followed by about a third fewer reviews. Moving the trial — a median of 2 days to 14 — we cannot tell from nothing at all.
The first suspect
What our pricing log records
Every day our scraper reads each app's page and compares the pricing section with what it read last time. When anything differs it writes a row. Over 144 days it wrote 14,080 rows about 7,526 apps, and most of them are not a price change.
| Rows in the pricing log | 14,080 |
| No plan list in the row: only the trial, billing or pricing-model text changed | −1,410 |
| Same plans, names, prices and trials: the feature bullets changed, or a plan card was read twice | −4,906 |
| The plan list came back empty on one side, which is a failed read, not a free app | −1,058 |
| What remains, rebuilt as each app's sequence of distinct plan lists; 196 changes undone within 14 days, and 37 that only reordered plans, set aside | |
| Changes to a plan list | 6,557 |
The sequence is shorter than the rows because a failed read or a duplicated card writes two rows, one going wrong and one coming back, and neither is a change. The second thing to rule out is the scraper changing how it reads the page. A parser change would show up as a burst on the day it was deployed.
Apart from that week the rows arrive at a steady few hundred a week, and the brick share does not jump anywhere. We also checked how late a change can be. Across the 4,554 rows for which we have kept every reading of the app's page, a changed plan list was first read a median of 1.0 days after the previous reading, and 2.2 days at the 99th percentile. A change is dated to within a day or two of when it appeared.
So the log can be read, and the changes can be dated. What the log is a log of turned out to be the harder question.
What we checked next
What a price change turns out to be
We sorted each change by comparing the two plan lists, and the sort is the reason this report is not about price. The largest group by far, 1,699 changes, added or dropped whole plans so that the cheapest or dearest price on the listing moved without any single plan being repriced. From the list alone we cannot say whether a merchant already on a plan paid more, less or the same, so those are set aside. Of what is left, a named plan's price went up 942 times and down 1,090.
Where a plan is repriced the step is large. The median raised plan went from $19.80 to $29.99 a month, and the median cut plan from $30 to $19.
A listing's free offer is not like that. It has two forms and both are written on the page in plain words. A free plan is a tier priced “Free” that a merchant can sit on indefinitely. A free trial is a number of days — Shopify prints it as “14-day free trial” — after which the merchant pays. Either is there or it is not, and when it moves we can say which way without knowing anything about what a particular merchant is charged.
The free plan
46.9% of the 22,779 listed apps we can read a plan list for offer one today. In our window 868 changes added a free plan and 472 took one away.
The free trial
61.9% offer one, at a median of 7 days. 458 changes added or lengthened a trial and 318 shortened or removed one, every plan's price staying exactly as it was.
15.9% of listings offer neither, and ask for money on the first day.
That is four movements, every one of them legible from the page alone: a free plan on or off, a trial longer or shorter. With the two directions a named plan’s price can move, six levers. How much there is of each decides what can be said about any of them, so that comes first.
How much there is
What each lever is measured on
Each app is counted once per lever, at its first clean change of that kind, and only if its five weeks after end by 15 September and its matching baseline starts inside 2026. That is most of the loss between a change and an app we can follow: the log runs for five months and a change in the last five weeks of it has no after.
Altogether 36,050 reviews at the changed apps and 23,558 at their controls fall inside the windows. They are not spread evenly, and the intervals later on are wide or narrow in proportion.
Those are the six groups. Before reading any of them we wanted to know what this comparison returns when nothing has happened.
The second suspect
A change that should have shown nothing
Each app that moved a lever is paired with 5 apps from the same category that had a similar number of reviews in the two months before the study window and changed nothing about their pricing within 60 days of the date. We count every review by the day it was posted, including reviews later removed from the listing, so the July removals and the September returns change which reviews are visible but not when they were written. The estimate is reviews per week after divided by before at the changed apps, divided by the same ratio at their controls. One means they kept pace.
Before reading any of it we ran the same machinery on 367 apps that only renamed their plans — same prices, same trials, same free plans, different words. That should come out at one. It came out at ×1.37, with an interval of ×0.75 to ×2.59.
The fault is in the estimate, not the renames. Reviews per week is a sum, so a group's number is mostly the behaviour of its biggest apps: in the renamed group one app's last week carries the whole result. Dropping the 10 apps with the most reviews in the window from every group is a check we had already written, and on the renamed apps it gives ×0.97 (×0.68 to ×1.35). We did not decide in advance which of the two to read. The renamed apps decided it, and the rest of this page reads the one they pass, with both drawn wherever they differ.
That is the licence for everything below. A multiple quoted from here on is one that a group of apps which changed nothing a merchant pays returns at one.
What survived
How many reviews arrived
Each panel below has four points: reviews per app per week at the two ends of the window, for the apps that moved the lever and for their matched apps. The axis is logarithmic, so two parallel lines mean the two groups moved by the same proportion and the change made no difference. Only the gap that opens at the right-hand end is the estimate.
After a free plan was removed the lines come apart: ×0.68 over the five weeks (×0.49 to ×0.95, 288 apps), and ×0.77 with the largest apps left in. It is the one result here that holds either way it is counted.
A price cut runs the other way, ×1.29 (×1.04 to ×1.64, 636 apps). Adding a free plan points the same way at ×1.30, on more apps (551) but with an interval that reaches back past one (×0.92 to ×1.76).
Nothing else separates. Raising a plan's price gives ×0.98 (×0.85 to ×1.15) across 567 apps, which is the flattest line on the page. Lengthening a free trial gives ×1.19 (×0.87 to ×1.64) and shortening one ×0.81 (×0.53 to ×1.17).
Read one at a time, four of the six say nothing. Read together they say something small. Every one of the six moved reviews in the direction a merchant's wallet would predict — cheaper up, dearer down — 6 times out of 6, on both estimates. If the six were coins, that is one arrangement in 64. It is not six findings. It is one finding measured six times, which is a smaller claim than six would have been and a sturdier one, and it is what the two trial groups contribute: not an effect of their own, but a sign in the right place.
That is not the same as the trial doing nothing, and the distinction matters here more than anywhere else on the page. With 305 and 207 apps the trial intervals are wide enough to contain an effect the size of the free plan's as comfortably as they contain no effect at all. What we can say is that the free plan separates from its controls on this data and the trial does not, measured the same way, in the same weeks, on apps of the same kind. What we would need to say more is another year of log.
The free-plan removals have a second oddity we cannot explain. The decline starts in the week before we read the new plan list, and a detection lag of a day or two does not reach that far. A developer may change the plan inside the app before the listing, or stop pointing new merchants at the free tier first. It could also be that apps already losing merchants are the ones that stop giving anything away.
A review is written by a merchant who installed the app, so a change in how many reviews arrive is most simply a change in how many merchants arrive. That is a guess about installs we do not see; Shopify does not publish them.
How many is one of the two things a review can move. The other is what it says.
The other outcome
What the reviews said
We expected the tone to move with the money, most of all when a free plan was taken away. It did not move by more than this data can resolve. Across the 829 apps that raised a price or removed a free plan, the share of one- and two-star reviews went from 3.04% to 3.35%, while at their controls it went from 4.39% to 4.3%: a difference of +0.4 points, interval −1.0 to +1.8. For the free-plan removals on their own it is −0.7 points and the interval runs from −3.9 to +3.3, which rules out very little.
One interval on the right of the forest does not cross zero. After a free plan was added, the share of one- and two-star reviews fell by 2.0 points against the controls (−3.6 to −0.4), and the average rating of new reviews went from 4.853 to 4.918. There are fourteen intervals on that chart and one landing clear of zero is roughly what chance alone would give, so it is a thing to look at again with more data rather than a finding.
Reputation shocks started from apps whose rating crashed and found most of the crashes were about money. Starting from the price changes instead, the crash is rare. Of the 567 apps that raised a plan's price, 5 (0.88%) received 3 or more one- and two-star reviews in the five weeks after, against 0.78% of their controls. Both can be true: a crash can usually be about money and a price rise can still rarely end in one.
Method and limits
What the data can and cannot say
We see the plan list on the listing, not what any merchant pays. We do not see installs, uninstalls, revenue, grandfathered plans, discounts, or plan changes inside the app. The pricing log begins on 30 April, so it covers one spring and summer. Reviews are counted to 15 September, a week before the snapshot, so that recent ones have been scraped.
Five weeks is short. A merchant on a yearly plan, or one who meets the new price at the next invoice, may write later than this window reaches, and a merchant who leaves because the free tier went away may not write at all. Everything here is about reviews arriving, which is the only trace of a merchant that we hold.
We never see a merchant decide against an app whose free tier has gone. Nothing the App Store publishes would show it, and the one group where the drop begins before we read the change is the group where that matters most. What we have instead is six levers, pulled by 4,387 developers over five months, and the direction the reviews moved each time. Those directions are not standing in for a mechanism we failed to find. They are the form the mechanism takes when it is looked at from outside the app.
Cleaning the log
A row is read as the plan list it records: each plan's name, price text and trial text, with duplicated plan cards collapsed. Rows without a plan list (1,410), rows whose two lists are identical once feature bullets are ignored (4,906), and rows with an empty list on either side (1,058) are not changes. The remaining readings are rebuilt into each app's run of distinct non-empty plan lists. A change followed by a change back to the earlier list within 14 days is a flicker (196).
Prices are compared as US dollars a month: “$X/month” as written, “$X/year” divided by twelve. Usage-based, one-time and “Free to install” plans carry no monthly price. A plan is repriced when the same plan name carries a different monthly price in the next list. A change that removes the last plan priced “Free” is a free-plan removal, whatever else changed, and one that adds one is a free-plan addition. 80 changes raised one plan and cut another and are not studied.
A listing's free trial is the longest “N-day free trial” written on any of its plan cards, and 0 if there is none. A change counts as a trial change only when every plan's price is unchanged and the free plan has not come or gone, so the trial groups carry no price movement with them. Adding a trial and lengthening one are pooled, as are removing and shortening; separately the four are too small to tell from each other.
The comparison
One unit per app and lever: the app's first clean change of that kind whose five weeks after end by 15 September and whose matching baseline starts in 2026. Controls are the 5 listed apps in the same primary category whose review count in days −120 to −56 is closest on a log scale, excluding any app with a clean pricing change within 60 days of the date. Ties break on a fixed hash. The baseline ends before the eight weeks shown, so the flat pre-period in the panels is not forced by the matching.
The volume multiple is (changed apps' reviews per week after ÷ before) ÷ (controls' reviews per week after ÷ before), pooled over apps. The share shift is (after − before) at the changed apps minus (after − before) at the controls. Intervals are the 2.5th and 97.5th percentiles of 2,000 resamples of the units with replacement; the weekly bands use 1,000. The largest-apps variant drops the 10 changed apps with the most reviews in the window and is the one quoted in the text, because it is the variant on which the renamed-plan group comes out at one.
Shopify prints a review count on each listing, which we snapshot daily and which would be an independent count of arrivals. Over these windows it moves by single digits per app, and the July removals and the September returns move it by more, so it could neither confirm nor contradict the volume estimates. These results rest on our own review scrape.
An earlier version of this report, published on 21 September 2026, asked what happens after a price change and quoted the all-apps estimates. It did not have the renamed-plan group, and the free trial was in it only as a single bucket of “trial wording changed”.
Download the frozen JSON Does paying more buy a better app? Reputation shocks