{"slug":"2026-08-01-the-great-purge","title":"The week our scraper thought Shopify deleted 148,172 reviews","generated_at":"2026-08-06T04:23:03Z","source":{"environment":"production","note":"Two measurements of the same thing. The first is ours: a review we had recorded on a listing which a later completed re-scan could not find. That is an inference about absence and it is only as good as the listing's own identifiers. The second is Shopify's: the review count printed on each app's listing, which we snapshot daily in app_stat_snapshots and which involves no inference of ours. Every cross-date comparison of Shopify's counts runs over a fixed panel: the 19304 apps carrying a snapshot on every one of the 30 days between 2026-07-01 and 2026-08-03 on which our app scrape covered the whole store. Days it did not (4 of them) are left out of the series entirely rather than drawn short, because a partial scan day understates the sum without meaning anything. Every point on that line is therefore the same apps. Baseline for our own event counts is 2026-05-04 to 2026-07-06; the week of 2026-04-27 is excluded from every baseline because it is our first full pass, which recorded every already-missing review at once. We only see reviews we had already captured, so our counts are lower bounds. No reviewer identities, review text or app names are published."},"metrics":{"recorded_deletions":148172,"recorded_apps":6978,"typical_week":5328,"multiple_of_typical":27.8,"event_deletions":143931,"event_share_of_week":97.1,"event_peak_day":"2026-07-17","event_peak_deletions":73602,"top10_app_share":12.1,"scanned_apps_before":10888,"scanned_apps_peak":10587,"panel_apps":19304,"panel_dates":30,"panel_excluded_dates":4,"panel_before":531764,"panel_trough":484334,"panel_trough_date":"2026-07-17","panel_after":528535,"panel_trough_drop_pct":8.9,"panel_net_change":-3229,"panel_net_pct":-0.6,"before_from":"2026-07-02","before_observed":1869,"before_ledger":-333,"before_removals":13924,"before_gap":2202,"before_gap_share":8.0,"across_observed":-11352,"across_ledger":-64503,"across_removals":93534,"across_gap":53151,"across_gap_share":43.4,"after_observed":8123,"after_ledger":5890,"after_gap":2233,"after_gap_share":14.7,"event_arrivals":42099,"event_arrivals_new_share":6.9,"event_arrivals_mean_age":1009,"normal_arrivals_new_share":83.4,"normal_arrivals_mean_age":59,"normal_daily_arrivals":1185,"event_peak_arrivals":21829,"arrivals_still_live_share":97.5,"distinct_reviews":147738,"back_now":74882,"back_share":50.7,"still_gone":72856,"peak_requests":113178,"peak_failed_requests":6,"peak_rate_limited":1,"peak_failure_pct":0.0,"twin_gone":143831,"twin_arrived":42099,"twin_matches":1206,"twin_share":0.8,"smallest_bucket_pct_lost":28.9,"largest_bucket_pct_lost":12.8,"agreement_apps":10016,"agreement_scraper_delta":-65221,"agreement_shopify_delta":-64633,"agreement_disagreement":588,"agreement_disagreement_pct":0.9,"agreement_pages_before":84449,"agreement_pages_trough":80044,"agreement_pages_drop_pct":5.2,"agreement_reviews_drop_pct":8.2,"perm_apps":22308,"perm_before":770387,"perm_trough":705837,"perm_vanished":64550,"perm_settled":755926,"perm_immediate_back":50089,"perm_step_down":14461,"perm_late_recovery":11892,"perm_net_now":-2569,"perm_shortfall_vs_trend":4223,"perm_control_change":1132,"perm_control_days":13,"perm_apps_that_lost":null,"perm_still_below":null,"perm_share_of_corpus":0.5,"still_gone_event":70196,"still_gone_event_pct":48.8,"typical_week_permanent_loss":3525,"permanent_multiple":19.9,"holdings_apps":6978,"holdings_we_hold":842720,"holdings_shopify_says":842983,"holdings_gap":-263,"holdings_gap_pct":0.0,"case_description":"a small post-purchase upsell app","case_before_reviews":28,"case_before_rating":4.9,"case_trough_reviews":34,"case_trough_rating":4.9,"case_trough_date":"2026-07-16","case_now_reviews":42,"case_now_rating":5.0,"case_rating_jump":0.0,"case_reviews_lost_at_trough":-6,"case_net_reviews":14,"case_net_rating":0.1,"case_zombies":17,"case_zombies_live":16,"case_zombies_age":922,"case_zombies_oldest":"2022-11-21","week_log_entries":148172,"week_reviews":147738,"week_flapped":431,"event_log_entries":143931,"event_reviews":143831,"event_flapped":100,"shelf_before":770387,"shelf_after":770302,"shelf_survived":715088,"shelf_from_event":37310,"shelf_since_event":17904,"shelf_missing":70196,"shelf_replaced_pct":4.8,"kept_off_apps":6192,"kept_off_median_per_app":3,"kept_off_top10_share":8.8,"arrivals_reviewer_overlap":2.5,"reach_lowest_kept_off":36.3,"reach_highest_kept_off":98.6,"reach_highest_bucket":"25+","reach_highest_returned":53,"reach_highest_total":3902,"default_name_keys":237,"default_name_widest":710,"default_name_reviews":8815,"days_since_event":18,"max_daily_since":714,"corpus_rows":1034720,"corpus_live":855685,"corpus_flagged_gone":161585,"corpus_flagged_share":15.6,"shopify_displayed":832838},"charts":{"weekly":[{"week":"2026-04-27","deletions":46123,"still_gone":42800,"apps":1387,"artifact":true,"partial":false,"event":false},{"week":"2026-05-04","deletions":5412,"still_gone":3311,"apps":1262,"artifact":false,"partial":false,"event":false},{"week":"2026-05-11","deletions":4294,"still_gone":2463,"apps":1054,"artifact":false,"partial":false,"event":false},{"week":"2026-05-18","deletions":4970,"still_gone":3171,"apps":1221,"artifact":false,"partial":false,"event":false},{"week":"2026-05-25","deletions":5928,"still_gone":3747,"apps":1245,"artifact":false,"partial":false,"event":false},{"week":"2026-06-01","deletions":5184,"still_gone":3502,"apps":1177,"artifact":false,"partial":false,"event":false},{"week":"2026-06-08","deletions":5243,"still_gone":3508,"apps":1144,"artifact":false,"partial":false,"event":false},{"week":"2026-06-15","deletions":6865,"still_gone":4047,"apps":1374,"artifact":false,"partial":false,"event":false},{"week":"2026-06-22","deletions":10055,"still_gone":3799,"apps":1461,"artifact":false,"partial":false,"event":false},{"week":"2026-06-29","deletions":8329,"still_gone":4181,"apps":1324,"artifact":false,"partial":false,"event":false},{"week":"2026-07-06","deletions":18143,"still_gone":13510,"apps":1998,"artifact":false,"partial":false,"event":false},{"week":"2026-07-13","deletions":148172,"still_gone":72856,"apps":6978,"artifact":false,"partial":false,"event":true},{"week":"2026-07-20","deletions":3220,"still_gone":2296,"apps":853,"artifact":false,"partial":false,"event":false},{"week":"2026-07-27","deletions":3933,"still_gone":3094,"apps":913,"artifact":false,"partial":false,"event":false},{"week":"2026-08-03","deletions":1371,"still_gone":1171,"apps":489,"artifact":false,"partial":true,"event":false}],"daily":[{"date":"2026-07-01","deletions":560,"arrivals":1235,"apps_scanned":10715,"event":false},{"date":"2026-07-02","deletions":1235,"arrivals":1622,"apps_scanned":10757,"event":false},{"date":"2026-07-03","deletions":872,"arrivals":1452,"apps_scanned":10793,"event":false},{"date":"2026-07-04","deletions":816,"arrivals":1086,"apps_scanned":10707,"event":false},{"date":"2026-07-05","deletions":755,"arrivals":807,"apps_scanned":10645,"event":false},{"date":"2026-07-06","deletions":767,"arrivals":822,"apps_scanned":10812,"event":false},{"date":"2026-07-07","deletions":2058,"arrivals":1444,"apps_scanned":10873,"event":false},{"date":"2026-07-08","deletions":3675,"arrivals":1612,"apps_scanned":10969,"event":false},{"date":"2026-07-09","deletions":2823,"arrivals":1530,"apps_scanned":10690,"event":false},{"date":"2026-07-10","deletions":2960,"arrivals":1449,"apps_scanned":8669,"event":false},{"date":"2026-07-11","deletions":3617,"arrivals":2872,"apps_scanned":10592,"event":false},{"date":"2026-07-12","deletions":2243,"arrivals":3423,"apps_scanned":10749,"event":false},{"date":"2026-07-13","deletions":897,"arrivals":1488,"apps_scanned":10720,"event":false},{"date":"2026-07-14","deletions":959,"arrivals":1302,"apps_scanned":10736,"event":false},{"date":"2026-07-15","deletions":2017,"arrivals":1420,"apps_scanned":10888,"event":false},{"date":"2026-07-16","deletions":24722,"arrivals":7448,"apps_scanned":10602,"event":true},{"date":"2026-07-17","deletions":73602,"arrivals":21829,"apps_scanned":10587,"event":true},{"date":"2026-07-18","deletions":45607,"arrivals":12822,"apps_scanned":10277,"event":true},{"date":"2026-07-19","deletions":368,"arrivals":3216,"apps_scanned":10146,"event":false},{"date":"2026-07-20","deletions":342,"arrivals":787,"apps_scanned":9995,"event":false},{"date":"2026-07-21","deletions":456,"arrivals":974,"apps_scanned":10107,"event":false},{"date":"2026-07-22","deletions":501,"arrivals":1302,"apps_scanned":10235,"event":false},{"date":"2026-07-23","deletions":471,"arrivals":1286,"apps_scanned":10267,"event":false},{"date":"2026-07-24","deletions":476,"arrivals":1185,"apps_scanned":9759,"event":false},{"date":"2026-07-25","deletions":508,"arrivals":1467,"apps_scanned":10114,"event":false},{"date":"2026-07-26","deletions":466,"arrivals":835,"apps_scanned":10248,"event":false},{"date":"2026-07-27","deletions":610,"arrivals":1001,"apps_scanned":10226,"event":false},{"date":"2026-07-28","deletions":636,"arrivals":1335,"apps_scanned":10118,"event":false},{"date":"2026-07-29","deletions":557,"arrivals":1411,"apps_scanned":10168,"event":false},{"date":"2026-07-30","deletions":316,"arrivals":1341,"apps_scanned":10393,"event":false},{"date":"2026-07-31","deletions":714,"arrivals":1397,"apps_scanned":10279,"event":false},{"date":"2026-08-01","deletions":609,"arrivals":1072,"apps_scanned":9393,"event":false},{"date":"2026-08-02","deletions":491,"arrivals":800,"apps_scanned":10285,"event":false},{"date":"2026-08-03","deletions":479,"arrivals":876,"apps_scanned":10176,"event":false},{"date":"2026-08-04","deletions":497,"arrivals":1136,"apps_scanned":9554,"event":false},{"date":"2026-08-05","deletions":333,"arrivals":1270,"apps_scanned":9118,"event":false}],"panel":[{"date":"2026-07-02","displayed":529895,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-03","displayed":530467,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-04","displayed":530855,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-05","displayed":531029,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-07","displayed":531340,"apps":19304,"gap_before":true,"event":false},{"date":"2026-07-08","displayed":530582,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-09","displayed":529788,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-10","displayed":528775,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-13","displayed":531872,"apps":19304,"gap_before":true,"event":false},{"date":"2026-07-14","displayed":532214,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-15","displayed":531764,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-16","displayed":521644,"apps":19304,"gap_before":false,"event":true},{"date":"2026-07-17","displayed":484334,"apps":19304,"gap_before":false,"event":true},{"date":"2026-07-18","displayed":519600,"apps":19304,"gap_before":false,"event":true},{"date":"2026-07-19","displayed":519978,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-20","displayed":520412,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-21","displayed":520904,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-22","displayed":521591,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-23","displayed":522299,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-24","displayed":522990,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-25","displayed":523806,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-26","displayed":524177,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-27","displayed":524631,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-28","displayed":525289,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-29","displayed":526001,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-30","displayed":526810,"apps":19304,"gap_before":false,"event":false},{"date":"2026-07-31","displayed":527439,"apps":19304,"gap_before":false,"event":false},{"date":"2026-08-01","displayed":527783,"apps":19304,"gap_before":false,"event":false},{"date":"2026-08-02","displayed":528141,"apps":19304,"gap_before":false,"event":false},{"date":"2026-08-03","displayed":528535,"apps":19304,"gap_before":false,"event":false}],"bridge":[{"key":"before","label":"Announced sweep","from":"2026-07-02","to":"2026-07-15","observed":1869,"ledger":-333,"arrivals":13591,"removals":13924,"gap":2202,"gap_share_of_activity":8.0},{"key":"across","label":"16–18 July","from":"2026-07-15","to":"2026-07-20","observed":-11352,"ledger":-64503,"arrivals":29031,"removals":93534,"gap":53151,"gap_share_of_activity":43.4},{"key":"after","label":"Fortnight after","from":"2026-07-20","to":"2026-08-03","observed":8123,"ledger":5890,"arrivals":10563,"removals":4673,"gap":2233,"gap_share_of_activity":14.7}],"arrival_age":[{"bucket":"same fortnight","event":6.9,"normal":83.4,"event_count":2892},{"bucket":"2 weeks–3 months","event":3.2,"normal":10.0,"event_count":1345},{"bucket":"3–12 months","event":10.6,"normal":3.3,"event_count":4442},{"bucket":"1–2 years","event":23.8,"normal":0.8,"event_count":10031},{"bucket":"2–3 years","event":23.8,"normal":0.6,"event_count":10006},{"bucket":"3+ years","event":31.8,"normal":1.9,"event_count":13383}],"recovery":[{"key":"back","label":"Back on the listing","value":74882,"share":50.7},{"key":"gone","label":"Still missing","value":72856,"share":49.3}],"scraper_health":[{"date":"2026-07-13","requests":115296,"failed":6,"rate_limited":0,"failure_pct":0.0,"event":false},{"date":"2026-07-14","requests":114256,"failed":10,"rate_limited":0,"failure_pct":0.0,"event":false},{"date":"2026-07-15","requests":113765,"failed":4,"rate_limited":0,"failure_pct":0.0,"event":false},{"date":"2026-07-16","requests":113178,"failed":6,"rate_limited":1,"failure_pct":0.0,"event":true},{"date":"2026-07-17","requests":110134,"failed":14,"rate_limited":0,"failure_pct":0.0,"event":true},{"date":"2026-07-18","requests":109185,"failed":10,"rate_limited":0,"failure_pct":0.0,"event":true},{"date":"2026-07-19","requests":112211,"failed":3,"rate_limited":0,"failure_pct":0.0,"event":false},{"date":"2026-07-20","requests":111711,"failed":4,"rate_limited":0,"failure_pct":0.0,"event":false},{"date":"2026-07-21","requests":111576,"failed":16,"rate_limited":0,"failure_pct":0.0,"event":false}],"loss_by_size":[{"bucket":"under 50","apps":4689,"lost":12197,"avg_pct_lost":28.9},{"bucket":"50–199","apps":1402,"lost":19248,"avg_pct_lost":13.6},{"bucket":"200–999","apps":661,"lost":41008,"avg_pct_lost":14.5},{"bucket":"1,000–4,999","apps":172,"lost":49133,"avg_pct_lost":13.9},{"bucket":"5,000+","apps":17,"lost":22345,"avg_pct_lost":12.8}],"fate":[{"key":"vanished","label":"Vanished","value":64550},{"key":"immediate","label":"Back in a day","value":50089},{"key":"late","label":"Back later","value":11892},{"key":"residue","label":"Still missing","value":2569}],"shelf":[{"key":"before","label":"Before · 15 July","total":770387,"parts":[{"key":"then","label":"reviews on the shelf","value":770387}],"ghost":0},{"key":"after","label":"Now · 3 August","total":770302,"parts":[{"key":"survived","label":"were there before and still are","value":715088},{"key":"from_event","label":"appeared during the three days","value":37310},{"key":"since_event","label":"arrived since","value":17904}],"ghost":70196}],"case_arc":[{"date":"2026-07-14","shown":28,"app_gone":0,"app_new":0,"app_new_age":null,"store_gone":959,"store_new":1302,"event":false},{"date":"2026-07-15","shown":28,"app_gone":0,"app_new":0,"app_new_age":null,"store_gone":2017,"store_new":1420,"event":false},{"date":"2026-07-16","shown":34,"app_gone":7,"app_new":8,"app_new_age":1044,"store_gone":24722,"store_new":7448,"event":true},{"date":"2026-07-17","shown":42,"app_gone":0,"app_new":6,"app_new_age":852,"store_gone":73602,"store_new":21829,"event":true},{"date":"2026-07-18","shown":42,"app_gone":3,"app_new":3,"app_new_age":738,"store_gone":45607,"store_new":12822,"event":true},{"date":"2026-07-19","shown":42,"app_gone":0,"app_new":0,"app_new_age":null,"store_gone":368,"store_new":3216,"event":false},{"date":"2026-07-20","shown":42,"app_gone":0,"app_new":0,"app_new_age":null,"store_gone":342,"store_new":787,"event":false}],"case_groups":[{"key":"came_back","n":4,"avg_rating":5.0,"critical":0,"median_length":194},{"key":"never_back","n":5,"avg_rating":5.0,"critical":0,"median_length":138},{"key":"untouched","n":41,"avg_rating":4.9,"critical":1,"median_length":136}],"case_examples":[{"group":"zombie","rating":5,"tenure":"About 1 month using the app","posted":"2022","text":"A superb upsell app. Indispensable if you want to lift your revenue without much effort."},{"group":"zombie","rating":5,"tenure":"6 days using the app","posted":"2023","text":"Responsive support who actually listen. Thank you [name]."},{"group":"zombie","rating":5,"tenure":"About 5 hours using the app","posted":"2023","text":"I recommend this app — the different upsells on offer are really well put together, and the support is quick."},{"group":"gone","rating":5,"tenure":"About 1 hour using the app","posted":"2022","text":"Exactly the upsell app we were looking for. Full control, and we can pick the trigger products."},{"group":"gone","rating":5,"tenure":"4 days using the app","posted":"2024","text":"Lovely app. Easy to use, and the support was patient and friendly. Looking forward to more features."},{"group":"back","rating":5,"tenure":"2 days using the app","posted":"2022","text":"I use this to offer clients new products after their first purchase. Happy with it, and the support is good."},{"group":"back","rating":5,"tenure":"10 days using the app","posted":"2022","text":"Very good app — it lets you add several upsells at once, and the customer service is excellent."}],"churn_daily":[{"date":"2026-07-15","total":770387,"net_change":null,"logged_gone":2017,"logged_new":1420,"event":false},{"date":"2026-07-16","total":755844,"net_change":-14543,"logged_gone":24722,"logged_new":7448,"event":true},{"date":"2026-07-17","total":705837,"net_change":-50007,"logged_gone":73602,"logged_new":21829,"event":true},{"date":"2026-07-18","total":755404,"net_change":49567,"logged_gone":45607,"logged_new":12822,"event":true},{"date":"2026-07-19","total":755926,"net_change":522,"logged_gone":368,"logged_new":3216,"event":false}],"group_profiles":[{"key":"corpus","label":"An ordinary live review","n":855685,"avg_rating":4.804,"pct_five":91.4,"pct_low":3.6,"median_length":110,"pct_empty":15.0},{"key":"returned","label":"Came back","n":73635,"avg_rating":4.802,"pct_five":91.2,"pct_low":3.5,"median_length":123,"pct_empty":10.9},{"key":"kept_off","label":"Still missing","n":70196,"avg_rating":4.632,"pct_five":87.1,"pct_low":8.0,"median_length":181,"pct_empty":7.8},{"key":"arrived","label":"Appeared instead","n":42099,"avg_rating":4.839,"pct_five":92.3,"pct_low":2.7,"median_length":99,"pct_empty":16.2}],"reach_gradient":[{"bucket":"1 app","returned":26401,"kept_off":20319,"pct_kept_off":43.5},{"bucket":"2–4","returned":31756,"kept_off":18081,"pct_kept_off":36.3},{"bucket":"5–9","returned":12036,"kept_off":12640,"pct_kept_off":51.2},{"bucket":"10–24","returned":3035,"kept_off":14489,"pct_kept_off":82.7},{"bucket":"25+","returned":53,"kept_off":3849,"pct_kept_off":98.6}],"examples":[{"group":"returned","rating":5,"tenure":"7 minutes using the app","text":"Awesome app, very clear, helps guide the customer toward free shipping. I'm going to install it on all our sites."},{"group":"returned","rating":5,"tenure":"5 days using the app","text":"Installing was a breeze with the step-by-step instructions, and now it's easy for my customers to download a one-page receipt."},{"group":"returned","rating":5,"tenure":"Over 1 year using the app","text":"Very good service. I had questions about importing reviews myself and couldn't work out how to upload the images. Now I know how. Much appreciated."},{"group":"kept_off","rating":1,"tenure":"12 minutes using the app","reviewed_apps":48,"text":"Very disappointed. No way to reach a person, and the app hasn't worked from the very start."},{"group":"kept_off","rating":3,"tenure":"7 months using the app","reviewed_apps":66,"text":"Good application and support is there, but no control over IP address or anything else to stop self-promotion. That is essential if you don't want to be scammed."},{"group":"kept_off","rating":5,"tenure":"23 days using the app","reviewed_apps":28,"text":"[name] and the team went well beyond sorting out several complications for me, making sure the app allocates our VAT correctly."},{"group":"kept_off","rating":1,"tenure":"About 1 month using the app","reviewed_apps":25,"text":"We've had a lot of problems recently. Our agent is on holiday too often and nobody is assigned to cover, so we start getting customer complaints."},{"group":"kept_off","rating":5,"tenure":"10 months using the app","reviewed_apps":25,"text":"This app helped us become the number one store in our niche, outperforming all of our competition."},{"group":"arrived","rating":5,"tenure":"3 days using the app","posted":"2018","text":"Their tech support fixed an unexpected integration problem with my theme in under 24 hours by adjusting the CSS. Far faster than I expected."},{"group":"arrived","rating":5,"tenure":"10 days using the app","posted":"2020","text":"Easy to understand. I haven't sold anything yet, but I'm looking forward to it and excited about what's ahead for my store."},{"group":"arrived","rating":5,"tenure":"Over 5 years using the app","posted":"2021","text":"I use this daily. Easy and reliable, and not so complicated that someone stepping in at the last minute can't pick it up."},{"group":"arrived","rating":5,"tenure":"About 1 hour using the app","posted":"2022","text":"Very useful. I contacted support for help and they were quick to sort my issue out. Will be upgrading my plan soon."}]}}