Feature adoption can be measured before analytics is instrumented

    August 23, 2026
    8 min read
    By Ansel Lindholm
    Updated October 26, 2026
    Share this article

    A feature can ship, run for 6 months, and still leave you unable to say who actually uses it. Someone says "most of our 80 eligible accounts use it", and nobody can name the query behind the claim.

    You do not need perfect event tracking to answer that honestly. You do need to stop quoting a number you cannot defend.

    Start with a defensible adoption claim

    Missing instrumentation does not mean missing evidence. Before event tracking exists, product databases, audit trails, CRM, support, and account reviews still provide behavioral evidence.

    The gap matters because the base rate is unforgiving. Pendo's 2019 Feature Adoption Report, analyzing usage across thousands of cloud applications, found that roughly 80% of features are rarely or never used, and tied tens of billions of dollars of cloud R&D spend to features nobody touches. An older Standish Group analysis presented at the XP2002 conference reached a similar place, with 45% of features never used and another 19% rarely used. Against numbers like those, assuming a feature is adopted because it shipped is the expensive default, and a defensible count is what tells you which side of that ratio you are on.

    Do not produce a neat-looking rate from incomplete records. Instead, keep an adoption proof artifact: a record anyone can inspect. It states how adoption is defined, which customers were eligible, what evidence was found, which data is missing and how confident you are. That record supports real decisions: improve onboarding, look into a segment, pause the rollout or pay for tracking.

    Keep the definition narrow. For workflow features, adoption rarely means seeing a button or switching on a setting. It means an eligible account completing the action that delivers value, and, where it matters, repeating it as often as the work naturally recurs.

    Manual adoption rate = eligible accounts with verified value behavior / eligible accounts exposed to the feature

    Check the bottom of the fraction as carefully as the top. Before launch, some accounts look eligible on paper but are blocked by their plan, permissions, an unfinished migration or a switched-off integration. Counting them invents an adoption problem that does not exist.

    An illustrative cohort shows how much the denominator moves the answer. Of 80 accounts with the flag on, 16 turn out to be blocked: 9 by plan limits, 4 by missing admin permissions, 3 by an unfinished migration. That leaves 64 truly eligible. The query finds 22 with a verified value action and 13 with a repeat.

    • Naive rate: 22 / 80 = 27.5%.
    • Honest rate: 22 / 64 ≈ 34%.
    • Repeat rate: 13 / 64 ≈ 20%.

    A 64-account cohort is small, and the estimate should say so. The 95% margin of error is about 1.96 × √(0.34 × 0.66 / 64) ≈ 12 points, so "between roughly 22% and 46%" is the defensible statement. If the result looks surprisingly good, treat that as a reason to reopen the query before anyone celebrates. A number that jumps out from a hand-built cohort is more often a double-counting join, a stray test account or a misread status than a real discovery.

    Build an evidence chain from product records

    Start with the strongest record nearest the action. A completed-workflow row is stronger than a CRM note that a customer “uses the feature.” Support can explain a failed attempt but cannot prove successful use. Layer evidence rather than blending it into a vague count.

    Evidence source Establishes Common distortion Best use
    Product database Saved object, completed job, changed state Test data, automated jobs, deleted records Primary count
    Audit log Actor and time Missing retention, system actors Repeat use, actor validation
    CRM Contract scope, plan, rollout, owner context Stale fields, optimistic notes Eligibility, segmentation
    Support Friction, failures, workarounds, requests Vocal customers are not the cohort Diagnose non-adoption
    Account review Whether use produced value Recall bias, uneven documentation Edge cases

    Query the outcome itself and resist the easy proxy. For collaboration, count approval records; for exports whose value requires an external destination, count completed deliveries; clicks on “Export” prove nothing about value.

    Audit trails reveal repeat behavior

    This query counts accounts completing an action after eligibility. Adapt schema names and statuses, and save the exact version in the proof artifact.

    WITH eligible_accounts AS (
      SELECT a.id AS account_id, a.plan, a.feature_enabled_at
      FROM accounts a
      WHERE a.feature_enabled_at IS NOT NULL
        AND a.status = 'active'
    ), verified_use AS (
      SELECT DISTINCT al.account_id
      FROM audit_log al
      JOIN eligible_accounts ea ON ea.account_id = al.account_id
      WHERE al.action = 'approval.completed'
        AND al.actor_type = 'user'
        AND al.created_at >= ea.feature_enabled_at
    )
    SELECT
      COUNT(*) AS eligible_accounts,
      COUNT(vu.account_id) AS adopted_accounts,
      COUNT(vu.account_id)::decimal / NULLIF(COUNT(*), 0) AS adoption_rate
    FROM eligible_accounts ea
    LEFT JOIN verified_use vu ON vu.account_id = ea.account_id;
    

    actor_type = 'user' removes non-user events only if your schema classifies them that way; check test accounts and admin repairs separately. If audit retention is only 90 days, report a 90-day view and label it that way.

    For repeat value, require two completed actions on different dates or a second within the expected interval. Payroll may be monthly; a shared inbox may need weekly activity. No universal repeat window is defensible.

    Match the manual method to data maturity

    Manual measurement has limits, so match the method to what your records can support. Pick it by the quality of the records, the cost of a wrong answer, and whether people use the feature alone or as a team.

    • One-off database query. The cheapest start. It works whenever the feature leaves a lasting object or status change behind. Its weak point is identity: a bare row rarely says which account or user it belongs to.
    • Audit-log cohort. A little more work, worth it once the log records who did what, when and to which object. Watch the retention period: anything older is silently cut off.
    • CRM-assisted account review. Worth the effort for features sold to enterprise accounts. Remember that CRM fields drift away from what really happens in the product.
    • Support-tag review. Similar effort, best when the real question is why people fail. What you see depends on how support tags tickets, not only on how customers behave.
    • Structured manual cohort. The richest view for the most effort, good for a small launch group or beta. The main risk is two reviewers using different criteria, so write the criteria down.

    A structured cohort also suits a multi-step workflow that ends outside the product. Check what each record proves first. A B2B integration can create a connection row while the value depends on data arriving correctly later. That row only shows setup. Adoption starts when the data actually lands.

    Do not spend a week reconciling edge cases for a low-risk question, or use a loose CRM field to decide whether to remove a paid-tier feature. Required evidence rises with the cost of being wrong.

    When account records impersonate adoption

    Before tracking exists, teams often mistake availability, setup or exposure for use. One account switches on a rules engine, creates one rule and never activates it. Another uses the API with no named person behind the calls. A third is marked “live” in the CRM after a good call, although support shows the integration failed the next day.

    Separate these states:

    • Eligible: usable under plan, permissions, and technical setup.
    • Configured: prerequisite completed, such as connecting a source or creating a rule.
    • Used: value-bearing action occurred once.
    • Repeated: action recurred in the expected interval.
    • Validated: product and customer-facing evidence agree that the intended outcome occurred.

    Without that split, a chart can rise after onboarding only because setup grew while repeat use stayed flat: activation improved while durable adoption stayed where it was. Telling the two apart changes the next decision.

    Also define the unit: account, user, workspace, project, or transaction. One active administrator is shallow use for a collaboration feature, yet exactly the intended pattern for a finance-control workflow.

    Run a two-week cohort review across product, CRM, and support

    For a meaningful launch cohort, a two-week review is more reliable than one dashboard request. Freeze the cohort and record eligibility date, plan, rollout condition, owner, and segment in one sheet, with the same columns for every account.

    The eligibility date is when the feature flag went live for that account, say 12 May, owned by product operations. First verified use pairs an audit event with the created object, an approval for instance, and is confirmed by a product analyst or engineer.

    Repeat use is a second approval about a week later, checked by the same analyst. Customer context (the CRM rollout stage and the named workflow the account actually runs) comes from customer success, and any friction signal, a tagged ticket or a call note, from the support lead. The review owner closes each row with a confidence grade of A, B, or C and the reason for it.

    Product/engineering owns the query and schema assumptions. Customer success verifies scope and external blockers. Support groups failures by cause: permissions, import errors, confusing setup, workflow mismatch. A product manager resolves ambiguity and records the definition.

    Request evidence tied to a known account ID, not verbal estimates. “Team is evaluating approvals” is useful context and zero evidence of usage. A ticket titled “approval notification delayed” can reveal a completed action missed by audit logs, but needs date, account, and ticket status before amending the cohort.

    Publish verified use, repeated use, and unverified or ambiguous accounts. The ambiguous bucket identifies the instrumentation gap.

    Grade confidence before reporting a rate

    A manual rate is an estimate with an audit trail. It should never pass as an instrumented metric, and grades stop readers from treating every percentage as equally trustworthy.

    Three grades are enough:

    • Grade A: a lasting product record, known eligibility, clear identity and a repeat check where it matters. Report it as “Verified adoption.”
    • Grade B: a strong product record with one known gap, usually incomplete historical logs. Report it as a “Directional adoption estimate.”
    • Grade C: CRM, support or interview evidence with no lasting product record behind it. Report it as “Customer-reported or manually reviewed use.”

    Keep calculation and caveats together: “Cohort: active Pro accounts enabled between 1 and 31 May. Use: approval.completed by a human actor after enablement. Repeat: a second completion within 14 days. Exclusions: internal accounts, migration jobs, disabled accounts. Source: production database query dated 15 June. Grade: B because audit retention begins after the earliest enablements.”

    This is more useful than a chart labeled “feature adoption”: it enables reruns, explains movement, and exposes weak evidence. Use grades to prioritize tracking. C-grade evidence for pricing, retention, compliance, or expansion makes instrumentation essential; a small beta with database-proven behavior may use manual measurement until it graduates.

    Instrumentation should close known evidence gaps

    Analytics makes recurring analysis cheaper, reveals sequences tables hide, and enables segmentation; it does not erase judgment. Base the first tracking plan on the blind spots the manual review exposed; a generic catalog of button clicks will miss them.

    Turn unresolved questions into events or properties. If exports cannot be separated into success and abandonment, track export_started, export_completed, failure reason, destination type, and account plan. If account-level adoption hides shallow team use, add user identity and role. If value depends on an external system, record delivery confirmation where privacy and technical constraints allow.

    Keep the original definition after tracking. Redefining adoption from completed workflow to button click because the event is convenient breaks trends. Run the manual query and event definition in parallel for one review cycle, reconcile differences, then retire the old artifact.

    Leave the review with a metric contract

    Limited records are still evidence. They justify neither giving up on measurement nor inventing certainty. Start with one value-bearing action, a clearly eligible cohort, and a saved query; add CRM and support only when they change interpretation.

    The review should end with a metric contract. It names the unit, the eligibility rule, the value event, the repeat interval, the exclusions, the source tables, the owner, how often it is reviewed and the confidence grade. The payoff comes twice. Leaders get an answer they can act on before any dashboard exists. And when tracking finally ships, analytics works from a precise brief instead of a wish list. The first dashboard either confirms the manual number or shows, field by field, why it differs.

    Related Articles