John Kirima / ResearchSource ↗

Case Study
№ 01

Detecting hidden risk in fintech customer feedback.

Standard sentiment models miss critical complaints. This project uncovers those failures and quantifies their impact across major fintech platforms.

10K+ Reviews4 AppsNLP + Topic ModelingSeverity Scoring
View GitHub→Read Methodology→
02The Problem

Sentiment is a poor proxy for product risk.

Models like VADER score reviews by emotional tone. But fintech complaints are rarely emotional: they are technical, procedural, and often written with calm precision.

The result: a review reporting that an account has been frozen for two weeks reads as neutral. A user describing an unresolved fraudulent charge is flagged as positive for using polite language.

Missing a complaint about “account locked” is not a sentiment error. It is a product risk failure.
03Approach

A five-phase analytical system.

  1. 01 / Phase 1

    Baseline Audit

    Measure where VADER's tone-based sentiment disagrees with the user's own 1/5 star rating.

  2. 02 / Phase 2

    Hidden Negative Detection

    Isolate low-rated reviews the model labels neutral or positive: the silent complaints.

  3. 03 / Phase 3

    Severity Scoring

    Apply a business-aware 1/5 scale weighted by impact terms: fraud, lockout, dispute, outage.

  4. 04 / Phase 4

    Competitive Benchmarking

    Compare hidden-negative and severity profiles across Venmo, Cash App, Chime, and PayPal.

  5. 05 / Phase 5

    Topic Modeling / BERTopic

    Surface the actual complaint categories driving severe, hidden negatives.

04Key Findings

The gap between perceived sentiment and actual user experience is measurable / and uneven across providers.

Severity of Hidden Negative Reviews / VADER Miss Rate by Severity

Fig. 01Hidden negatives / reviews the model failed to flag / cluster at the higher end of the severity scale, where impact on user trust is greatest.

Competitive Gap Heatmap / Hidden Negative Counts by App and Severity

Fig. 02Each provider exhibits a distinct gap signature between rating-based and model-based sentiment, exposing where monitoring blind spots concentrate.

Competitive Intelligence / VADER Failure Analysis by App

Fig. 03Benchmarking across Venmo, Cash App, Chime, and PayPal surfaces relative exposure: no platform is uniformly best, and weaknesses differ in kind, not only degree.
05Competitive Insight

Each platform fails differently.

Cash App
Highest rate of hidden negatives: calm, factual complaints that bypass tone-based detection.
Venmo
Highest concentration of severe missed complaints, weighted toward trust and money-movement failures.
PayPal
Account-dispute and resolution-process language dominates its missed-negative profile.
Chime
Recurring system reliability and support-response themes drive its hidden-negative volume.
06Topic Discovery

BERTopic surfaces the categories sentiment alone cannot see.

  • / Fraud & Scam Protection Failures (highest severity)
  • / AI Support & Bot Frustration
  • / Venmo Transaction Friction
  • / PayPal Account Disputes
  • / App Performance & Support
  • / Fund Holds & Card Issues
  • / Chime Banking System Issues

The 'Why' Behind the Miss / Complaint Themes per App (Hidden Negatives Only)

Fig. 04Topic concentration varies sharply by platform: the shape of a provider's complaint mix is itself a competitive signal.

Average Severity per Complaint Theme

Fig. 05Fraud-adjacent topics carry the highest mean severity, regardless of how the underlying review was scored by tone.

The failure is not just misclassification: it is missing the categories that matter.

07Business Impact

From sentiment metric to risk instrument.

  1. 01Improves continuous risk monitoring by separating tone from impact.
  2. 02Surfaces high-impact complaints earlier in the feedback pipeline.
  3. 03Helps product teams prioritize issues that move trust, not just volume.
  4. 04Enables competitor benchmarking beyond aggregate star ratings.
08Artifacts

Read the work.