Picks Pipeline Notes

notestechnical

Picks Pipeline Notes

Updated: 2026-02-04 14:25 EST

Architecture (IMPLEMENTED ✅)

TG Parser + Scrapers → AllPicks tab → Supabase → Apps

PM2 Processes

ProcessTypeStatusNotes
dailyai-picksCron (30 min)"stopped" = normalRuns, scrapes, exits. Not a daemon.
cappers-parserDaemon✅ RunningLive TG parser → AllPicks + Supabase
sheets-supabase-syncCron (10 min)✅ RunningAllPicks → Supabase

Data Flow (LIVE)

  1. Scrapers (dailyai-picks) → AllPicks tab every 30 min
  2. TG Parser (cappers-parser) → AllPicks tab (live) + Supabase (live)
  3. Sync (sheets-supabase-sync) → AllPicks → Supabase every 10 min
  4. Daily AI + HiddenBag → Read from Supabase

AllPicks Tab Format (7 columns)

ColumnDescription
SiteSource (TG-FreeCapper, BetFirm, etc.)
LeagueSport (NBA, NFL, NCAAB)
DatePick date (YYYY-MM-DD)
MatchupGame if available
ServiceCapper name
PickFull pick with odds/units
RunDateWhen scraped

Key Files

FilePurpose
C:\Users\mpmmo\dailyai-picks-local\index.jsWeb scrapers
C:\Users\mpmmo\cappers-raw\cappers_free_live.pyTG parser
C:\Users\mpmmo\hiddenbag-v2\scripts\sheets-to-supabase.pySync to DB
C:\Users\mpmmo\cappers-raw\README.mdFull documentation

Google Sheet

  • ID: 1dZe1s-yLHYvrLQEAlP0gGCVAFNbH433lV82iHzp-_BI
  • Tab: AllPicks (primary), Picks (legacy)

Completed Tasks ✅

  • Update cappers-parser to write to AllPicks (same format as scrapers)
  • Add sheets-supabase-sync to PM2 (every 10 min)
  • Test full flow: Scrapers → AllPicks → Supabase → Apps
  • Remove hardcoded API keys (security)
  • Create README documentation
  • Clean up dead code (removed backfill_today.py)
  • PM2 save for reboot persistence