Operations

Troubleshooting

Known failure modes and exactly what to do about each.

Every failure mode below has happened at least once. Each lists what you see, why, and what to do.

Gmail refuses mail: 550 5.4.5#

Symptom: Emails go to failed with Daily user sending limit exceeded long before 500 a day.
Cause: Gmail’s spam protection on a burst of near-identical mail to one domain, made worse by bounces.
Fix: Stop sending (unload com.zonuly.tick.send) and wait about 24 hours. Put refused emails back to approved. Keep front_load: false, max_per_run: 1 and the 25 cap — finish a company over several days.

Symptom: “Address not found” replies for firstname@company addresses.
Cause: The company’s pattern doesn’t hold for everyone — newer hires use another format, or an example was a person who left.
Fix: Bounced addresses are retired automatically; the brake stops guessed sends after 6 in 7 days. Approve verified addresses first, and watch the guessed flag in the Queue.

Symptom: A handful of emails over a whole afternoon.
Cause: The Mac was asleep — a closed lid sleeps it even with caffeinate — and jobs only run during brief wakes.
Fix: Keep the lid open and on power from 10:00 to 19:00. With max_per_run 1 nothing bursts when it wakes; slots simply wait.

Symptom: A command or send fails with sqlite3.OperationalError: database is locked.
Cause: Another job — usually the referral cycle — held a long write transaction past the 30-second busy timeout.
Fix: Find the holder with lsof jobhunter.db. Stopping a referral run is safe (it reruns in 15 minutes; SQLite rolls back). Pause com.zonuly.referrals during a manual send.

Symptom: actor status shows a route cooling down; people searches find nothing new.
Cause: LinkedIn throttles logged-out visitors from one IP after only a few profile views.
Fix: Let the breaker cool down (it resumes by itself). Don’t rotate IPs or use accounts — use Apify for People tabs instead.

Symptom: The button turns to Fill again and the company still shows missing parts.
Cause: The fill ran but found nothing new — e.g. roles only on an aggregator, or no careers page.
Fix: Read the line under the company: it names the reason for each missing part. Those gaps are real absences, not bugs.

Symptom: Pages load but every panel errors or is empty.
Cause: The API on :8000 isn’t running, or was started before a backend change.
Fix: Start or restart it: python scripts/run.py serve. Check curl localhost:8000/api/health.

Symptom: today's Apify cap reached ($0.6 of $0.6), or the Mixer panel says “not buying today: referrals first”.
Cause: The daily guard: $0.60 a day spreads the $19 Starter credit over the month, and the mixer must leave $0.30 of it for the People-tab job. Nothing was charged past the cap.
Fix: Wait until midnight, run a smaller --apify-max, or raise actors.apify.daily_usd knowing the month's credit runs out sooner.

Symptom: The send log says 3 cold mails into … already today or cold mail pauses for 48h after a block.
Cause: Not an error: at most three cold emails a day go into one company’s domain, and after Gmail refuses any mail as spam no cold mail leaves for 48 hours (that company’s domain: 21 days).
Fix: Nothing to do — the email stays approved and goes on a later day. Approve asks across several companies so each day has something it may send.

Symptom: Today’s card is empty.
Cause: The brain has not run since roles were scored (the free tick that runs it is not loaded, or the scrapers are paused with tick.scrapers_frozen), or every shortlisted company was picked in the last 30 days or is already being written to.
Fix: Run python scripts/run.py decide run, then reload.

Still stuck? The hourly audit and the watchdog record most problems before you see them:

diagnose

python scripts/run.py doctor
python scripts/run.py audit show
python scripts/run.py notify watchdog
python scripts/run.py research doctor