Discovery

Discovery answers one question: which companies exist that we should know about? It reads the places early-stage startups actually post — their own job boards — and admits a company only through one gate.

Sources#

Of 8,000 roles on record, the largest sources are Ashby (2,185), careers pages (1,706), Greenhouse (1,654) and Wellfound (1,296) — company-owned boards, exactly where the hidden jobs are.

The admission gate#

Every new company — from any source — passes admission.admit():

• Clean the name and reject bogus websites (link shorteners, social profiles, aggregators).
• De-duplicate by exact name, normalised name key, and domain.
• Apply the overrated rule: famous, listed, valued at $5B+, or raised $1B+ → excluded. Borderline companies stay only while visibly hiring and able to pay.

A rejected company's postings are dropped and counted, not stored.

India discovery#

India is restricted to Bengaluru and Hyderabad, and a company is admitted only with a quote that places it in one of them. New finds get india_found_at, which moves them to the front of the fill queue for 30 days. A posting-led channel (india_jobs.py) runs every two hours: it reads LinkedIn's logged-out job search and Indian boards, filters out body shops and giants, and admits the employers.

The pay floor#

An India company whose best full-time pay evidence is under targeting.india.min_lpa (₹20 LPA) is hidden as below_pay_bar — not deleted — and comes back by itself when a posting states more.

Roles from boards#

pipeline.scrape() runs every enabled scraper concurrently, keeps roles whose title and location fit, removes duplicates by fingerprint (keeping the richest description), parses salary into ₹ LPA, and upserts by URL. The first posting from a company becomes its recorded hiring claim.