hackathon-explorer is a Vite + React 19 SPA that ships a static 4 MB JSON file containing 10,014 hackathon-winner entries. The set covers 464 hackathon series across 53 countries, from 1999 to 2026. There is no backend. Every visitor pulls the entire dataset over the wire and filters it in the browser. Live at hackathon-explorer.vercel.app, first and only commit 8e8bea9 on 2026-05-12.
This post is about that decision and its consequences. The blob-in-the-bundle approach is fine for 10K rows and dishonest for 100K. Along the way I want to walk through what the scrape actually gave me, and where the data quietly lies.
Why bundle a 4 MB JSON
The database is src/data/hackathons.json. It is imported at build time, chunked into the JS bundle, and hydrated once when the app boots. From that point on the entire table lives in a React state object. Every search, filter, sort, and export runs against that in-memory array.
The upside is real. There is no API to keep alive. Vercel's static edge caches the whole app. Latency for every filter is a JS array operation, which on modern hardware means single-digit milliseconds even for a 10K-row scan. Nothing to authenticate, rate-limit, or fix at 3 AM.
The downside is that I am shipping four megabytes of data to every visitor whether or not they need it. Anyone on a phone-tethered connection pays for the entire corpus on first paint. The React render itself is a bigger problem, which I get to at the bottom.
The rule of thumb I settled on: if the dataset is under 10 MB and updates less than once a week, a bundled JSON beats an API by every metric I care about. Above that, the tradeoff flips.
The scrape
Every row has 11 fields: hackathon, year, country, city, region, theme, winner_team, project_name, description, tech_stack, prize. Two of those came cleanly from one endpoint. The rest needed a second pass.
Devpost exposes an internal API at /api/hackathons. Passing winners_announced=true returns paginated hackathon metadata: name, dates, prize tiers, categories, location. That gets me the hackathon-level rows, the year, and the theme. What it does not give me is the actual winning project details. Those live on each hackathon's /project-gallery page, one HTML page per event, with project cards that link out to individual project detail pages.
So the scrape is two passes. Pass one hits the paginated API and collects hackathon URLs. Pass two walks each project gallery and pulls the project name, team, description, and tech tags. The intersection of "hackathon that has announced winners" and "project marked winner on Devpost" is the row set.
The actual scraper is not checked into this repo. There is a single "Initial commit" and the JSON. CONTINUE.md describes the two-pass shape, but the scripts live somewhere else on my disk. That is a mistake I keep making with data-collection code.
CONTINUE.md as an honest data-quality diary
The file I am proudest of in this repo is CONTINUE.md. It is a running list of the ways the data lies. Some of those lies are the scraper's fault. Most are Devpost's.
Here are the numbers I flagged when I stopped scraping:
- 78.2% of rows have
country = "Global". Devpost lets online-only events set their location to "Global" or leave it blank, and most of them do. Fixing this would mean guessing the organizer's location from other signals, and I would rather have an honest blank than an invented country. - 86.6% of
tech_stackvalues are literally the string"AI/ML". Devpost changed its tag schema at some point in 2023, and older events show a single coarse category instead of the per-project tag list. Everything past 2023 has real tags. Everything before it is compressed. - 93.5% of
prizevalues are the string"Winner". Same story. The finer-grained prize tiers (Grand Prize, First Place, Best Use Of ...) are on the project gallery page for recent events, and were not exposed the same way in the older HTML. - 73.7% of rows have
year = 2024. This one is a scraper bug, not a source limitation. When the API returns a hackathon whose start date I could not parse, my fallback defaulted to 2024. That is fixable in a second pass without re-scraping the gallery pages. - 117 rows are true duplicates that my initial dedup missed. That is a smaller number than it looks like, and I want to talk about it separately.
The two categories matter. "Source limitations" (Global, AI/ML, Winner) are honest gaps. A future scrape cannot fix them without inference. "Scraper bugs" (2024, duplicates) are things I can fix by rerunning the pass with a better key.
The dedup problem
My first dedup key was project_name.toLowerCase(). That collapsed my raw row count from around 12K down to about 9K. Which felt clean until I realised that Spyder, an unremarkable name, won three unrelated hackathons run by three different organizers in three different years. My dedup had eaten two of them.
The fix was to key on (hackathon + project_name).toLowerCase(). Same project name at different hackathons is a different row. Same project name at the same hackathon, which sometimes happened because of retries or gallery-page reshuffles, is one row. That got me back to 10,014 unique winners, with 117 residual duplicates flagged for manual review.
The lesson is that dedup keys are load-bearing. A project-name-only key looks correct on the ten rows you inspect, and quietly deletes seven percent of your data on the ten thousand you do not.
The pipeline, drawn
flowchart LR
A[Devpost /api/hackathons] -->|winners_announced=true| B[464 hackathon URLs]
B --> C[per-project gallery scrape]
C --> D[10,014 rows, 11 fields]
D --> E[dedup on hackathon + project_name]
E --> F[hackathons.json · 4 MB]
F --> G[Vite bundle]
G --> H[browser: filter + sort + xlsx/csv export]Nothing about that shape is clever. It is the standard two-pass shape for any site where the list endpoint is one thing and the detail endpoint is another. The only novel choice is that the artifact is not a database, it is a JSON file with the app wrapped around it.
What actually wins hackathons, 1999-2026
The earliest row in the set is the OpenBSD c99 hackathon in Calgary, 1999. The winning project was an IPv6 and IPsec integration for the OpenBSD kernel, which is a reminder that "hackathon" used to mean something closer to "week-long systems-programming retreat" than it does now. TechCrunch40 2007 is also in the set. That is where Mint.com launched, which stretches the definition toward "product-launch pitch competition". I let it stay because Devpost tagged it as one.
The most recent rows are from the Google Gemini Competition 2024 and a cluster of student ML hackathons that all have the same generic tech stack because of the tagging issue above.
What can you actually learn from this? Not as much as the raw count suggests. The AI/ML compression flattens "which framework do winners use". The Global compression flattens "where do winning teams live". What the data does answer well is the shape of the winner-announcement pipeline: events per year, how the count grew from single-digits in 1999 to hundreds per year by 2023, and which series have run the longest.
Where it hurts
The single largest problem with the app is not the data, it is the render. Bundling a 4 MB JSON works. Rendering all 10,014 rows into a single HTML table at once does not. Chrome takes multiple seconds on the initial paint, the tab hangs briefly on every filter that widens the result set, and scrolling past the first thousand rows on a mid-tier laptop is not smooth.
The fixes are obvious. Virtual scrolling would drop the DOM node count from 10K rows to whatever fits in the viewport, which is where a table like this should have started. Lazy-loading the JSON, so the empty-state renders first and the data hydrates after, would take the largest-contentful-paint from "multi-second" to "sub-second".
I labelled both of those P1 in CONTINUE.md. I have not shipped either. The repo is one commit old and there is no follow-up work. If you fork this, start with virtual scrolling. The rest of the app is fine.
What I would do differently
Check the scraper into the repo. Log the year-parse failures instead of defaulting them to 2024. Ship virtual scrolling on day one. And keep the CONTINUE.md diary format, because looking back at those numbers a month later was more useful than any commit message.