What the project is
In June 2022 I pushed a repo called Leetcode-Stats-Scraping-Heroku. Four files. A Procfile with a single line, clock: python temp.py. A runtime.txt pinning python-3.7.8. A requirements.txt. And temp.py, about ninety lines of Python that opened a headless Chrome, walked to leetcode.com/S_Kaushik/, read eight CSS selectors off the page, packaged the numbers into a dict, and wrote a document to MongoDB Atlas. Once when the dyno booted, and then every day at 3 AM UTC via APScheduler.
There was no web endpoint. Nobody ever hit an HTTP route. The Heroku app existed to hold a Python process open and let a BlockingScheduler wake up once a day and take a snapshot of my LeetCode profile. The snapshot went into a Mongo collection called data in a database called leetcode on a cluster called lc.6kgul, keyed by a date string like "June 04, 2022". Fields: rank, easy, medium, hard, beats_per_easy, beats_per_medium, beats_per_hard, submissions.
I never wrote the reader. Nothing else in the repo, and nothing else in any of my other repos from that year, consumes that Mongo collection. The data was written for the future me who never showed up.
flowchart LR
A[Heroku clock dyno] --> B[APScheduler cron hour=3]
B --> C[headless Chrome via Selenium]
C --> D[leetcode.com/S_Kaushik/]
D --> E[7 Tailwind class selectors]
E --> F[PyMongo insert_one]
F --> G[(MongoDB Atlas lc.6kgul)]The Heroku free-tier era
This whole stack does not really exist anymore. Heroku killed the free tier in November 2022, five months after I pushed the last commit. The clock process type still exists on paid dynos, but the specific magic that made this repo work was two custom buildpacks: heroku-buildpack-google-chrome and heroku-buildpack-chromedriver. They installed a Chrome binary and a matching ChromeDriver into the slug at build time, and set GOOGLE_CHROME_BIN and CHROMEDRIVER_PATH env vars so Selenium could find them. When those buildpacks stopped being the default recipe, half the LeetCode-scraping blog posts on the internet stopped working.
A clock dyno was the community convention for a long-running Python process whose only job was to hold a scheduler open. No HTTP port, no queue, just a BlockingScheduler that never returned.
APScheduler did the actual timing:
sched = BlockingScheduler()
@sched.scheduled_job('cron', hour=3)
def scheduled_job():
scrape_and_write()
scrape_and_write() # one eager run at boot
sched.start()Hour 3 in the dyno's local timezone. Heroku defaults to UTC. So the scrape ran at 8:30 AM in India, right after I had usually done a fresh submission the night before.
Why Selenium was the obvious wrong tool
LeetCode has had a public, undocumented, no-auth GraphQL endpoint at leetcode.com/graphql the whole time. The query is matchedUser(username: "S_Kaushik") and it returns everything I was scraping and more, as clean JSON, in one request, in under 200 milliseconds. No browser, no headless Chrome, no eighty megabytes of Chromium in the slug.
I did not use it. I did not know it existed. What I knew was that I could open Chrome DevTools, right-click on the rank number, hit "Copy selector", and get back something like #__next > div > div > div > div > div > div > div.text-label-1.dark\:text-dark-label-2 > span. Feed that to driver.find_element(By.CSS_SELECTOR, ...), get the text back. Eight selectors, eight find_element calls, done.
The failure mode is exactly what you would expect. LeetCode uses Tailwind and Next.js. Every time they run their build, class names shuffle and DOM depth changes. That selector is not a locator, it is a bet against the frontend team. My bet lost within a year, probably within months. I never went back to check because the code stopped mattering to me the day after I pushed it.
The mental step I did not take in 2022 was "before I automate the browser, does the site expose an API." That step is free. One page load in DevTools with the Network tab open, filter by XHR, refresh, see what the frontend itself calls. LeetCode's own React app pulls the profile data from /graphql. The information was visible. I just did not look.
The commit history is only requirements.txt
Ten commits, all between the initial push and June 4, 2022. Almost every one of them is requirements.txt. Pin pymongo up. Add pymongo[srv] for the mongodb+srv:// scheme. Downgrade chromedriver_autoinstaller because 0.4 broke on 3.7. One commit strips 246 lines of accidental pip freeze bloat from a laptop with a full data-science environment installed. Two commits touch temp.py.
That is what a working script looks like when it is deployed. The logic freezes on day one. Everything after is churn around the edges: dependency pins, buildpack quirks, the small unglamorous work of keeping the ambient environment stable so the code that already works keeps working. I expected a repo to grow features. Real ones grow scaffolding around a small unchanging core.
There is a bug in temp.py I noticed only today. There are three webdriver.Chrome(...) initializer calls in the file. The first two both set chrome_options.binary_location = os.environ.get('GOOGLE_CHROME_BIN'), which is how Selenium is supposed to find the Heroku-buildpack Chrome. The third, inside save_data_to_mongo(), is a bare webdriver.Chrome() call with no options at all. Whichever run I most recently tested probably worked by accident. Two carefully configured driver constructions get overwritten by an unconfigured one at scrape time.
The credential leak
Now the part I have been dreading. In temp.py, on a visible line in the public repo, I committed:
client = MongoClient("mongodb+srv://<username>:<password>@lc.6kgul.mongodb.net/...")except the angle brackets are not there. The username and password are the real admin credentials of the Atlas cluster, in plaintext, in a public GitHub repo, for three years. I did not touch that file between June 2022 and today, July 9, 2026.
I noticed while writing this post. I opened temp.py to describe the ninety lines and I saw the string. The moment you read this, treat the credential as fresh only if I have already rotated it on Atlas. The clean way to rotate:
- Atlas dashboard, Database Access, add a new admin user with a fresh password.
- Update anything that still uses the old one, in my case nothing on live infra.
- Delete the old user.
- Network Access, view active connections, force-disconnect any live session on the old user.
- Rotate the Atlas API keys for that project too, since a compromised admin could have created them.
The Mongo data itself is not sensitive. Rank and beat-percentage numbers of a college kid on LeetCode in mid-2022. The credential was the sensitive thing, and rotation is the actual security event, not deletion from HEAD.
If you have ever pushed a script like this, stop reading and run this now on every old repo you own:
git grep -E 'mongodb\+srv://|postgres://|mysql://|api[_-]?key' $(git rev-list --all)
That scans every commit, not just the current tree. Even if you deleted the secret from HEAD, the old commit still holds it. The permanent fix is git filter-repo --replace-text and a force-push, plus rotating the credential itself. Rotation is the actual security event. Rewriting history is cleanup.
The habit going forward, and it is the habit I have used since late 2022: no secret ever appears in a source file. os.environ["MONGO_URI"] in the code, the actual URI in Heroku's config vars or a local .env that is gitignored. Two extra lines of setup per project.
What I would build today
Delete Heroku. Delete Selenium. Delete Chrome. Delete Mongo. Delete APScheduler.
A GitHub Action, cron 0 3 * * *, runs a twenty-line Python file. The file makes one POST to leetcode.com/graphql with a matchedUser query and a username variable. It writes the response into data/leetcode-YYYY-MM-DD.json in the repo. Commits with a bot token. Pushes.
No server, no dyno, no cluster, no buildpack, no ChromeDriver mismatch, no Atlas connection string to leak. The data lives in the repo, publicly, next to the code that produced it, and every historical value is a git log away.
That is the version a slightly older me would have written in an afternoon. The one I actually shipped in 2022 was a monument to not knowing what I did not know. Which is fine. It is what the college version of anyone's code looks like. The point of writing it down is that the mistake in it, the credential in a public repo, is the exact mistake that keeps being the mistake for people at every level. Rotate it now, then rotate the habit.
See also
- /blog/college-python-utility-scripts, same era, same one-file shape, no credentials leaked.
- /blog/cipherstack-vault-full-architecture, where I put secrets four years later so this could not happen again.
- /blog/aikeyrotator-before-cipherstack, the intermediate step from "hardcoded" to "vaulted".