What this post is
I have a blog post on the site called "Why we fine-tuned DeBERTa-base and not XLM-R for German PII." That post is the retrospective version. It is the version where I already knew the answer, already had the eval numbers, and could tell the story cleanly in one arc.
This post is about the notebook that sits underneath the retrospective. It lives in a public repo called PII-Deberta-finetuning-Ai4Bharat-naamapadam, was originally exported from Kaggle in June 2024, and is exactly 27 cells long. It has only two thin markdown cells between sections. It has variables called k, arr, and i. The steps are numbered as inline comments because that is how you write a training notebook when you are trying to make the loss curve move, not when you are trying to explain what you did.
The two things are honest about different parts of the same work. The retrospective is honest about the reasoning. The notebook is honest about the mess.
The dataset the notebook actually loads
The notebook opens by loading ai4bharat/naamapadam for eleven Indian language codes: as, bn, gu, hi, kn, ml, mr, or, pa, ta, te. Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu. Naamapadam is Ai4Bharat's named-entity dataset built from parallel corpora, tagged for PERSON, LOCATION, and ORGANIZATION in IOB2, with the tags projected across languages via alignment from an English annotated source. It is the largest publicly available NER dataset for Indic languages that I know of, and for a token-classification fine-tune it is the obvious starting point.
What you get from Naamapadam that you do not get from anything else is coverage across scripts. Bengali script for Bengali and Assamese. Devanagari for Hindi and Marathi. Gurmukhi for Punjabi. Tamil, Malayalam, Kannada, and Telugu each have their own script. Gujarati and Odia have theirs. The tokenizer for whatever base model you pick has to handle all of them, and the fact that most base models will tokenize non-Latin scripts into single-character byte-pair fragments is exactly the failure mode that the retrospective post gets into.
The load_dataset call in cell 0, looped over the eleven language codes in arr, returns a list called k, and cell 3 uses concatenate_datasets to stitch the train, test, and validation splits into one combined pile. This is the first honest thing about the notebook: I did not stratify. I did not check whether Hindi was overrepresenting the combined corpus, which it almost certainly was, because the Hindi split of Naamapadam is by an order of magnitude the largest. The notebook trains on the merged pile and moves on.
What "Step 1, Step 2, Step 3" actually looks like
The cell comments in the notebook are numbered from Step 1 through Step 35. This is a real convention I use during training. When I am working on a Kaggle GPU with a time limit, I keep a linear stepped log inside the code because the alternative, which is jumping between markdown cells to describe what a code cell does, breaks the runtime rhythm. Numbered comments let me scroll and find where I was. That is the whole reason.
The steps in this notebook are:
- Steps 1-3, load the eleven languages
- Steps 5-10, merge and set up combined_dataset
- Steps 11-12, pull the NER label names off the schema
- Steps 13-16, tokenize with
google-bert/bert-base-uncased(yes,bert-base-uncased, not DeBERTa, that is what the notebook starts with) - Steps 17-19, DataCollatorForTokenClassification and a sanity print
- Steps 20-25, seqeval evaluation with a
compute_metricsthat filters -100 labels - Steps 26-27, id2label and label2id
- Steps 28-30, a
shutil.rmtree('./tmp_trainer')that is there because a previous run left a directory - Steps 31-35, re-import Trainer, reload the model with the mapping, kick off
trainer.train()andtrainer.push_to_hub
The redundant model reload at step 32 is the honest part. In a clean write-up you would delete it. In the actual notebook it stays, because when you are debugging why the trainer will not initialize you sometimes end up re-instantiating the model after fixing the id2label mapping, and once training runs you do not go back and clean up the cell. The shutil.rmtree at step 29 is the same kind of artifact. Neither of those things belongs in a polished write-up. Both of them are what the notebook actually contains.
What the notebook does not show you
The notebook does not show the eval numbers. The trainer.train() call in the final cell was run on a Kaggle GPU, the model was pushed to my Hugging Face account, and the loss and F1 that came out of it live in the run logs on Kaggle. I did not paste them back into the notebook. That is a real limitation of the artifact. If you clone the repo today and want to know what the fine-tune scored, you have to rerun it.
The notebook also does not show what the head of the model looks like for the entity classes I actually wanted to detect at the hackathon. Naamapadam has three-way tags: PER, LOC, ORG. The GE Healthcare Hackathon PII challenge, which this notebook was the Naamapadam-side experiment for, needed sixty-plus PII and PHI classes. Names, dates, addresses, phone numbers, medical record numbers, drug doses, diseases, chemicals, IP addresses, credit card CVVs. The gap between "three IOB2 classes on Indic corpora" and "sixty PII/PHI entity classes on healthcare text" is not a fine-tune. It is a different problem. What I got from the Naamapadam run was a tokenizer and a set of contextual embeddings that had seen eleven scripts of PER/LOC/ORG data, which was a warm start for the multilingual half of the hackathon pipeline. The other half was rule-based, spaCy, Presidio, and DeBERTa-v3 fine-tuned separately.
The GE Healthcare Hackathon sibling repo
The private companion repo is GE-Healthcare-Hackathon-PII. It has the same PII Fine tuning.ipynb at the root, mirrored from the same Kaggle notebook, and then it has the rest of the stack around it: a models/ directory with subfolders for DeBERTa, PHI, PII, QLoRA, rule-based, spaCyNER, and presidio; multiprocessing/ and multithreading/ scripts to run the ensemble in parallel over a document stream; a Flask backend and a frontend under website/; a schema.sql and a test.db; and a recognized_entities.md that lists which of the sixty-plus classes each subsystem is responsible for. AADHAAR is rule-based. IBAN is the PII model. Disease is PHI. Event and Language are spaCy. City and Company_Name are DeBERTa. Money is spaCy but Currency_Symbol is DeBERTa, which is the kind of decision that makes sense only once you have looked at where each subsystem was actually right.
The hackathon team, Sliverine Artana, made the top twelve at the GE Healthcare finals. The public Naamapadam notebook is the one visible artifact from that work. The full stack around it stays private because the entity classes and the routing table were the actual contribution.
Why the two posts exist
The retrospective post argues, from the SAP German-PII vantage point, that tokenizer coverage on a target language beats parameter breadth across many languages, and that XLM-R lost to DeBERTa-base on the specific compound-noun identifier classes that Bundesdatenschutzgesetz treats as sensitive. That argument is defensible because I have the eval set and the failure-mode notes to back it. It is a clean claim.
The notebook is the same shape of experiment on a different corpus. Naamapadam, eleven Indic scripts, PER/LOC/ORG. The lesson traveled: when you tokenize non-Latin scripts through a base model, the tokenizer choice dominates. That is the shared spine between the two artifacts. Everything else in the notebook, the numbered steps, the redundant reloads, the shutil.rmtree, is the texture of doing the work.
If you want the argument, read the retrospective. If you want the code, the notebook is public. Both of them are true. The notebook is just less flattering to me, which is why it needed its own post.
See also
- /blog/deberta-over-xlmr-german-pii, the polished retrospective this notebook is behind.
- /blog/sap-rag-privacy-full-stack-architecture, the RAG system that used the resulting PII model.
- /blog/redact-at-retrieval-gdpr-rag, the redaction architecture in the same system.