I chose markdown for a large project due to the reduce storage size and forced simplicity and abstraction.

I have to work with static data which will not change for 10 years, the only way to keep it alive is upcasting the internal structure and updating the conversion to html.

The project lives in a scientific environment but is used by a wide variety of people.

The project is now 5 years old and until next week there was no wysiwyg-editor (I'm currently working on it).

What kind of project is it that the storage size of text documents is a big deal? And why don't you use compression?

It's an electronic testing and examination environment for German universities.

It's very far from big data, but we have to support very small universities with very limited tech budget. We pre-generate and store the exams to ensure that we do not have any compatibility issues in the future (e.g. by changing minute elements of the randomisation).

The biggest database currently is about 700GB and I assume that we'll reach an equilibrium at around 2 TB.

As someone who works in a central service unit not directly affiliated with academic research, I was given the chance to touch the lives of probably generations of students.

Emphasis on "touch", not necessarily "change". Still, it sometimes gives me goosebumps when walking through the halls and seeing all the people who are "forced" to use my work.

Pay is not great, but conditions are otherwise excellent.

(Founder/ Dev/ Lead of an electronic exam infrastructure at a large German university)

I'm quite curious about this myself...

I would like to extend the question by asking the owners of event-sourced systems how they handle it.

In my case, we will have to do manuell, software-assisted deletions

I have recently created a small project called rocket (https://github.com/joernlenoch/rocket) to provide myself with a simple dependency injection module for exactly this reason.

This is rather a starting point than a solution for your problem. But combined with some "fakers" and supplied testing frameworks this might remove some of the boilerplate.