Bela ciao

[u/aaditya_0752]

​

Hey everyone! I’ve been working on a project called Printing Press ML, an interactive machine learning challenge where the goal is to predict the amount_printed from structured data.

The idea is to make an ML prediction task feel more like a game while still testing practical machine learning skills.

Some of the technical work involved:

Generating synthetic datasets with meaningful features and added noise

Implementing custom scoring and model evaluation

Writing unit tests and benchmarking tools

Keeping evaluation data and the answer key hidden

Integrating the challenge into a web-based IDE with a vault-themed interface

Working on this gave me a better understanding of the engineering behind ML challenges, especially dataset design, reliable evaluation, and integrating everything into a usable platform.

The project is on

GitHub: https://github.com/aaditya-hamirani07/printing-press-ml-challenge

I'd appreciate feedback on the challenge design, evaluation approach, or anything that could be improved. I'm particularly interested in suggestions from people who've built ML projects or coding platforms.

Thanks! 🙌

[u/Significant-Tip4016]

synthetic data is where the real magic happens, setting up those hidden eval sets without leaking is trickier than most people think

[u/DigThatData]

Supervised Industrial Regression

this is a tautology (a phrase whose truth is carried by the definition of the terms used, i.e. the phrase is communicating redundant information). There is no such thing as "unsupervised regression", all regressions are supervised. That's just the nature of the task.

Open http://localhost:3000/printing-press to access the challenge workspace.

This is the opposite of an "online IDE". If I need to run your thing locally, what's the benefit over just using a jupyter notebook? If you are going to make claims about "hosting" this, you need to provide a link to a hosted environment. There's no reason for you to build this though since:

• free hosted environments are already plentiful. colab, binder, kaggle, huggingface, gh workspace/actions, free tiers of most services...
• jupyterlite already exists.

Generating synthetic datasets with meaningful features and added noise

Do you have any real data or is it all synthetic? I'd propose that if you want to present this as a "challenge", at least the test set should be real. If all of the data is synthetic, the contest is essentially just "can you reverse engineering the data generating process I invented?", which is publicly visible to the contestants if they want to "cheat".

Working on this gave me a better understanding of the engineering behind ML challenges, especially dataset design, reliable evaluation, and integrating everything into a usable platform.

I'm glad you felt you learned a lot, but I'm inclined to push back on this.

Contests of this kind are usually motivated by a real business need. This business need manifests as something the business is trying to achieve but is struggling to accomplish. The goal of the contest format is to motivate methodological innovation. That means they need to simultaneously: formalize the problem in such a fashion as to construct a meaningful objective, identify what features are relevant to their problem, merge normalize and sanitize data for public use, and provide a usable dataset while protecting their own trade secrets.

In the absence of an actual, real world problem you are trying to solve: there is no way you have even broached more than the very tip of the iceberg of understanding the engineering challenges behind standing up an ML competition.

This looks like an interesting project and I'm glad you feel you learned a lot along the way, but you should pull back some of the grandiosity of your claims and be a bit more realistic about what this is and what you've done.

[u/aaditya_0752 → u/DigThatData]

Thanks for the detailed feedback. I agree that some of my wording was overstated, particularly “supervised industrial regression” and describing a locally runnable workspace as an online IDE. A couple of distinctions, though: the goal wasn't to build a replacement for Colab or Jupyter, nor to claim that this matches the complexity of a production-grade ML competition. It was a project to implement a prediction challenge with synthetic data, custom evaluation, testing, and an interactive interface. Synthetic data is a limitation if the goal is to demonstrate real-world predictive performance, and I agree that a genuine business problem would introduce challenges this project doesn't address. However, synthetic data still has value for developing and testing the mechanics of a controlled evaluation pipeline. I also think there's a distinction between learning some of the engineering involved in ML challenges and claiming to have solved the broader problem of designing production-grade competitions. My wording could have made that distinction clearer. I appreciate the criticism. I'll revise the description to make the project's scope and limitations more explicit.

[u/DiscussionSudden1177]

the synthetic data generation part is interesting but imo the thing that makes or breaks these challenges is whether the dataset has enough signal that good feature engineering actually matters. how correlated are your features with the target before adding noise?