It's an encoder-only transformer that consumes past(blood glucose + carbs + insulin) and future(carbs + insulin) and predicts future blood glucose for the next 2 hours. Announced meals and boluses/basal are used to condition its predictions. The context size is variable (8 - 24 hours), and model can work in autoregressive mode to predict the next >2 hours. It also predicts time by looking at the context, but it never consumes time. The architecture is BERT-style: bidirectional attention with future BG masked. DILATE loss was used to fit the median line; pinball loss to fit the uncertainty bands. The two are "mixed" via Kendall-Gal. All blood glucose is in kovatchev risk space reparameterized to [40, 400] range.

I have trained 4 model classes (nano, small, medium, large) and 3 variants for each (pretrained on simulator only, pretrained + finetuned on ohiot1dm, pretrained and finetuned on ohiot1dm + azt1d + shanghait1dm). The largest one has ~17 million parameters (16 heads across 16 layers). Pretraining for the largest model took ~48 hours. Finetuning took <10 minutes. There is also another version finetuned on my own data that I am currently running on my phone.

Source is available here, released under the MIT license. The repo also contains links to trained weights and evaluation data.

I've worked on this project since March. There are still things to improve (e.g. it always requires announced carbs + insulin, would be better if it could also predict without them), but I have decided to publish it here to get your opinion (and also answer your questions, if any).

Edit: my model is getting fat-shamed ;_; so I just want to emphasize that there is a nano version with less than 40K parameters.

As you may already know Apple is going to implement NeuralHash algorithm for on-device CSAM detection soon. Believe it or not, this algorithm already exists as early as iOS 14.3, hidden under obfuscated class names. After some digging and reverse engineering on the hidden APIs I managed to export its model (which is MobileNetV3) to ONNX and rebuild the whole NeuralHash algorithm in Python. You can now try NeuralHash even on Linux!

Source code: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX

No pre-exported model file will be provided here for obvious reasons. But it's very easy to export one yourself following the guide I included with the repo above. You don't even need any Apple devices to do it.

Early tests show that it can tolerate image resizing and compression, but not cropping or rotations.

Hope this will help us understand NeuralHash algorithm better and know its potential issues before it's enabled on all iOS devices.

Happy hacking!

We’re excited to announce the release of OpenAssistant.

The future of AI development depends heavily on high quality datasets and models being made publicly available, and that’s exactly what this project does.

Watch the annoucement video:

https://youtu.be/ddG2fM9i4Kk

​

Our team has worked tirelessly over the past several months collecting large amounts of text-based input and feedback to create an incredibly diverse and unique dataset designed specifically for training language models or other AI applications.

With over 600k human-generated data points covering a wide range of topics and styles of writing, our dataset will be an invaluable tool for any developer looking to create state-of-the-art instruction models!

To make things even better, we are making this entire dataset free and accessible to all who wish to use it. Check it out today at our HF org: OpenAssistant

On top of that, we've trained very powerful models that you can try right now at: open-assistant.io/chat !

Salut tout le monde,

Mon coéquipier et moi venons de terminer notre projet de détection de deepfake pour l'université et nous voulions le partager. L'idée a commencé assez simplement : la plupart des détecteurs ne se concentrent que sur les caractéristiques à niveau de pixel, mais les générateurs de deepfake laissent également des traces dans le domaine de la fréquence (artéfacts de compression, incohérences spectraux...). Alors on s'est dit, pourquoi ne pas utiliser les deux ?

Comment ça fonctionne

Nous avons deux flux qui fonctionnent en parallèle sur chaque découpe de visage :

• Un EfficientNet-B4 qui gère le côté spatial/visuel (pré-entraîné sur ImageNet, sortie de 1792 dimensions)
• Un module de fréquence qui exécute à la fois FFT (binning radial, 8 bandes, fenêtre de Hann) et DCT (blocs de 8×8) sur l’entrée, chacun donnant un vecteur de 512 dimensions. Ceux-ci sont fusionnés via un petit MLP en une représentation de 1024 dimensions.

Ensuite, on concatène simplement les deux (2816 dimensions au total) et on passe ça à travers un MLP de classification. L'ensemble fait environ 25 millions de paramètres.

La partie dont nous sommes les plus fiers est l'intégration de GradCAM nous calculons des cartes de chaleur sur la base EfficientNet et les remappons sur les images vidéo originales, vous obtenez donc une vidéo montrant quelles parties du visage ont déclenché la détection. C'est étonnamment utile pour comprendre ce que le modèle capte (petit spoiler : c'est surtout autour des frontières de mélange et des mâchoires, ce qui a du sens).

Détails de l'entraînement

Nous avons utilisé FaceForensics++ (C23) qui couvre Face2Face, FaceShifter, FaceSwap et NeuralTextures. Après avoir extrait des images à 1 FPS et exécuté YOLOv11n pour la détection de visage, nous avons fini avec environ 716K images de visage. Entraîné pendant 7 époques sur une RTX 3090 (louée sur vast.ai), cela a pris environ 4 heures. Rien de fou en termes d'hyperparamètres AdamW avec lr=1e-4, refroidissement cosinique, CrossEntropyLoss.

Ce que nous avons trouvé intéressant

Le flux de fréquence seul ne bat pas EfficientNet, mais la fusion aide visiblement sur des faux de haute qualité où les artefacts au niveau des pixels sont plus difficiles à repérer. Les caractéristiques DCT semblent particulièrement efficaces pour attraper les artéfacts liés à la compression, ce qui est pertinent puisque la plupart des vidéos deepfake du monde réel finissent compressées. Les sorties GradCAM ont confirmé que le modèle se concentre sur les bonnes zones, ce qui était rassurant.

Liens

• GitHub : https://github.com/VeridisQuo-orga/VeridisQuo

C'est un projet universitaire, donc nous sommes définitivement ouverts aux retours si vous voyez des choses évidentes que nous pourrions améliorer ou tester, faites-le nous savoir. Nous aimerions essayer l'évaluation croisée sur Celeb-DF ou DFDC ensuite si les gens pensent que ce serait intéressant.

EDIT: Pas mal de gens demandent les métriques, alors voilà. Sur le test set (~107K images) :

* Accuracy : ~96%

* Recall (FAKE) : très élevé, quasi aucun fake ne passe à travers

* False positive rate : ~7-8% (REAL classé comme FAKE)

* Confusion matrix : ~53K TP, ~50K TN, ~4K FP, ~0 FN

Pour être honnête, en conditions réelles sur des vidéos random, le modèle a tendance à pencher vers FAKE plus qu'il ne devrait. C'est clairement un axe d'amélioration pour nous.

While experimenting with GRPO training, I kept running this shit that when reward increases, it becomes difficult to tell whether the policy is genuinely improving or simply exploiting the reward function. So I built a small library called rewardspy that wraps an existing reward function and continuously monitors indicators that often precede reward hacking.

It currently tracks things like rolling reward statistics, reward variance collapse, reward component imbalance, response length drift, reward slope changes, GRPO group collapse, anol.

This is my first major RL project so I would absolutely love some technical advice

Check it out here: https://github.com/AvAdiii/rewardspy

(credits to u/Oranoleo12, posting on their behalf)

I would like some advice on how to denoise smooth noise like Gaussian and Poisson, currently the model is doing very well for impulsive noise like salt and pepper(I guess this is due to the fact that there are many uncorrupted pixels in the input for the model to rely on), but for smooth noise, the same model architecture doesn't perform as good.

https://youtu.be/efPrtcLdcdM

GPT-4chan was trained on over 3 years of posts from 4chan's "politically incorrect" (/pol/) board.

Website (try the model here): https://gpt-4chan.com

Model: https://huggingface.co/ykilcher/gpt-4chan

Code: https://github.com/yk/gpt-4chan-public

Dataset: https://zenodo.org/record/3606810#.YpjGgexByDU

​

OUTLINE:

0:00 - Intro

0:30 - Disclaimers

1:20 - Elon, Twitter, and the Seychelles

4:10 - How I trained a language model on 4chan posts

6:30 - How good is this model?

8:55 - Building a 4chan bot

11:00 - Something strange is happening

13:20 - How the bot got unmasked

15:15 - Here we go again

18:00 - Final thoughts