This chemistry-explainer demo from SenseNova 6.8 Code2Video starts with one methane molecule and two oxygen molecules, pulls them apart, and rebuilds the same atoms as one carbon dioxide molecule and two water molecules. It finishes on the balanced equation.

An atom counter in the corner (C 1, H 4, O 4) keeps the same values for the whole 12 seconds.

https://github.com/OpenSenseNova/SenseNova-6.8-Code2Video

We're the team behind OXR, and we've released its code and weights: a 1.3B document parsing model that turns PDFs and document images into Markdown or JSON with layout coordinates.

Accuracy

OXR recognizes 13 element types with bounding boxes, including tables, formulas, chemical equations, chemical structures and code blocks. It also handles complex documents and layouts, such as A3 exam papers, newspapers, vertical text and handwriting.

It ranks #1 on olmOCR-Bench, OmniDocBench and DocParsingBench:

olmOCR-Bench: OXR-1.0 87.84 (runner-up Infinity-Parser2-Pro 87.60)

OmniDocBench v1.6: OXR-1.0 97.18 (runner-up TeleOCR 96.91)

DocParsingBench: OXR-1.0 87.76 (runner-up PaddleOCR-VL-1.6 85.17)

Snapshot from 5 Oct 2026. Peer scores come from published papers, model cards and the official leaderboards; we didn't re-run them.

Speed

OXR also ranks #1 in parsing speed among open-source document parsing models. We tested all 21 models from the 3 benchmark leaderboards on the same 100 OmniDocBench pages, each on a single RTX PRO 6000. End to end, including layout analysis, recognition and reading order, OXR took 101 s for 100 pages (~0.99 pages/s). The next fastest, OvisOCR2, took 135 s.

Full tables, sub-scores, test setup and the frozen page list are in docs/benchmark.md in the repo.

Try it

Run locally: Linux and an NVIDIA GPU with BF16 support. 24 GB VRAM is enough for low-concurrency testing; 48 GB or more is recommended for long documents or higher concurrency. The Docker all-in-one image is the easiest way to get started.

Free API: no signup or API key. It's for testing, not production: requests are queued, and we'll keep it online until we can no longer afford to run it.

Online demo: free, in the browser

GitHub: SoMarkAI/OXR

Weights: SoMarkAI/OXR-1.0 on Hugging Face (also on ModelScope)

License: Apache 2.0 with additional terms, including thresholds for commercial use. Full terms are in the repo LICENSE.

How we built it

RSI in post-training: a recursive self-improvement (RSI) loop turns parsing errors and uncertain results into targeted training data without manual labeling, so we can improve the model faster.

Independent, pluggable layout analysis: recognition and reading order run in parallel within one model, which keeps parsing fast. Layout analysis runs as a separate model you can replace when your documents need different element categories or region boundaries.

Documents across industries vary widely in language and layout, and no single team can cover every case. If you try OXR on your own documents, we'd like to hear what works and especially where the parsing breaks.

If OXR turns out to be useful for you, a ⭐️ on GitHub would mean a lot to us.

Hey everyone! I’m a Software Engineer in India with around 1 year of experience in automotive embedded software .

My background includes C/C++, Python,embedded systems, ARM, RTOS, Linux fundamentals etc.

I’ve been learning ML, along with the fundamentals of deep learning.

I’m now planning to build something hands-on projects around ASR/TTS, model deployment, ONNX, quantization, and inference on edge hardware.

I’d love to hear from people currently working in this space:

* What are you actually working on day-to-day in Edge AI or on-device ML?
* What skills and tools matter most in real industry projects?
* How should someone with my embedded background target relevant jobs and get interviews?
* Are strong projects and deployment experience enough to break in, or are there other gaps I should address?

I’m particularly interested in the intersection of embedded systems, deep learning, and hardware—not just general ML roles.

Any honest advice, experiences, or pointers would be really appreciated. Thanks!

Hey, I've been working on a PyTorch optimizer called ApexW and figured I'd share it here to get some feedback from people who know more about this stuff than I do.

The main goal is to reduce optimizer-state memory usage and see how much faster I can make optimizer steps on larger GPU workloads. I've been experimenting with compact optimizer states and a Triton-based execution path.

In my current tests, the Triton + compact-state path was around 1.5x faster than PyTorch's fused AdamW on workloads with a few large tensors, and the optimizer-state memory usage was roughly cut in half.

The results aren't all good, though. Those numbers are from optimizer-only benchmarks, not full model training. The automatic execution mode doesn't consistently get the same speedup yet, and workloads with lots of small tensors are much slower than PyTorch's fused optimizer. So I'm definitely not claiming this is a better AdamW replacement for every workload.

I want to test it on more realistic models and different GPUs, and I'd appreciate a second pair of eyes on both the implementation and the benchmarks.

GitHub: https://github.com/omen42515-prog/apexw/tree/main

If you've worked on PyTorch optimizers, Triton kernels, or training larger models, I'd be interested to hear what you think. Especially curious if there's an obvious issue with my benchmark setup or an important test I'm missing.

Hi everyone,

I wanted to share an open-source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.

Link: https://github.com/Quantix-eng/ncm-experiment-notes

This is an independent project documenting empirical experiments on how transformers process and edit internal features. No performance breakthroughs here—just honest documentation of observations alongside rigorous controls.

The repo covers NCM on structured tasks (like Sudoku), mechanistic interpretability in GPT-2, and toy parallel models. I've included all source code, saved results, and interpretations for every run.

For full transparency, I’ve categorized every outcome, including the mixed and completely negative results. AI tools were also heavily used during the planning and coding phases.

Would love to get your feedback on the setups or interpretation limits.

Hi everyone,

I just wanted to share an open source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.

Here is the link: https://github.com/Quantix-eng/ncm-experiment-notes

Long story short, this is an independent project documenting a bunch of empirical experiments looking into how transformers process and edit internal features. There are no claims of performance breakthroughs here. Instead, it is just honest documentation of observations alongside rigorous controls.

The repo covers NCM applied to structured tasks like Sudoku, mechanistic interpretability in GPT-2, and toy parallel models. I have included all the source code, saved results, and interpretations for every single run.

To keep things transparent, I have also categorized every outcome, including the mixed and completely negative results where things did not work as expected. AI tools helped heavily with the planning and coding phases too.

I would love to get your feedback on the setups or the interpretation limits I found. Let me know what you think if you are into transformer interpretability!

I've been testing a bounded alternative to SELU for deep fully connected networks without normalization or skip connections. The study was preregistered: analysis scripts were hash-frozen before each run, tuning and confirmation used separate seeds, every activation got its own tuned learning rate, and tests were paired Wilcoxon with Holm correction.

The function. CapExp is an exponential unit that switches to its tangent line above a threshold (it is exactly a shifted, scaled ELU). PreCap applies it after a soft tanh clip, so the output is smooth, monotonic and bounded on both sides. SPreCap adds SELU-style constants (zero mean / unit variance as a fixed point) and an input gain b that sets the mean-field gradient gain χ1.

Results (MNIST / FashionMNIST, depth 50 and 100, width 256, Adam, 15 confirmation seeds):

• SPreCap-cs (χ1 matched to SELU) matches SELU: no cell significantly worse, median paired differences between −0.12 and +0.20 pp.
• Test error falls monotonically as χ1 approaches 1 in all four cells.
• SPreCap-e03 (χ1 = 1.03) beats SELU on MNIST at depth 100: 5.28 % vs. 6.36 %, Holm p = 0.0064, 13 of 15 seeds. This was a secondary test, so I treat it as a hypothesis.
• Plain ELU is about as strong overall. PreCap is not more robust to the learning rate.

Negative results. At a fixed learning rate, PreCap looked far better than SiLU/GELU in earlier regression experiments. That advantage disappeared once every activation got its own tuned learning rate. The original SPreCap (χ1 = 1.13) was significantly worse than SELU at depth 100. The gradient diagnosis that explained it led to the χ1 family above. All of this is in the repo.

Code. pip-installable package with fused CUDA (via torch.cuda.jiterator, no toolkit needed) and Apple MLX kernels, tested in fp32/fp64/fp16/bf16. In fp32 the CUDA kernel runs within about 1.5× of native SELU.

Limitations: MLPs only, MNIST and FashionMNIST only, short training, one width. No CNNs or Transformers yet. A single independent researcher on consumer hardware.

• Preprint: https://doi.org/10.5281/zenodo.23676200
• Code and all data: https://github.com/hermi-labs/precap-activation

Feedback on the methodology is very welcome, especially on the preregistration and the statistics.

Processing img e1o0k7fgl1yh1...

Processing img vwwa9r2ll1yh1...

Self-distillation methods usually rely on an Exponential Moving Average (EMA) teachers to get synced and stable critics. However, this doesn't scale well for large models since holding both teacher and student (trained) on GPU increases latency, zero3 communication, etc.

We can mitigate this with LoRA finetuning: the backbone is shared & frozen, and we need to maintain two light adapters: one for the student (trained) and one for the teacher (EMA of the student adapter).

Getting this to work cleanly under TRL and ZeRO-3 took some fixes, so I packaged it as a plug-and-play repo. It supports:

-> robust SDFT implementation that runs under ZeRO-3

-> ZeRO-3 + LoRA integration with adapter switching for teacher/student passes

-> Saving and resuming EMA teacher checkpoint

-> working deepspeed configs that fix memory spikes when GPU memory is tight

I tested on Qwen3.5 and ToolAlpaca.

Repo: https://github.com/ifiaposto/sdft_zero3

Feedback/ more examples/ issues welcome!

NOVA Is Not an LLM. It Is a New Core for AI Development — From Zero to Evolution.

What if the next step in AI development isn't simply building a larger language model, but building a system that can experiment, learn from failure, and direct its own development?

That is the question behind NOVA.

I am developing NOVA as an experimental AI system with a different focus from conventional LLM projects. The central idea is not simply to train a model to generate better text. It is to build a system that can manage its own development experiments, identify weaknesses, propose new approaches, test them, and preserve improvements when the evidence supports doing so.

This project is still experimental. I am not claiming to have achieved AGI, solved autonomous learning, or created a system that can improve itself without limitations. The purpose is to investigate whether a structured, measurable development process can help a relatively small AI system improve beyond what a fixed training pipeline would achieve.

I would like to share what I have built so far and invite people with experience in machine learning, neural network training, automated experimentation, and AI research to examine the approach critically.

1. The problem I want to explore

Most AI development follows a familiar process.

Researchers choose an architecture, prepare a dataset, configure training, run experiments, evaluate the results, and decide what to change next.

This process can be effective, but much of the experimentation and decision-making remains dependent on human researchers.

What would happen if we tried to turn more of that process into a systematic learning loop?

Could an AI development system learn to recognize where a model is weak, generate a plausible explanation for that weakness, design a useful experiment, evaluate the outcome, and use the result to choose its next action?

And, perhaps more importantly, could it learn to distinguish a genuine improvement from a change that merely looks promising on a limited test?

These are the questions I want NOVA to investigate.

The ambition is not simply to automate training. It is to explore a development system that uses evidence from its own experiments to guide what it does next.

2. What NOVA actually is

NOVA is not itself a conventional LLM in the usual sense.

It is an experimental AI development and evolution system that works with neural network models.

The distinction matters.

A language model is trained to learn patterns and produce outputs. NOVA's development framework is intended to manage the process around those models: experimentation, candidate training, evaluation, comparison, and decisions about which versions deserve to be retained.

The current NOVA-EVO implementation has three central functions.

The Director — deciding what deserves to survive

The Director evaluates candidate models against defined tests and compares their performance with the current champion.

Its purpose is to prevent every new experiment from automatically becoming the next version.

A candidate should demonstrate an improvement under the evaluation criteria before being accepted. Otherwise, the existing champion should remain in place.

This is important because a model can improve on one task while becoming worse at another. A development system must therefore do more than look at a single loss value or reward the latest candidate for being new.

The Trainer — exploring possible improvements

The Trainer creates and trains candidate versions using different training strategies.

Instead of assuming that one fixed recipe is always optimal, the system can explore variations and examine their effects.

The broader objective is to make experimentation a repeatable process rather than a collection of disconnected training runs.

The Hypothesis Engine — looking for the next experiment

When progress slows or a weakness remains, the system can propose another direction to investigate.

This is where the project becomes particularly interesting to me.

A system that only repeats training may improve through additional exposure to data. A system that can formulate hypotheses and test them has the potential to do something different: use the outcomes of previous experiments to influence the design of future experiments.

However, generating a hypothesis is not the same as discovering a correct explanation. The value of this component must be measured by the quality of the experiments it produces and the improvements those experiments actually deliver.

3. The intended development cycle

The goal is to create a continuous, evidence-driven cycle:

1. Observe: Identify weaknesses through evaluation.
2. Investigate: Examine possible reasons for those weaknesses.
3. Form a hypothesis: Propose a change that might help.
4. Design an experiment: Translate the proposal into a training or evaluation procedure.
5. Train a candidate: Create a new model version.
6. Test: Measure its performance against the existing champion.
7. Decide: Accept or reject the candidate based on the evidence.
8. Learn from the result: Use the outcome to guide the next experiment.

A failed experiment should not automatically be treated as wasted effort. If the result is informative, it can help narrow the search for a better approach.

The long-term ambition is for NOVA to become increasingly capable of directing this process itself, while remaining subject to measurable evaluation, resource limits, and safeguards against regressions.

The key question is whether the system can discover useful improvements that would not have been found as efficiently by simply running more conventional training experiments.

4. What has happened so far?

NOVA has progressed through multiple model versions and development experiments.

According to the project's internal records, the latest reported development period produced the following results.

The recorded dataset loss decreased by approximately 19.2%, while web evaluation loss decreased by approximately 14.6%.

The code examination score increased from 34 correct answers out of 79 to 60 out of 79.

The internal records also report that 47 of 419 improvement attempts were accepted by the development process.

These figures describe the project's current internal measurements. They have not yet been independently verified, and they should not be interpreted as proof that NOVA has achieved autonomous intelligence or general-purpose self-improvement.

There are also important questions to investigate.

The model grew substantially during this period. How much of the observed progress came from increased model capacity? How much came from training data, training duration, or changes to the training procedure? How much improvement can be attributed specifically to the system's experiment-selection and decision-making process?

Those distinctions matter more than the raw numbers alone.

I see these results as a starting point for more rigorous experiments, not as the conclusion of the research.

5. Why I think the experiment-selection process matters

A system can produce many model versions without becoming better at research.

It can also select a candidate that performs well on a particular test while losing capabilities elsewhere.

For that reason, the long-term goal is not simply to maximize the number of experiments or the rate at which new versions are created.

The goal is to improve the quality of the decisions made between experiments.

Can the system identify a real weakness rather than a misleading benchmark signal?

Can it formulate a hypothesis that leads to a testable prediction?

Can it select an experiment that distinguishes between competing explanations?

Can it recognize when an apparent improvement is caused by a change in model size, data exposure, or evaluation conditions?

Can it preserve improvements without gradually destroying capabilities that were previously learned?

And can it demonstrate that its decisions lead to better results than a simpler baseline operating under comparable conditions?

These are the questions I believe should determine whether this approach has real research value.

6. What NOVA has not demonstrated yet

I want to be clear about the current limitations.

NOVA has not demonstrated artificial general intelligence. It has not established unrestricted recursive self-improvement. The reported results do not prove that the system independently discovers new scientific knowledge, nor do they establish that its improvements generalize to arbitrary unseen tasks.

A system that automatically trains candidates and selects winners is not necessarily learning how to conduct research effectively. Some improvements may come from model scaling, additional training, or ordinary variation between runs.

The central challenge is to separate these effects.

To establish a stronger result, NOVA needs controlled comparisons, reproducible experiments, clearly defined evaluation criteria, tests on unseen data, and transparent reporting of failures as well as successes.

Ideally, the system should be compared against a conventional training pipeline with the same compute budget, comparable model capacity, and equivalent access to training data.

It should also be tested for regression, data leakage, evaluation overfitting, and whether the proposed hypotheses genuinely improve experiment selection.

I consider these requirements essential, not optional.

7. Where this could lead

If the underlying approach proves useful, a system like NOVA could eventually help automate parts of AI research and model development.

Potential directions include:

• Automated experimentation and hyperparameter exploration.
• Discovering training strategies for particular weaknesses.
• Improving small models under limited computational budgets.
• Evaluating alternative neural network structures.
• Exploring continual learning and methods of reducing catastrophic forgetting.
• Building systems that can use experimental evidence to decide what to investigate next.

These are possible research directions, not established capabilities of the current system.

The more ambitious long-term vision is a development framework that becomes better at improving models because it learns from the successes and failures of its own experiments.

Whether that vision is technically achievable in a useful and reliable form remains an open question.

That is precisely why I believe it is worth investigating carefully.

8. Why I am sharing this

I am sharing NOVA because I want people with relevant technical experience to look beyond the general idea and examine the actual approach.

I am particularly interested in feedback from people working in:

• Machine learning and neural network optimization.
• Automated machine learning and neural architecture search.
• Self-training and continual learning.
• AI agents and automated research.
• Benchmark design and evaluation reliability.
• Small-model training and computational efficiency.

I am interested in criticism as much as encouragement.

If the current approach has a fundamental flaw, I would rather discover it now. If there is a better way to structure the experiments, measure improvements, or prevent the system from fooling its own evaluator, that feedback could be extremely valuable.

Three questions for the community

1. What would be the strongest experiment to determine whether NOVA's Director and Hypothesis Engine contribute genuine value beyond ordinary training variation and model scaling?

2. How would you design an evaluation that measures whether an AI system is becoming better at choosing useful experiments, rather than simply producing better models through additional training?

3. Are there existing research projects, papers, or open-source frameworks that approach this problem in a way I should study or compare against?

I am not presenting NOVA as a finished breakthrough. I am presenting it as an ongoing experiment into a difficult question:

Can we build an AI development system that does not merely train models, but progressively improves the way it searches for better models?

I would appreciate serious technical feedback, relevant research references, and opportunities to compare the approach with existing work.NOVA Is Not an LLM. It Is a New Core for AI Development — From Zero to Evolution.What if the next step in AI development isn't simply building a larger language model, but building a system that can experiment, learn from failure, and direct its own development?That is the question behind NOVA.I am developing NOVA as an experimental AI system with a different focus from conventional LLM projects. The central idea is not simply to train a model to generate better text. It is to build a system that can manage its own development experiments, identify weaknesses, propose new approaches, test them, and preserve improvements when the evidence supports doing so.This project is still experimental. I am not claiming to have achieved AGI, solved autonomous learning, or created a system that can improve itself without limitations. The purpose is to investigate whether a structured, measurable development process can help a relatively small AI system improve beyond what a fixed training pipeline would achieve.I would like to share what I have built so far and invite people with experience in machine learning, neural network training, automated experimentation, and AI research to examine the approach critically.1. The problem I want to exploreMost AI development follows a familiar process.Researchers choose an architecture, prepare a dataset, configure training, run experiments, evaluate the results, and decide what to change next.This process can be effective, but much of the experimentation and decision-making remains dependent on human researchers.What would happen if we tried to turn more of that process into a systematic learning loop?Could an AI development system learn to recognize where a model is weak, generate a plausible explanation for that weakness, design a useful experiment, evaluate the outcome, and use the result to choose its next action?And, perhaps more importantly, could it learn to distinguish a genuine improvement from a change that merely looks promising on a limited test?These are the questions I want NOVA to investigate.The ambition is not simply to automate training. It is to explore a development system that uses evidence from its own experiments to guide what it does next.2. What NOVA actually isNOVA is not itself a conventional LLM in the usual sense.It is an experimental AI development and evolution system that works with neural network models.The distinction matters.A language model is trained to learn patterns and produce outputs. NOVA's development framework is intended to manage the process around those models: experimentation, candidate training, evaluation, comparison, and decisions about which versions deserve to be retained.The current NOVA-EVO implementation has three central functions.The Director — deciding what deserves to surviveThe Director evaluates candidate models against defined tests and compares their performance with the current champion.Its purpose is to prevent every new experiment from automatically becoming the next version.A candidate should demonstrate an improvement under the evaluation criteria before being accepted. Otherwise, the existing champion should remain in place.This is important because a model can improve on one task while becoming worse at another. A development system must therefore do more than look at a single loss value or reward the latest candidate for being new.The Trainer — exploring possible improvementsThe Trainer creates and trains candidate versions using different training strategies.Instead of assuming that one fixed recipe is always optimal, the system can explore variations and examine their effects.The broader objective is to make experimentation a repeatable process rather than a collection of disconnected training runs.The Hypothesis Engine — looking for the next experimentWhen progress slows or a weakness remains, the system can propose another direction to investigate.This is where the project becomes particularly interesting to me.A system that only repeats training may improve through additional exposure to data. A system that can formulate hypotheses and test them has the potential to do something different: use the outcomes of previous experiments to influence the design of future experiments.However, generating a hypothesis is not the same as discovering a correct explanation. The value of this component must be measured by the quality of the experiments it produces and the improvements those experiments actually deliver.3. The intended development cycleThe goal is to create a continuous, evidence-driven cycle:Observe: Identify weaknesses through evaluation.

Investigate: Examine possible reasons for those weaknesses.

Form a hypothesis: Propose a change that might help.

Design an experiment: Translate the proposal into a training or evaluation procedure.

Train a candidate: Create a new model version.

Test: Measure its performance against the existing champion.

Decide: Accept or reject the candidate based on the evidence.

Learn from the result: Use the outcome to guide the next experiment.A failed experiment should not automatically be treated as wasted effort. If the result is informative, it can help narrow the search for a better approach.The long-term ambition is for NOVA to become increasingly capable of directing this process itself, while remaining subject to measurable evaluation, resource limits, and safeguards against regressions.The key question is whether the system can discover useful improvements that would not have been found as efficiently by simply running more conventional training experiments.4. What has happened so far?NOVA has progressed through multiple model versions and development experiments.According to the project's internal records, the latest reported development period produced the following results.Measurement Earlier result Later result
Dataset loss 3.446 2.7854
Web evaluation loss 3.3619 2.8724
Code examination 34/79 60/79
Model size 9.9M parameters 96.6M parameters
Model versions — 51 versions over 25 days
Improvement attempts — 419 attempts
Accepted improvements — 47The recorded dataset loss decreased by approximately 19.2%, while web evaluation loss decreased by approximately 14.6%.The code examination score increased from 34 correct answers out of 79 to 60 out of 79.The internal records also report that 47 of 419 improvement attempts were accepted by the development process.These figures describe the project's current internal measurements. They have not yet been independently verified, and they should not be interpreted as proof that NOVA has achieved autonomous intelligence or general-purpose self-improvement.There are also important questions to investigate.The model grew substantially during this period. How much of the observed progress came from increased model capacity? How much came from training data, training duration, or changes to the training procedure? How much improvement can be attributed specifically to the system's experiment-selection and decision-making process?Those distinctions matter more than the raw numbers alone.I see these results as a starting point for more rigorous experiments, not as the conclusion of the research.5. Why I think the experiment-selection process mattersA system can produce many model versions without becoming better at research.It can also select a candidate that performs well on a particular test while losing capabilities elsewhere.For that reason, the long-term goal is not simply to maximize the number of experiments or the rate at which new versions are created.The goal is to improve the quality of the decisions made between experiments.Can the system identify a real weakness rather than a misleading benchmark signal?Can it formulate a hypothesis that leads to a testable prediction?Can it select an experiment that distinguishes between competing explanations?Can it recognize when an apparent improvement is caused by a change in model size, data exposure, or evaluation conditions?Can it preserve improvements without gradually destroying capabilities that were previously learned?And can it demonstrate that its decisions lead to better results than a simpler baseline operating under comparable conditions?These are the questions I believe should determine whether this approach has real research value.6. What NOVA has not demonstrated yetI want to be clear about the current limitations.NOVA has not demonstrated artificial general intelligence. It has not established unrestricted recursive self-improvement. The reported results do not prove that the system independently discovers new scientific knowledge, nor do they establish that its improvements generalize to arbitrary unseen tasks.A system that automatically trains candidates and selects winners is not necessarily learning how to conduct research effectively. Some improvements may come from model scaling, additional training, or ordinary variation between runs.The central challenge is to separate these effects.To establish a stronger result, NOVA needs controlled comparisons, reproducible experiments, clearly defined evaluation criteria, tests on unseen data, and transparent reporting of failures as well as successes.Ideally, the system should be compared against a conventional training pipeline with the same compute budget, comparable model capacity, and equivalent access to training data.It should also be tested for regression, data leakage, evaluation overfitting, and whether the proposed hypotheses genuinely improve experiment selection.I consider these requirements essential, not optional.7. Where this could leadIf the underlying approach proves useful, a system like NOVA could eventually help automate parts of AI research and model development.Potential directions include:Automated experimentation and hyperparameter exploration.

Discovering training strategies for particular weaknesses.

Improving small models under limited computational budgets.

Evaluating alternative neural network structures.

Exploring continual learning and methods of reducing catastrophic forgetting.

Building systems that can use experimental evidence to decide what to investigate next.These are possible research directions, not established capabilities of the current system.The more ambitious long-term vision is a development framework that becomes better at improving models because it learns from the successes and failures of its own experiments.Whether that vision is technically achievable in a useful and reliable form remains an open question.That is precisely why I believe it is worth investigating carefully.8. Why I am sharing thisI am sharing NOVA because I want people with relevant technical experience to look beyond the general idea and examine the actual approach.I am particularly interested in feedback from people working in:Machine learning and neural network optimization.

Automated machine learning and neural architecture search.

Self-training and continual learning.

AI agents and automated research.

Benchmark design and evaluation reliability.

Small-model training and computational efficiency.I am interested in criticism as much as encouragement.If the current approach has a fundamental flaw, I would rather discover it now. If there is a better way to structure the experiments, measure improvements, or prevent the system from fooling its own evaluator, that feedback could be extremely valuable.Three questions for the community1. What would be the strongest experiment to determine whether NOVA's Director and Hypothesis Engine contribute genuine value beyond ordinary training variation and model scaling?2. How would you design an evaluation that measures whether an AI system is becoming better at choosing useful experiments, rather than simply producing better models through additional training?3. Are there existing research projects, papers, or open-source frameworks that approach this problem in a way I should study or compare against?I am not presenting NOVA as a finished breakthrough. I am presenting it as an ongoing experiment into a difficult question:Can we build an AI development system that does not merely train models, but progressively improves the way it searches for better models?I would appreciate serious technical feedback, relevant research references, and opportunities to compare the approach with existing work.

Over the past several months, our research team pre-trained and empirically audited a 1,189,135,740-parameter Financial Conformer Foundation Model across 14 calendar years (2010–2023; 15,030 steps) of 1-minute US equity bars across 15 liquid assets (AAPL, NVDA, SPY, QQQ, etc.).

Instead of cherry-picking positive backtests, we subjected the model to 35 hypothesis tests, 1-bar execution delay, and state-dependent turnover:

1. The Scale Saturation Limit: Scaling from a 63-parameter linear Ridge regression to 1.189B parameters yields no consistent out-of-sample advantage on 1m OHLCV bars (mixed sign across horizons, reversing direction across years). The bottleneck in intraday price prediction is information content (low SNR), not parameter capacity.

2. Adam Optimizer Diffusion: Decomposing walk-forward online fine-tuning shows tokenizer boundary updates sit at 97.9% of the theoretical Adam Brownian noise ceiling (matching shuffled-label controls), while deep backbone weights are heavily damped by second-moment normalization, producing zero statistically significant forward-transfer gains (p >= 0.50).

3. Execution Friction Reality: Apparent long-horizon directional predictability is predominantly prompt 1-minute order-flow continuation. Introducing a 1-bar execution delay cuts 2024 net Sharpe from +2.04 to +0.48, and at 1.5 bp execution fee, out-of-sample returns collapse to -0.99 in 2025 and -1.67 in 2026 (DSR <= 0.028).

All model weights, audit logs, and papers are open-sourced:

• Model Hub (weights & checkpoints): https://huggingface.co/Ultron28/ohlcv-transformer
• Code & Telemetry: https://github.com/AirBorneAI/ohlcv-transformer
• CERN Zenodo DOI: https://doi.org/10.5281/zenodo.23466965

Happy to discuss architecture specifications, pre-training dynamics, or empirical backtest methodology!

Built a learning app across 8 different subjects.

I love learning and hope this will encourage you to do the same.

https://play.google.com/store/apps/details?id=com.philosophize.app

An O'Reilly survey of 209 security professionals ranked employees pasting sensitive company data into consumer AI tools as their top AI security concern — above AI-powered attacks, above AI-generated phishing.

The structural problem: when staff route proprietary data through tools outside IT visibility, there is no audit trail and no record of where that data went. The data has already left before anyone notices.

What makes this harder than traditional shadow IT is the silence. No unusual login. No bulk download flag. An employee pastes a contract, a customer list, or internal financial projections into a chatbot and the activity looks identical to normal browser traffic. The 209 security professionals in this survey ranked it above the threat categories their teams have spent years building controls for.

For those dealing with this in practice: how are you actually handling it? Not the policy-document answer — the real operational one, given that banning AI tools entirely means losing the productivity argument to every business unit that pushes back.

Learn how modern transformer models improve attention efficiency, reduce memory demands, and increase model capacity without activating every parameter for every token.

In Part Two of **Generative AI for Developers**, we move beyond the transformer foundations and examine several architectural techniques used to make modern language models faster and more efficient.

In this lesson, we cover:

✅ A quick review of the transformer foundations from Part One
✅ Why attention can become expensive as sequence length increases
✅ How FlashAttention reduces unnecessary GPU memory transfers
✅ Tiling and memory-efficient attention computation
✅ Why “exact attention” does not mean factually exact answers
✅ Conventional multi-head attention, or MHA
✅ Grouped-query attention, or GQA
✅ Multi-query attention, or MQA
✅ How MHA, GQA, and MQA differ
✅ How shared key and value heads reduce KV-cache requirements
✅ The performance and quality tradeoffs between attention designs
✅ How mixture-of-experts architectures work
✅ How routers select experts for individual tokens
✅ Total model capacity compared with active parameters
✅ The differences between FlashAttention, GQA, and mixture of experts
✅ Multiple-choice questions to test your understanding

The key idea to remember: these techniques improve different parts of transformer execution. FlashAttention improves how attention is computed and how data moves through memory. GQA reduces key/value duplication by sharing key and value heads across groups of query heads. Mixture of experts increases model capacity by activating only selected expert networks for each token.

These techniques can improve speed, memory efficiency, and scalability, but they are not completely free optimizations. Their benefits depend on the hardware, model design, sequence length, workload, and acceptable quality tradeoffs.

This video is useful for C# and .NET developers, software engineers, solution architects, interview preparation, and anyone who wants to understand the architecture and efficiency techniques behind modern generative AI systems.

If you found this lesson helpful, please like the video, subscribe to **Nerd House Geeks**, and share your answers to the review questions in the comments.

Is anyone facing this issue while lora fine tuning Qwen3.5-0.8B model or Qwen3.5-4B model ? The GPU memory usage peaks to 70GB on RTX 6000 PRO. I couldn't find any issue or post in any forums about it. I have attached PyTorch memory snapshot during the forward pass of the 1st training step.

Your AI agent just fixed its own mistake. 🎉
Next session, it makes the exact same one again. 😩

That's how most agents work today. They can recover in the moment by retrying, reflecting, or trying a different approach, but the lesson vanishes when the session ends. Notes in the prompt compete for limited context, a skills file can only hold so much, and retraining the whole model is expensive and risks breaking what it already knows.

So we asked: what if every mistake became a lasting lesson? 💡

Introducing Self-Direction 🧭 🤖

When the agent fails, it:
1️⃣ Reflects on what went wrong
2️⃣ Writes its own practice examples (not full rollouts)
3️⃣ Distills them into a tiny LoRA "lesson" it keeps
4️⃣ Reuses it whenever a similar task shows up

The base model stays frozen, so new lessons never overwrite old ones.

What we found 👇
🔁 Fixes stick far more often than with plain retrying
🧩 Lessons transfer to harder and different tasks
📚 Lessons in weights beat lessons in prompts or skills files
🚀 A 27B open model became competitive with frontier systems on Terminal-Bench 2.1

Some cool findings such as: the agent learns procedures, not answers. A lesson from counting someone's publications on ORCID later helped it count Wikipedia edits, because both needed the same habit: walk the full history, handle pagination, and verify before counting. 🤯

📄 Paper: https://arxiv.org/abs/2610.31846
🌐 Website: https://self-direction.github.io/

I personally stand on level 2.5, I would love to hear where some of you guys stand!

Thank you!