We firstly simulated disease dynamics by KAN (Kolmogorov-Arnold Networks) nearly 4 years ago, but the kernel functions in the edge include the exponential number of infected and discharged people and is also in line with the Kolmogorov-Arnold representation theorem, and the shared weights in the edge are the infection rate and cure rate, and used activation function by tanh at the node of edge. And this Arxiv preprint version 1 of March 2022 is an upgraded version of KAN, considering the invariant coarse-grained which calculated by residual or gradient of MSE loss. The improved KAN is PNN (Plasticity Neural Networks) or ELKAN (Edge Learning KNN), in addition to edge learning, it also considered the trimming of the edge. We not inspired by the Kolmogorov-Arnold representation theorem but inspired by the brain science. The ELKAN to explain brain, the variables correspond to different types of neurons, the learning edge can be explained by rebalance of synaptic strength and glial cells phagocytose synapses, and the kernel function means the discharge of neurons and synapses, different neurons and edges mean brain regions. Through testing by cosine, the ELKAN or ORPNN (Optimized Range PNN) is better than the KAN or CRPNN (Constant Range PNN).The ELKAN is more general to explore brain, such as mechanism of consciousness, interactions of natural frequencies in brain regions, synaptic and neuronal discharge frequencies, and data signal frequencies; mechanism of Alzheimer's disease, the Alzheimer's patients has more high frequencies in the upstream brain regions; long short-term relatively good and inferior memory which means gradient of architecture and architecture; turbulent energy flow in different brain regions, turbulence critical conditions need to be met; heart-brain of the quantum entanglement may occur between the emotions of heartbeat and the synaptic strength of brain potentials.
Bennett, Charles Henry, David Peter DiVincenzo, and Ralph Linsker. "Digital recording system with time-bracketed authentication by on-line challenges and method of authenticating recordings." U.S. Patent No. 5,764,769. 9 Jun. 1998.
Abstract
An apparatus and method produce a videotape or other recording that cannot be pre- or post-dated, nor altered, nor easily fabricated by electronically combining pre-recorded material. In order to prevent such falsification, the camera or other recording apparatus periodically receives certifiably unpredictable signals ("challenges") from a trusted source, causes these signals to influence the scene being recorded, then periodically forwards a digest of the ongoing digital recording to a trusted repository. The unpredictable challenges prevent pre-dating of the recording before the time of the challenge, while the storage of a digest prevents post-dating of the recording after the time the digest was received by the repository. Meanwhile, the interaction of the challenge with the evidence being recorded presents a formidable obstacle to real-time falsification of the scene or system, forcing the would-be falsifier to simulate or render the effects of this interaction in the brief time interval between arrival of the challenge and archiving of the digest at the repository.
Abstract: "We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum. Despite the transformer-based progress seen in question-answering and scientific text processing, we find that current models cannot reason about or explain learned science concepts in novel contexts. For instance, models can easily answer what the conductivity of a known material is but struggle when asked how they would conduct an experiment in a grounded environment to find the conductivity of an unknown material. This begs the question of whether current models are simply retrieving answers by way of seeing a large number of similar examples or if they have learned to reason about concepts in a reusable manner. We hypothesize that agents need to be grounded in interactive environments to achieve such reasoning capabilities. Our experiments provide empirical evidence supporting this hypothesis -- showing that a 1.5 million parameter agent trained interactively for 100k steps outperforms a 11 billion parameter model statically trained for scientific question-answering and reasoning from millions of expert demonstrations."
Abstract: "Despite the surge of interest in autonomous scientific discovery (ASD) of software artifacts (e.g., improved ML algorithms), current ASD systems face two key limitations: (1) they largely explore variants of existing codebases or similarly constrained design spaces, and (2) they produce large volumes of research artifacts (such as automatically generated papers and code) that are typically evaluated using conference-style paper review with limited evaluation of code. In this work we introduce CodeScientist, a novel ASD system that frames ideation and experiment construction as a form of genetic search jointly over combinations of research articles and codeblocks defining common actions in a ___domain (like prompting a language model). We use this paradigm to conduct hundreds of automated experiments on machine-generated ideas broadly in the ___domain of agents and virtual environments, with the system returning 19 discoveries, 6 of which were judged as being both at least minimally sound and incrementally novel after a multi-faceted evaluation beyond that typically conducted in prior work, including external (conference-style) review, code review, and replication attempts. Moreover, the discoveries span new tasks, agents, metrics, and data, suggesting a qualitative shift from benchmark optimization to broader discoveries."
> Overall, it seems we are starting to recycle ideas because there isnt enough lit review and or mentoring from senior deep learning / ML folks who can quickly look at a paper and tell the author where the work has been already investigated.
Arguably, the literature synthesis and knowledge discovery problem has been overwhelming in many fields for a long time; but I wonder if, in ML lately, an accelerated (if not frantic) level of competition may be working against the collegial spirit.
I think it's been accelerated by the review community being overwelmed and the lack of experienced researchers with the combination of classic ML, deep learning, transformers, and DSP backgrounds -- a rare breed but sorely needed.
Original title, "SYNTHETIC-1: Scaling Distributed Synthetic Data Generation for Verified Reasoning," edited to fit HN's "title" length limit.
From their blog post dated Feb. 6:
"Today, we are excited to introduce SYNTHETIC-1, a collaborative effort to create the largest open-source dataset of verified reasoning traces for math, coding and science, leveraging DeepSeek-R1. Our dataset consists of 1.4 million high-quality tasks and verifiers, designed to advance reasoning model training.
We invite everyone to contribute compute and join us in our effort to scale distributed reinforcement learning to o3-scale and beyond.
In our recent post, Decentralized Training in the Inference-Time-Compute Paradigm, we explored how this paradigm shift will fundamentally reshape compute infrastructure, making globally distributed training the way forward.
The DeepSeek-R1 paper highlights the importance of generating cold-start synthetic data for RL. As our first step toward state-of-the-art reasoning models, SYNTHETIC-1 generates verified reasoning traces across math, coding, and science using DeepSeek-R1."
Thanks for posting this. I had just ran across this one and wondered if I should post it myself. Came here to see if someone had. I'd gone looking for this after reading about DeepScaleR yesterday.
I'm excited about the prospect of using formal methods to generate, analytically (not via LLM), synthetic problem+solution sets of perfect quality, of progressive size & complexity, for training; and then using RL, with progressive scaling, as in DeepScaleR.
>>That placebo effect study is 20 years old.
>Has it been contradicted anywhere? I don't see any other studies that have looked into it.
Somewhat tangentially, and not specific to Parkinson's or DBS, but placebo has been getting stronger for at least some classes of treatment. E.g.:
Tuttle AH, Tohyama S, Ramsay T, Kimmelman J, Schweinhardt P, Bennett GJ, Mogil JS. Increasing placebo responses over time in U.S. clinical trials of neuropathic pain. Pain. 2015 Dec;156(12):2616-2626. doi: 10.1097/j.pain.0000000000000333. PMID: 26307858.
Walsh BT, Seidman SN, Sysko R, Gould M. Placebo Response in Studies of Major Depression: Variable, Substantial, and Growing. JAMA. 2002;287(14):1840–1847. doi:10.1001/jama.287.14.1840
So I would not be surprised if placebo got stronger in this area as well.
Abstract: "The Evidence Lower Bound (ELBO) is a widely used objective for training deep generative models, such as Variational Autoencoders (VAEs). In the neuroscience literature, an identical objective is known as the variational free energy, hinting at a potential unified framework for brain function and machine learning. Despite its utility in interpreting generative models, including diffusion models, ELBO maximization is often seen as too broad to offer prescriptive guidance for specific architectures in neuroscience or machine learning. In this work, we show that maximizing ELBO under Poisson assumptions for general sequence data leads to a spiking neural network that performs Bayesian posterior inference through its membrane potential dynamics. The resulting model, the iterative Poisson VAE (iP-VAE), has a closer connection to biological neurons than previous brain-inspired predictive coding models based on Gaussian assumptions. Compared to amortized and iterative VAEs, iP-VAElearns sparser representations and exhibits superior generalization to out-of-distribution samples. These findings suggest that optimizing ELBO, combined with Poisson assumptions, provides a solid foundation for developing prescriptive theories in NeuroAI."
We firstly simulated disease dynamics by KAN (Kolmogorov-Arnold Networks) nearly 4 years ago, but the kernel functions in the edge include the exponential number of infected and discharged people and is also in line with the Kolmogorov-Arnold representation theorem, and the shared weights in the edge are the infection rate and cure rate, and used activation function by tanh at the node of edge. And this Arxiv preprint version 1 of March 2022 is an upgraded version of KAN, considering the invariant coarse-grained which calculated by residual or gradient of MSE loss. The improved KAN is PNN (Plasticity Neural Networks) or ELKAN (Edge Learning KNN), in addition to edge learning, it also considered the trimming of the edge. We not inspired by the Kolmogorov-Arnold representation theorem but inspired by the brain science. The ELKAN to explain brain, the variables correspond to different types of neurons, the learning edge can be explained by rebalance of synaptic strength and glial cells phagocytose synapses, and the kernel function means the discharge of neurons and synapses, different neurons and edges mean brain regions. Through testing by cosine, the ELKAN or ORPNN (Optimized Range PNN) is better than the KAN or CRPNN (Constant Range PNN).The ELKAN is more general to explore brain, such as mechanism of consciousness, interactions of natural frequencies in brain regions, synaptic and neuronal discharge frequencies, and data signal frequencies; mechanism of Alzheimer's disease, the Alzheimer's patients has more high frequencies in the upstream brain regions; long short-term relatively good and inferior memory which means gradient of architecture and architecture; turbulent energy flow in different brain regions, turbulence critical conditions need to be met; heart-brain of the quantum entanglement may occur between the emotions of heartbeat and the synaptic strength of brain potentials.
reply