neuralclones.ai Case study · October 2026 Contact us

Case study

Built from Qwen3.5. Scores above Open-1B.

We combined Qwen3.5 0.8B and Gensyn's Open-1B into one hybrid architecture, then carried Qwen's knowledge into it with about 370 million tokens of training. On Open-1B's own ten-task suite, the hybrid outscores Open-1B itself.

Discuss your architecture
52.8 / 50.3

On Open-1B's ten-task suite: our hybrid against the published Open-1B. Ahead on 7 of the 10 tasks.

~370M

Tokens of training, starting from a published Qwen3.5 0.8B. Open-1B was trained on 400 billion.

100%

Of planted numbers recalled, at every distance tested.

01The idea

Training a new model from scratch is expensive. Gensyn trained Open-1B on 400 billion tokens using 48 H100 GPUs for about 28 days: some 32,000 GPU-hours, or roughly $86,000 to $128,000 at today's cloud prices.

We sought to reduce those costs by deriving the new model from one that already exists. Its knowledge carries over, so training starts ahead instead of from random numbers, with less training left to do. Here, we derived a new hybrid architecture from Qwen3.5 0.8B, aimed at Gensyn's Open-1B design.

02What Open-1B cost to train

Gensyn trained Open-1B from scratch on 400 billion tokens. Its training software was built so every run gives exactly the same result, bit for bit, and that made it about five times slower than usual: Gensyn reports using only about 5% of each GPU's arithmetic. So there are two honest prices: what the run actually used, and what the same training would cost on ordinary software.

$86–128K

What Gensyn's run actually used: 48 H100 GPUs for 27.8 days, about 32,000 GPU-hours, at today's cloud prices.

$17.5–26K

The same 400 billion tokens on ordinary, full-speed training software: about 6,500 GPU-hours.

Open-1B's training compute, priced at today's Runpod H100 rates of $2.69 to $3.99 an hour
H100 GPU-hoursPer billion tokensCost
As Gensyn ran it (exactly repeatable software)~32,000~80$86,000–$128,000
At normal training speed (our estimate)~6,500~16$17,500–$26,000

The normal-speed figure is our estimate, made by removing the roughly fivefold slowdown Gensyn describes. The 48 GPUs, 27.8 days and 400 billion tokens are Gensyn's published figures.

03Our solution

Instead of training your architecture from random numbers, we derive it from a model that has already been trained.

  1. Start from two designs. The architecture you want, and the architecture of a strong model that's already trained.
  2. Work out the transformation between them. We map every difference between the two: what carries across exactly, what is rebuilt and fitted, and how damaging each change is.
  3. Lock the design and deliver. We finalise the architecture with you, keeping source parts wherever they carry the most knowledge, as this hybrid does. You get a checkpoint that starts as far along as a model trained from scratch for tens to hundreds of billions of tokens.
36–400B+

Tokens into its training when Open-1B matched our hybrid, by test. On three tests it never did, even at its 400B finish.

$1.6–26K

What those tokens cost at normal training speed; $7,700 to $128,000 at the pace Gensyn's run actually went.

Strong published models have already been trained on trillions of tokens, and larger models cost millions of dollars to train from scratch. The bigger the model, the more a head start is worth.

04The hybrid, layer by layer

Qwen's own layer, kept whole (3) Open-1B's attention layer (6) Open-1B's attention, plus Qwen's short-range mixing and sense of what to forget (15)
  1. Keep what matters most. Qwen's first three layers stay exactly as they are. They do more for the model than any others.
  2. Rebuild the rest as Open-1B's attention. The other 21 layers become Open-1B's attention layer, including its 512-token window where Open-1B uses one. Fifteen of them keep two small parts of Qwen: its short-range mixing and its sense of what to forget.
  3. Fit each layer, then the whole. Every new layer is first trained to reproduce what its original wrote. Then the whole model is trained together to give Qwen's own answers, with practice at recalling facts planted far back.

05The results

We score with two separate test suites. They ask the same kinds of questions in different ways, so the same test gives different numbers in each. We only ever compare models within one suite.

Ten-task suite

Open-1B's own evaluation suite: ten standard tests. Five solved examples are shown before each question, and each question is asked two ways (pick the answer's letter, or score the answer's text), keeping the better. This is the standard comparison, used in the table below.

Training-curve suite

Our own fast scoring of eight standard tests, which we also ran on 19 of the checkpoints Gensyn published along Open-1B's training. Questions are asked cold, with no solved examples, so every model scores lower. We use it for one thing: finding where our hybrid sits on Open-1B's training run, in section 06.

Qwen3.5 0.8B (source)66.5
Our hybrid, ~370M tokens52.8
Open-1B, 400B tokens50.3
Ten-task suite: percent correct on each test (higher is better). Orange: the higher of Open-1B and our hybrid.
TaskQwen3.5 0.8BOpen-1BOur hybrid
ARC-Easy85.766.469.0
ARC-Challenge71.236.740.9
BoolQ81.354.766.3
CommonsenseQA61.653.755.1
HellaSwag56.747.547.4
OpenBookQA64.443.246.4
PIQA71.167.367.2
SocialIQA59.746.348.9
WinoGrande58.055.053.0
MMLU54.932.133.4
Average66.550.352.8

Ahead of Open-1B on seven of ten tasks, level on two, behind on one. The biggest lead is on BoolQ, at 11.6 points.

06Measured against Open-1B's own training

Gensyn published checkpoints along Open-1B's entire 400-billion-token training run. We scored 19 of them, and our hybrid, on the training-curve suite, and found how far into its training Open-1B was when it first matched each of our scores.

Training-curve suite, a separate suite from the ten-task table above: percent correct, except the last row. The last row measures how well each model predicts ordinary text, in bits per character: lower is better. Orange: the better of our hybrid and Open-1B's finished model.
TestOur hybridOpen-1B, finishedOpen-1B first matched us
ARC-Easy59.6%55.9%Never: we beat its finish
ARC-Challenge34.3%28.4%Never: we beat its finish
WinoGrande55.0%54.1%Never: we beat its finish
OpenBookQA33.6%33.6%Same score as its finish
HellaSwag47.2%49.0%63% of the way through
PIQA68.9%71.4%18% of the way through
LAMBADA40.0%60.3%9% of the way through
Text prediction, bits per character (lower is better)0.9010.82610% of the way through
Our training, end to end
StageTokens
Layers prepared for conversion102M
Each layer trained to match its original49M
Whole model trained together215M
Totalabout 370M

07Memory carried across

Benchmarks don't show whether a model still remembers what it read thousands of words earlier. So we plant a word or a five-digit number in real text and ask for it at the end. Guessing scores 10%.

Share of 40 questions answered correctly, by distance between the planted item and the question (tokens). Orange: the better of our hybrid and Open-1B on the planted word.
Model and test~80~440~1,090~2,170~3,270~3,800
Our hybrid, planted number100%100%100%—100%—
Our hybrid, planted word100%100%82.5%57.5%32.5%37.5%
Open-1B (published), planted word100%100%97.5%95%60%35%

Numbers are recalled perfectly at every distance tested. Planted words are perfect up to about 440 tokens back, trail Open-1B through the middle distances, and edge ahead at the farthest. A dash means that distance wasn't tested.

08What this means for you

You don't have to choose between the model you already have and the architecture you want.

We beat the benchmark average of a model that costs up to $26,000 to train, for under $1,200 of compute.

More than 21× cheaper. Over 70× against what Gensyn's run actually used.

If you want to train a new architecture, you can start from scratch and invest hundreds of thousands to millions of dollars, depending on its size. Or you can invest a fraction of that by building a hybrid from a model that's already trained.

09Methodology and notes

Get in touch

Have an architecture you want to train?

In one call, you'll learn which of your changes transfer and what a pilot would take.

Discuss your architecture

Gensyn is not affiliated with this case study and has not reviewed or endorsed it. Open-1B is published by Gensyn and Qwen3.5 by the Qwen team; Neural Clones is independent of both.