Give it a photo
Nothing leaves your browser — upload a photo and both networks run on it right here, live. A single clear subject (a mug, a dog, a bike) gets a more confident guess than a busy scene.
Watch your photo move through both
Top: a plain network — straight through, no exceptions. Bottom: a network with shortcuts — the bright arcs skip over some steps and carry the photo's signal forward untouched. Every tile is a real snapshot of your photo at that point.
And here's what that buys: how much of your photo's signal is still recognizable after passing through, on average.
What each one thinks it's looking at
Real guesses, nothing scripted — they may agree or disagree.
Plain network
Network with shortcuts
Take it with you
Downloads one image: the block-flow diagram and the numbers above, with your photo's actual results baked in.
Since every image here is built from your own photo, this one's best kept personal rather than worn — a magnet, not a t-shirt.
- Download the PNG above.
- Open a Zazzle "upload your own image" template and personalize it with that file.
Open a Zazzle "upload your own image" template → — pick a magnet-style product; most of Zazzle's blank templates accept any uploaded image the same way.
How this actually works — skip connections, and why depth used to break
MobileNetV1 is a plain network — every block feeds straight into the next, no exceptions. MobileNetV2 is a residual network from the same lineage — most of its blocks also carry an identity shortcut: the block's own input, added back in unchanged, in parallel with the transformed version, so the block only has to learn the difference a real one would need to add. Some of V2's blocks don't have a shortcut — that's real too, not an oversight; a block that changes the number of channels or the spatial size can't just add its input back unchanged, so the shortcut only appears exactly where the shapes line up.
Stack enough layers on a plain network and, past a certain depth, accuracy gets worse, not better — and not from overfitting (training error rises too, not just validation error). This is the degradation problem: past some depth, plain networks get measurably harder to optimize, not just harder to generalize. It's counterintuitive, because a deeper plain network can always, in principle, just learn to copy its input forward through the extra layers and match the shallower network's accuracy exactly — that's a function every one of those extra layers is fully capable of representing. In practice, plain networks trained with ordinary gradient descent don't reliably find that solution once they get deep enough. Learning to pass a signal through completely unchanged turns out to be a surprisingly awkward thing to ask an ordinary stack of layers to do.
A residual or skip connection sidesteps the problem by making "pass it through
unchanged" the network's default, free of charge: instead of asking a block to learn some
function F(x) outright, it computes F(x) + x — the block's own
transformation, plus an identity copy of its input, added back in via the shortcut arcs
you saw above. If the best thing a block can do is nothing, it just has to learn to output
zero, which turns out to be a far easier thing for gradient descent to settle into than
learning an exact identity mapping through an ordinary stack of weights. That's the entire
mechanism — one addition, in parallel with the block's normal path — and it's what let
networks confidently go from a few dozen layers to over a hundred without the accuracy
collapse plain networks were already starting to show well before that.
This page can't reproduce the classic degradation result live — the plain 18/34-layer networks it was originally measured on were never released as pretrained weights, since nobody had a reason to publish a network they'd deliberately shown was worse. But the original numbers are public record: on ImageNet, He et al.'s plain-34 network scored a higher (worse) top-1 error than the shallower plain-18 — 28.54% vs. 27.94% — despite having nearly twice the layers. Adding residual connections to the same 34-layer design (ResNet-34) dropped that error to 25.03%, beating both plain networks outright. Depth alone made things worse; depth plus shortcuts made things much better.
The "plain network" above is MobileNetV1 (Howard et al., 2017); the "network with shortcuts" is MobileNetV2 (Sandler et al., 2018) — real, published, and widely deployed on real phones, not built for this page. Their own published numbers:
| MobileNetV1 (plain) | MobileNetV2 (shortcuts) | |
|---|---|---|
| Parameters | 4.2 million | 3.4 million |
| ImageNet top-1 accuracy | 70.6% | 72.0% |
| Blocks with a skip connection | 0 of 13 | 10 of 17 |
The classic version of this comparison (VGG vs. ResNet50) is more dramatic — "twice as deep, a sixth of the parameters" — but those two networks are together over 600MB to download, which didn't feel right to ship just for this page. MobileNetV1 and V2 are small enough to load in seconds, and the one variable that actually changed between them — the skip connection — is exactly the thing this page is about. The tradeoff: a subtler published payoff than the classic comparison, honestly reported above rather than oversold (the live "how much survives" number you saw in the lab tells a more dramatic story than these published figures do).
The line of research this page sits inside, in order:
- Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). "Learning representations by back-propagating errors." Nature, 323(6088), 533–536. — the foundational result behind training any multi-layer network at all.
- LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). "Gradient-Based Learning Applied to Document Recognition." Proceedings of the IEEE, 86(11), 2278–2324. — introduces the convolutional architecture (LeNet-5). PDF
- Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). "ImageNet Classification with Deep Convolutional Neural Networks." NeurIPS 2012. — AlexNet, the moment CNNs overtook prior approaches on real vision benchmarks. Paper
- Simonyan, K., & Zisserman, A. (2014). "Very Deep Convolutional Networks for Large-Scale Image Recognition." arXiv:1409.1556. — the VGG family, the last major "just stack more plain layers" architecture before the degradation problem caught up with it. arXiv:1409.1556
- He, K., Zhang, X., Ren, S., & Sun, J. (2016). "Deep Residual Learning for Image Recognition." CVPR 2016, 770–778. — residual connections and the degradation problem this page is built around. arXiv:1512.03385
- Howard, A. G. et al. (2017). "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications." arXiv:1704.04861. — MobileNetV1, the plain network running on this page. arXiv:1704.04861
- Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018). "MobileNetV2: Inverted Residuals and Linear Bottlenecks." CVPR 2018. — MobileNetV2, the residual network running on this page, and its own account of why adding shortcuts to V1's design helped. arXiv:1801.04381