This is a companion piece to Consciousness, Computability, Decidability, and Evolvability. That piece argued for attacking weaker, well-formed questions about consciousness-adjacent proxies instead of the metaphysical question directly, and singled out evolvability as the more empirically checkable of the two tracks, since a negative evolvability result can be tested against biology in a way a negative decidability result cannot. This piece works through what that actually looks like as a formal object, and what a controllable synthetic world buys you when trying to study it.

Large language models are experiencing their Cambrian explosion. They may not be the path to AGI, but at least they give a taste of what it could be. The current mainstream approach to rely on scaling, may not be the sole solution, as the data runs out, and the models are plateauing in benchmarks, though we may very well see emergent behaviors that surprise us, as the bitter lesson taught us. In this article, I first review Transformers model, then summarize useful workflows with LLMs, which I intend to keep updating.

Essentially all modern processors have the capability to apply instructions on a vector in one processing unit cycle instead of operating on a single scalar. Language designers and compiler developers have been trying hard to leverage these hardware capabilities by compiling scalar programs into vector instructions. One possible approach is using SIMD (Single Instruction, Multiple Data) intrinsics, supported by all modern C/C++ compilers, through SSE (Streaming SIMD Extension), AVX (Advanced Vector Extensions) others for x86 architectures, and ARM NEON extensions.

Tiled algorithms, in general, divide the problem into smaller, manageable tiles that fit into faster, but limited-size memory, be it cache, shared memory, or registers, to improve memory access patterns. Many problems which often involve matrices, can be broken down into tiles. This improves cache utilization which improves cache hit/miss ratio, by reusing data within smaller subproblems. In this post we implement tiled matrix multiplication.

At CES 2025 event, NVIDIA announced, among other interesting things, including a single chip made out of72 Blackwell GPUs, with 1.5 ExaFLOPS performance, it’s first world foundation model, Cosmos. DeepMind is working on a similar model, and this is also one of xAI’s core missions. I suspect that these models are among the next frontier in scientific and AI research, following AlphaFold’s breakthroughs in protein folding prediction.