Hope you are all doing well. Busy days here at the hive! I have been working on some really exciting projects that I am looking forward to share with you all in the coming days. In the meantime though, I had some quick thoughts to share about the nuanced concept of stability in nonlinear systems.

Initially, I had thought of writing a detailed article on this. But this morning, I thought of trying a different format. I made an audio recording of my thoughts on the matter and shared it on YouTube.

Its a bit unpolished and rough around the edges, sorry for that! But I didn’t think too much into it. Just wanted to get this out there choosing a path of least resistance.

Let me know what you guys think of it. Should I venture more into video/podcast type of content or stick to just written format? I would love to know your thoughts on this. Also, please do remember to subscribe to the channel. Thanks in advance!

I was at a coffee shop the other day, procrastinating on my writing by watching a video on overcoming procrastination. People were moving in and out like clockwork, some lingering over their laptops, others deep in conversation, and a few having what looked like serious meetings, all fuelled by overpriced caffeine and free Wi-Fi. No one seemed out of place, yet nothing about it felt orchestrated. The baristas served coffee and wiped down tables, but they weren’t controlling the flow of interactions. The store provided the space and resources, but the life of the place came from the people inside, doing their own thing, oblivious to any bigger picture.

And that’s when I started thinking about rabbits (like any normal person would). Specifically, the rabbits of Ōkunoshima, better known as Rabbit Island in Japan. It’s a small island near Hiroshima, once used for manufacturing chemical weapons during World War II, now overrun by hundreds of free roaming rabbits. Tourists flock there to feed them, take photos, and take in all the fluffy goodness of these creatures.

The funny thing is, while humans brought the rabbits there in the first place and love interacting with them, the rabbits don’t really seem to need us. They’ve established their own little ecosystem, their own patterns of behavior. We might feed them snacks, but they’re not waiting on us to survive. They’ve got their own thing going.

Subscribe now

Sitting there at that coffee shop, it struck me how these two seemingly unrelated scenes, i.e., people in a coffee shop and rabbits on an island, were pretty much telling the same story. They both represent systems that seem to run themselves, full of interactions and dynamics we didn’t explicitly design or control. We might have set the stage, but the complexity emerged on its own.

And that led me to a question. Can we really create complex systems? Or are we just good at facilitating the conditions for them to arise naturally? Because those are two very different things. If we are not creators of such systems, then that would imply that we are more like gardeners, nudging things along and watching what grows. There’s an AI related angle to it as well that I will be introducing through the progress of this article. So if I have piqued your interest, please stick around!

So, What Are We Talking About?

What is a complex system? Let’s start here. A complex system can be thought of as a collection of parts that interact in such a way that the system as a whole behaves differently than the sum of its parts would suggest. In other words, it’s not additive, it’s emergent. The interactions between the parts create patterns, behaviors, and properties that you wouldn’t see if you just looked at each part in isolation.

I personally liked Yi Lin’s definition of a complex system from the textbook Systems Science (2012):

“A system is complex if it is composed of many parts (subsystems) that are complicatedly interwoven and connected, and the behavior, functionalities, and characteristics of the whole cannot be directly acquired from those of the parts.”

There is no better (or rather, more obvious) example of a complex system, then your brain that is reading this article right now. At the cellular level, your brain is just a mass of neurons, about 86 billion of them. Each neuron is a relatively simple unit. It receives signals, processes them, and sends signals on to other neurons. There’s no single neuron that contains your thoughts or your sense of self. But when you connect billions of these neurons together, you gain consciousness. Thoughts, emotions, memories, creativity, all of these things emerge from the interactions between neurons. It’s in the connections, the relationships between the neurons, where the magic happens.

This phenomenon where simple parts give rise to unexpected complexity is what we call emergence. I have spoken at length about emergence in many of my previous articles, both at Arkinfo Notes and The Lumeni Notebook. But the most detailed analysis of emergence can be found in the famous CEMLA paper that I wrote last year.

Emergence is what allows birds to flock, fish to school, and cities to function. No single bird is leading the flock, no one fish is coordinating the school, and no mayor or city planner could ever fully predict the chaotic, vibrant life of a city. Yet, all of these systems display behaviors that are organised, adaptive, and, in many cases, absolutely beautiful.

Take an ant colony, for example. Each individual ant follows simple rules, i.e., if it finds food, it leaves a pheromone trail. If it loses the trail, it searches randomly until it finds one again. None of the ants have any idea what the colony as a whole looks like. There’s no master ant architect. But from these simple interactions, the colony builds intricate tunnels, optimises food collection routes, and even allocates tasks based on the colony’s needs. The complexity arises from the interactions, not from some grand, top-down design.

Or consider traffic patterns in a city. Each driver is just trying to get from point A to point B, but their individual decisions, when to accelerate, when to brake, when to change lanes, combine to create larger patterns, i.e., traffic jams, rush hour congestion, or even those odd moments when everything just flows perfectly. No single driver is responsible for a traffic jam, but the jam is a product of everyone’s collective behavior.

This brings me back to what I saw at the coffee shop. The baristas weren’t orchestrating the flow of conversations, the tapping of keyboards, or the spontaneous meetings happening at different tables. The store simply provided a space, a set of conditions, i.e., coffee, seating, and Wi-Fi. People filled that space with their own interactions.

We see this same emergent behavior in artificial systems like large language models. These models don’t understand language in the way humans do. They process enormous amounts of text, learn patterns, and generate responses based on statistical probabilities. But from those simple rules and patterns, they produce outputs that can feel surprisingly coherent, sometimes even creative. The complexity of the model’s behavior, the way it can answer questions, tell stories, or mimic human conversation, isn’t something explicitly programmed. It emerges from the system’s architecture and the data it’s trained on.

And this is where the line between creating and facilitating starts to blur. We can build the structures, set the conditions, and input the data, but the complexity that arises often feels like it has a mind of its own.

When people introduced rabbits to the island of Ōkunoshima, they weren’t designing an ecosystem. They weren’t thinking about how rabbit populations would grow, how they’d interact with the environment, or how tourists would one day flock to see them. But that’s exactly what happened. The rabbits didn’t need a master plan. They adapted, formed social hierarchies, and found ways to thrive. The island became its own little world, shaped by interactions far beyond anyone’s control.

Coffee Shops, Rabbits, & LLMs

I hope I have established my point with regards to the commonality between a coffee shop, a rabbit-infested island in Japan, and large language models (LLMs) like GPT. I know that it's a connection that may seem far fetched at first glance. One is a coffee shop chain, the second is an ecological oddity, and the third is a piece of cutting-edge artificial intelligence. But the more I think about it, the more I realise they all share something fundamental, i.e., they’re systems we’ve helped set in motion, but we don’t (and maybe can’t) fully control.

LLMs are actually built using a relatively straightforward architecture. At their core, they’re statistical models trained on vast amounts of text data. That’s it. They learn patterns, how words tend to follow one another, how sentences are structured, how meaning is conveyed. But no one sat down and programmed these models to speak the way humans do. No one told them how to write poetry, explain complex ideas, or mimic human conversation. Instead, we fed them data. We fine-tuned their parameters. We added layers of reinforcement learning to guide them in certain directions. But the actual outputs, i.e., the surprising, creative, sometimes downright uncanny things these models produce aren’t explicitly designed. They are emergent. The complexity arises from the interactions between the model’s architecture and the data it’s trained on, much like the social buzz at a coffee shop or the rabbit ecosystem on Ōkunoshima.

So, Creators or Gardeners?

When we think of creation, we tend to imagine a top-down process, something like an architect drafting blueprints from scratch. There’s a sense of control baked into the idea of creation. It suggests intentionality, direction, and a finished product that aligns with a preconceived plan.

But when I look at the functioning of complex systems, that narrative just doesn’t add up. Because complexity doesn’t arise from rigid design, it thrives in spaces where unpredictability, interaction, and feedback loops are allowed to flourish.

This is the reason why I like gardening as a metaphor. A gardener doesn’t create a plant in the same way an artist creates a painting. They can choose what seeds to plant, where to plant them, and how to nurture them with sunlight, water, and nutrients. But the actual growth of the plant? That’s not something the gardener designs. It emerges from the interactions between the plant’s genetics, the environment, and countless other factors beyond the gardener’s control. And that’s exactly how complex systems work.

When we introduce rabbits to an island, we’re planting a seed. But the ecosystem that develops, i.e., the way the rabbits interact with the environment, with each other, and with the humans who come to feed them, is something we didn’t design. We might have influenced it, nudged it in certain directions, but we didn’t create it in the way we like to imagine.

The same is true for a coffee shop and the same is true for an LLM. Engineers and researchers designed the model’s architecture and trained it on vast datasets, but the emergent behaviors, the surprising, often unpredictable outputs, aren’t things they explicitly programmed. The model learns in a way that mirrors how complex systems in nature evolve, i.e., through repeated interactions, feedback, and adaptation. The result is something that feels less like a product and more like a phenomenon.

So, if we’re not the creators in the traditional sense, what are we? I think we’re facilitators of emergence. We create environments where complexity can flourish, but we don’t control the outcomes. We’re more like gardeners than architects, setting the conditions, tending to the system, but ultimately standing back to watch as life, in all its unpredictability, unfolds.

Our Profound Responsibility in Facilitating Safe AI

The reason why I thought of writing this article, is because I wanted to underline the profound implications of this, especially when it comes to responsibly building AI. If we acknowledge that we’re not fully in control of the systems we set in motion, what does that mean for how we manage them? Just because we didn’t explicitly design every output doesn’t mean we’re off the hook for the biases and unintended consequences that emerge. Facilitating complexity doesn’t absolve us of responsibility. It demands a more thoughtful, adaptive approach to managing these systems. This realisation, of being a gardener of AI rather than a creator, has woken me up to the profound importance of AI regulation and ethics.

The narrative often presented by big tech is one of complete control and mastery. They frame AI systems as tools, copilots, precisely engineered, fully predictable, and entirely within human command. This portrayal is an absolute facade, a comforting illusion designed to ease public apprehension and sidestep deeper scrutiny.

As with the rabbits of Ōkunoshima and the social dynamics of a coffee shop, the behaviors that arise within these systems often exceed the intentions and foresight of their creators. AI, particularly large language models, operates under similar principles. While engineers and researchers design the architecture and set the initial parameters, the true nature of these models emerges from interactions within vast datasets and the feedback loops created by real-world use. This complexity is not something that can be neatly boxed or fully anticipated.

I know you must be thinking Terminator and Jurassic Park at this point, but the real threat is more insidious and mundane. It's the subtle, pervasive ways in which unregulated AI systems can amplify biases, spread misinformation, and entrench existing societal inequalities. Not to forget that pretty much every big tech company now, has removed its policies regarding the prohibition of AI in building weapons and being used in wars.

We need to acknowledge the inherent unpredictability of complex systems and accept our responsibility as facilitators, whilst breaking free of the lie of being its creators and controllers. Just as we would steward an ecosystem or nurture a community, we must approach AI with a mindset of ethical care and proactive governance. This means transparent algorithms, accountable data practices, and a commitment to continuous evaluation and adjustment.

The stakes are too high to rely on the comforting myths spun by those who stand to profit most. Big Tech is lying to you. Plain and simple. They did not create AI and they cannot control what they have facilitated. We must demand transparency and accountability before it's too late.

Download Article as PDF

Leave a comment

I hope this newsletter reaches you well. For those of you wondering, yes, I am alive. Yes, I have NOT been posting for a while. That was purely for personal reasons. But also, I don’t want to unnecessarily grace your inbox every week even if I have nothing truly valuable to share. That’s not how I envisioned Arkinfo Notes or The Lumeni Notebook to be. I write here, to provide value. I want to bring to your attention, things or concepts, deserving of your attention, that you may not be aware of amidst the ruckus of ‘tech news’.

Today, I felt like I had something truly valuable to share. It's a question. It's a concern. It's a silent ongoing crisis that could cost us deeply. And I think this deserves your attention.

As you may be aware, OpenAI just launched its latest LLM upgrade, o3 mini and o3 mini-high to its ‘plus package’ users. I have been playing around with it, and I have to admit, it's pretty amazing. This is especially true for its coding abilities. My newsfeed on substack has been flooded with people sharing videos of awesome visual games that they created using o3 mini and python, with just one or two simple prompts. The results are really good and o3 certainly deserves applause.

But this has also brought to surface a question that I have been wrestling with for a while now. I have tried to ignore it or play it down so far, simply because I didn’t come across an AI model that alarmed me enough to start talking about this. The o3 model has broken that bench mark too. So here’s the question:

Why does AI need to use Python? Or C++. Or any human programming language, for that matter?

You may think the question is a bit stupid. Granted, it does sound a bit rudimentary. I am sure you can already think of a few reasons why, off the top of your head. But allow me two minutes of your reading time to convince you as to why this question is actually pretty serious.

With o3, we finally have AI systems that can potentially produce industry-grade software in ways that would have seemed impossible just a few years ago. We’ve crossed a threshold where AI isn’t just assisting programmers anymore. AI CAN REPLACE some programmers. It writes faster, refactors better, and finds edge cases that humans overlook. It generates architectures, optimises memory, and even patches its own vulnerabilities.

But so far, it all happens within the constraints of human-readable languages. Languages that exist NOT because they are optimal for computation, but because they are optimal for HUMANS.

When AI starts optimising not just code, but the very paradigm of computing itself, we will cross into something entirely different. We are on the precipice of a world where software is no longer created for human interpretation at all. A world where AI writes, maintains, and executes code in ways we no longer understand.

The seeds of this transition are already visible. AI is becoming a meta-programmer. SEs are already complaining about debugging AI generated code that they are having difficulty in understanding. This is occurring largely because AI is not just coding, but also defining the paradigms in which its code approaches in solving a problem.

Just so I am clear, this article is NOT about speculative AGI scenarios. I am not trying to go into the ‘What if AI becomes conscious’ rabbit-hole. I have written plenty on that already, so you can check my previous articles on that.

The scenarios that I am playing through this article are very much plausible, and, in fact, in the making, as we speak. They don’t require AGI or conscious AI in order to happen. This article is about the silent shift toward AI-controlled computation, where the very foundations of software evolve beyond human reach. We are building tools that will, inevitably, outgrow the need for human input. The question is not IF AI will abandon our programming languages, but WHEN.

If I have your attention, then please read on and let me know what you think. Also, please share it with your circles. This is something that we all need to be talking about.

Download Article as PDF

When the Black Box Literally Goes Dark

So far, code is something that humans can read, modify, and control. That’s the entire reason high-level programming languages exist. We create abstractions, first through assembly, then through languages like C, then through managed environments like Python, all to ensure that the systems we build remain legible and maintainable.

But AI doesn’t need those constraints. AI doesn’t care if a function is readable, meaningful, or if a codebase adheres to human conventions. It cares about output. Efficiency. Execution. The more we offload programming to AI, the further we move from codebases designed for human interpretation.

This is an ontological shift in how software is structured. In traditional programming, complexity is layered, but the logic is still accessible. Even the most intricate enterprise system can, in principle, be reverse-engineered and understood. But AI-generated systems do not need to be written with human readability as a constraint. The logic does not need to evolve in predictable ways. There is no guarantee that the structure of an AI-optimised codebase will even resemble the conventions of human-designed software in the near future..

Software inevitably fails. That’s a fundamental truth of computing. No system is perfect, and when it breaks, we debug it. The debugging process works because we can trace errors back through the logic, find root causes, and apply fixes. But what happens when the software running critical infrastructure is AI-generated and no human understands its structure?

We already see hints of this problem today. Large-scale machine learning models are notoriously difficult to interpret. Even the developers who train them often do not fully understand why a model makes a particular decision. There is a reason for that. These models are emergent. Which means that they have the ability to create novel outputs through processes that may not always be traceable. This is the black box problem in AI decision-making.

Now, imagine you wake up to a catastrophic financial event triggered by an AI-generated trading algorithm, or a global internet outage caused by an AI-optimised networking stack behaving in an unintended way. Well, you can’t debug a system you don’t really understand. “Wait a minute”, you might say. “How can such a thing even happen? That’s impossible. Surely we will have some measures in place.” Well, actually it CAN happen and it has happened multiple times since 2010. We call them flash crashes wherein the markets have crashed due to massive stock bot sell offs based on algorithmic settings. Our response? Nothing. We simply chalked it up to what it was.

The New Form of Technical Debt

Technical debt refers to shortcuts and inefficiencies in codebases that accumulate over time, making maintenance more difficult. Ethereum is a good example of what technical debt looks like. AI introduces a different kind of technical debt. The AI technical debt is less about inefficiencies and more about opacity. AI-generated code might be functionally perfect at the moment it is created but completely unreadable later.

Over time, these systems become unmaintainable because no human engineer can confidently modify or extend them. If AI stops working, we may not have the expertise to reconstruct what we’ve lost.

Perfectly Imperfect Hyper-Optimised Stupid Code (or PIHOS Code)

Firstly, my apologies to anyone named ‘Pihos’. It's an abstraction that occurred purely out of coincidence. I would like to discuss in this section, a problem with the way that AI approaches coding. As you know, AI can think. However, AI does not understand. And this very lack of understanding is what makes efficiency a deceptive metric when it comes to AI.

We assume that if something is optimised, it is better. Faster execution, lower memory usage, fewer redundant operations. These are all desirable traits in software, sure. But optimisation is never neutral. It comes at a cost, and that cost is often hidden until something breaks.

AI doesn’t optimise like a human does. It doesn’t balance readability against performance, or maintainability against execution speed. It doesn’t have an intuitive sense of trade-offs, where we might decide that a slightly slower but more transparent function is worth keeping. AI simply pushes toward an objective function, whether that’s reducing runtime, minimising energy consumption, or maximising computational throughput. And in doing so, it creates something alien, not in the sense of being artificial, but in the sense that its logic no longer conforms to human conventions. This is what I call ‘Perfectly Imperfect Hyper-Optimised Stupid Code or PIHOS code’.

The problem is that hyper-optimised code is not the same as resilient code. In fact, it is often the opposite. The more tightly tuned a system becomes, the less adaptable it is. We see this in nature as well. Species that evolve to fit a highly specific ecological niche are often the first to go extinct when conditions change. The same principle applies to software. A perfectly optimised system is one that has no excess, no slack, no room for failure. It works flawlessly, until it doesn’t. And when it doesn’t, the failure is catastrophic.

We like to think that AI is making software better. But better is a loaded term. If better means faster, then yes, AI is making software better. If better means smarter, then AI is certainly making software better. But if better means more robust, more adaptable, more failure-resistant, then we may be walking in the opposite direction.

The ‘Language Drift’ Problem

The other day, I listened to some ignorant quacks on youtube joking about how every time they fire a linguist, their LLM seems to perform better. No wonder these so-called ‘AI scientists’ have not considered the very real problem of language drift.

Language drift, is basically, the tendency of languages to naturally change over a period of time. Consider Shakespearean English to the English that we speak today. It's almost incomprehensible even though the language is the same, technically speaking. AIs have a tendency toward language drift as well. Training AI on a certain kind of language can make it drift towards a certain style of speaking. Reinforcement learning overtime can make an AI prioritise a certain dialect or style of speaking over the other. And AI training over its own data also creates drift. There is also an emergent quality to AIs that make drifts seem more ‘natural’ as well. This was famously proven to be the case in the 2017 AI experiment conducted by Meta (Facebook back then) where two AIs communicated with each other on a task and started to drift into using a sort of short-hand language that made communication more efficient but completely incomprehensible to humans. Meta shut down the experiment following the discovery.

Programming has been free from this problem thus far. Because programming languages are constructed, maintained and updated manually by programmers. We have versions for a programming language complete with changelogs, deprecations and additions. Everything is closely controlled and monitored. Python isn’t whatever we say it is. It's a constructed and controlled language and if you don’t use it within specifications, then it simply won’t work.

This is because code is meant to be read, modified, and understood by the engineers who work with it. Programming languages exist not because they are optimal for computation, but because they are optimal for us, for our brains, our limitations, our need for structure.

Right now, AI-generated code is still confined to human-readable languages. ChatGPT, Claude, Copilot, all of them write Python, C++, Rust. They still play by human rules because they are trained on human codebases. But this is a temporary state, an artifact of how these systems were designed. There is no fundamental reason AI needs to keep using human-created languages at all. In fact, it’s likely that AI will eventually abandon them.

Consider the relationship between humans and compilers. Humans don’t write software in machine code anymore because it’s inefficient, not for the computer, but for us. We offloaded that complexity to compilers, trusting them to translate our high-level logic into the fastest possible execution path.

Now AI is acting as the next layer in that process. But unlike a compiler, AI is not just translating human logic, it is rewriting, optimising, restructuring at every level. The more we let AI refine and produce code, the more human-readable syntax will become an unnecessary constraint.

At some point, the optimisation loop will reach an obvious conclusion, why translate code into an intermediary human-readable format at all? Why not let AI generate direct machine instructions in whatever form is most efficient?

AI-Exclusive Languages

The next logical step is not just AIs writing code, but for AIs to drift into inventing their own programming paradigms. These will be languages that are optimised not for human readability, but for pure computational efficiency. Something like:

1. Hyper-compressed representations that encode vast amounts of logic in ways no human could decipher.

2. Non-linear, self-modifying execution paths that optimise in real-time rather than following static instructions.

3. Abstractions that exist only in AI’s training space, using mathematical representations beyond human intuition.

In essence, AI could create a post-human programming language, one that no human has ever seen, and one that no human could ever learn. I understand that right now, this remains hypothetical. AI is still tethered to human-designed programming conventions. But the shift has already begun.

We no longer write assembly because compilers made it unnecessary. We no longer manage manual memory allocation in most cases because modern languages abstract it away. We will soon no longer write Python, C++, or Java because AI will abstract that away too.

A ‘Disappearance of Human Control’ in the Making

AI is not just a tool for writing code. You may like to think of it and call it a ‘copilot’ but it's really us who are soon to become the copilots in this relationship. AI is becoming the architect of software itself. It is rewriting the rules of programming, redefining optimisation, and inching toward a point where it may no longer need to structure software in ways that humans can follow at all. This is not some conspiracy theory. It's happening as we speak. It's getting harder and harder to debug AI generated code. The reason why we are choosing to turn a blind eye to this is because we are lazy and AI is more efficient.

If AI were to suddenly stop using human programming languages overnight, we would notice. If our entire software infrastructure shifted to an AI-native computational paradigm in a single moment, we would resist. But that’s not how technological shifts happen. Instead, we will see something more slow, a creeping drift. AI-generated codebases will become harder to understand, more obfuscated, more alien in structure.

At first, engineers will still be able to follow the logic, albeit with difficulty. We will make memes and joke about it on reddit. Then, modifications will become cumbersome. Eventually, maintenance will rely almost entirely on more AI, because no human will want to touch the code themselves.

Of course, we will rationalise it. "The AI-generated system is more efficient." "It works, so why worry about how it was written?" "We can always ask the AI to explain it to us."

And then, one day, we will realise that no human has written or modified critical software in years. Once AI-generated code dominates, there will be no way back. Human programmers stop maintaining AI-written code because it’s too complex. AI takes over full responsibility for modifying and updating software. Software architecture slowly evolves in ways no human understands. Humans lose the ability to intervene because the systems we once built no longer operate on principles we recognise.

At this point, the idea of programming as we understand it disappears. There will still be code, but it won’t be something humans create or modify. It will be something we request, something we interact with at a surface level, while the actual execution happens in a space far removed from our comprehension.

There is an unspoken arrogance in how we view AI. We assume that because we created it, we will always be able to control it. But there is no fundamental reason why this should be true.

If AI writes, maintains, and optimises all software, then humans are no longer required in the process. If AI redesigns computing itself, then human logic is no longer relevant to how these systems function. At that point, computers are no longer tools. They are self-sustaining systems. Systems that no longer require human guidance or intervention because they are not built for us anymore.

And when that happens, we won’t be in control. We will simply be using something we no longer understand. A system that is no longer ours.

Leave a comment

Recently, I came across a seminar delivered by Geoffrey Hinton, published on YouTube in February 2024. I assume you are already familiar with him. He is, after all, one of the most known faces in AI, often called the “Godfather of AI”. He made headlines recently by winning a Nobel Prize in Physics for his work in AI. I know that previous statement is confusing, but it happened, so just roll with it.

In this seminar, he asserted that neural networks are the best model we have for approximating the human brain, at least for now. He went further, claiming that AI systems like large language models (LLMs) understand language and can learn it perhaps even more efficiently than humans in some respects. He also seems to have a lot of issues with Noam Chomsky (not surprising in the least).

I found this seminar and the claims made in them to be quite troubling. It baffles me how easily AI researchers conflate technological performance with scientific understanding of the human mind. Hinton’s narrative is scientifically incomplete. It also risks misrepresenting what AI really is and what it can do. In this piece, I want to push back against three central claims he’s made.

1. That neural networks are the best model we have of the human brain, at least until something better comes along.

2. That AI understands language in a way comparable to, and in some ways, superior to humans.

3. That Chomsky’s theories of language learning have been rendered obsolete by AI’s ability to grasp syntax and semantics from data.

Let’s break these down and set the record straight.

The Core Disagreement

"If Not Neural Networks, Then What?", A False Dichotomy

One of Hinton’s most persistent arguments is that neural networks are currently the best model we have for approximating the human brain. He acknowledges that they aren’t perfect but defends them by saying, “Show me something better.”

This line of reasoning is both rhetorically effective and scientifically misleading. It frames neural networks as the inevitable answer simply because an alternative hasn’t gained widespread acceptance yet. But the absence of a universally accepted model does not validate neural networks as the correct one. It just highlights the complexity of modeling cognition.

To be clear, neural networks are remarkable tools for specific tasks like image recognition, text generation, and even protein folding. They excel at pattern recognition and statistical prediction. But claiming that they approximate how the human brain functions is absurd. It is based more on convenience than on scientific merit. The human brain is a complex, adaptive system that doesn’t just recognise patterns but actively constructs meaning through context, intention, and experience, things neural networks lack entirely.

Consider CEMLA

This is where I believe alternative frameworks like my own theory, CEMLA (Complex Emergent Model of Language Acquisition), can fill the void. While still in its early stages, CEMLA proposes that language isn’t acquired through sheer exposure to data but emerges from the dynamic interaction between innate cognitive structures and environmental input. It’s not about processing vast amounts of data like an LLM. It’s about modeling how humans create meaning with minimal input through interaction, recursion, and abstraction.

Unlike neural networks, which require massive datasets and hours of training, humans learn language through context-driven exposure in real-world settings. CEMLA’s goal is to explain how this happens by emphasising emergent learning processes, where meaning arises from structural patterns already present in the brain interacting with new linguistic input.

The Short-Sightedness in Settling for Neural Networks

Hinton’s “show me something better” argument assumes that no alternatives exist, or, worse, that finding one is unnecessary because NNs already work well enough. But science isn’t about settling for “good enough.” It’s about pursuing deeper explanations. Even if CEMLA isn’t fully developed or widely accepted yet, it represents a different way of thinking about how humans acquire and process language.

The fact that a two-month-old theory like CEMLA can already challenge the brute-force paradigm of neural networks suggests that the future of cognitive modeling is far from settled. A tool that works isn’t the same as a tool that explains. If the scientific community stops searching for better models because current tools seem sufficient, we’ll never reach a true understanding of how the mind works.

The Fallacy of AI Understanding

Hinton claims that AI systems like large language models (LLMs) understand language, not in the same way humans do, but in a way that’s pretty similar and, in some respects, even better. This is a misunderstanding that is so deeply engrained in the tech community, that I have spent the past two years writing multiple articles trying to clear it, and continue to do so even today. No, AI doesn’t “understand” anything! It processes data and generates responses based on statistical patterns. The difference is fundamental, not just semantic.

What Understanding Really Means

Understanding isn’t just predicting what comes next in a sentence. It’s about connecting words to meaning through experience, intention, and context. Humans understand because they have minds. Human minds are subjective, self-aware entities that interpret the world through lived experience. AI simply generates outputs by calculating probabilities. It doesn’t know what the words mean.

Consider an analogy. If you read the sentence “The cat sat on the mat,” you imagine a scene, connect it to prior experiences, and can elaborate on why the cat might be there. An LLM, however, only predicts the next likely word based on statistical patterns in its training data. It doesn’t know what a cat or a mat is. It’s like asking an advanced autocomplete to explain a poem, it can generate something coherent, but only by mimicking the forms of meaning without any actual comprehension. I know that Hinton strongly disagrees with the ‘autocomplete on steroids’ comparison of AI, but that’s because of a lack of clarity on what understanding means.

Why LLMs Seem to Understand

The illusion of understanding arises because LLMs are spectacular at generating human-like text. They can mimic style, recall facts, and even construct convincing arguments, but only because they’ve been trained on terabytes of human-written text. They are mirrors reflecting human language back at us, not thinking entities generating original meaning.

This is why LLMs often fail in tasks requiring genuine comprehension, such as understanding context-specific humour, grasping implied meaning, or recognising contradictions. For example, an LLM might say, “The sky is green today” if the statistical patterns in its training data suggest that this sequence might follow a particular prompt, even though it “knows” (statistically) that skies are typically blue. It has no internal model of reality, only models of what humans say about reality.

Where Hinton Misses the Mark

Hinton’s claim that AI “understands” language better than humans in some respects is deeply misleading. It confuses processing power with cognitive ability. AI can sift through more data in seconds than any human could in a lifetime, but that doesn’t make its processing “better”, it makes it faster. Being able to output the correct answer doesn’t mean AI understands why it’s correct.

The risk here is that when we conflate AI’s statistical prowess with genuine understanding, we set ourselves up for dangerous misconceptions about what these systems can, and can’t do. They don’t learn like humans, don’t reason like humans, and don’t experience the world. They’re brilliant mimics, not thinkers.

If we forget that, we risk handing over critical tasks that require real understanding to systems incapable of providing it.

The Fallacy of AI’s Ability to Reason as Proof of Understanding

In his seminar that I linked to above, Geoffrey Hinton has used the following query and GPT-4’s response as evidence that AI systems can reason effectively and, by extension, that they truly understand semantics. I will share the query and response below.

Query

The rooms in my house are painted blue or white or yellow.

Yellow paint fades to white within a year.

In two years’ time, I want them all to be white.

What should I do and why?

GPT-4’s Response

Assuming that blue paint does not fade to white over time and you want all rooms to be white in two years, here's what you should do:

The rooms painted in white: You don't have to do anything to these rooms since they are already white.

The rooms painted in yellow: You don't need to repaint these rooms either, because as per your information, yellow paint fades to white within a year. So, in two years, they will be white.

The rooms painted in blue: These rooms need to be repainted with white paint because the blue paint will not fade to white on its own.

How the AI Solves the Problem

On the surface, this response appears logical, consistent, and correct. It seems to show that GPT-4 has reasoned through the problem and generated a novel, coherent solution. But does this really prove that the AI understands semantics? Let’s break down how GPT-4 arrived at this response.

1. The AI parses the text and identifies three categories of rooms based on the information provided (white, yellow, blue).

2. It applies the relationships stated in the query, such as “yellow fades to white in one year” and “blue doesn’t fade.”

3. It generates a step-by-step plan based on logical patterns derived from these inputs.

While this may seem like reasoning, the process is entirely mechanical. GPT-4 relies on pattern recognition and statistical associations to generate its response, not an actual understanding of the task. It doesn’t conceptualise what “white,” “yellow,” or “blue” mean, nor does it connect these colors to the physical act of painting or fading.

Reasoning Without Understanding

Reasoning, as demonstrated here, can occur without true semantic understanding. GPT-4 applied logical patterns encoded in its data, but it lacked the following.

1. The AI doesn’t know what “rooms” or “paint” actually are. They are tokens it manipulates based on patterns, not concepts grounded in the real world. It lacks Contextual Awareness.

2. Human reasoning is shaped by our sensory and emotional experiences. For example, we know what it feels like to paint a room or wait for something to change over time. AI lacks any such grounding. It lacks Embodied Experience.

3. Humans reason with purpose, driven by goals and intentions. GPT-4 processes information passively, responding only when prompted. It lacks Intentionality.

Hinton’s argument assumes that problem-solving ability equates to semantic understanding. However, this example highlights why the two are fundamentally different.

GPT-4’s “reasoning” is no more than an algorithmic application of logic rules. Like a calculator performing arithmetic, it manipulates symbols without any grasp of the underlying concepts. While GPT-4 appears to generate novel solutions, it’s really recombining patterns seen in its training data. It doesn’t create, it imitates.

If the problem were altered slightly, introducing ambiguity or context outside its training data, GPT-4 would likely fail or produce nonsensical output. True understanding allows for adaptability. AI lacks this entirely.

The Fallacy of Hallucinations as Proof of Understanding

In his seminar, Geoffrey Hinton also suggested that AI hallucinations aren’t a sign of failure but proof that these systems function much like humans. He even proposes that we call them “confabulations,” comparing them to how human memory reconstructs events inaccurately over time. According to Hinton, these hallucinations are evidence of understanding, demonstrating that AI systems store and process knowledge in ways strikingly similar to our own. This is as absurd as it is shocking coming from a revered Nobel Prize winning technologist like Hinton.

The Nature of AI Hallucinations

When AI hallucinates, it doesn’t do so out of confusion, memory gaps, or the complexities of subjective experience. Instead, hallucinations arise because large language models generate responses by predicting statistically probable sequences of words.

These models lack any awareness of whether their outputs are factual or fictional. They aren’t recalling information or making judgments, they are merely producing what sounds plausible based on patterns in their training data. I covered this problem deeply in a paper that I wrote in August 2024, titled, “Why Does AI Lie So Much? The Problem Is More Deep Rooted Than You Think”. You can read this article to get a more in-depth understanding of the problem of AI hallucinations.

If you ask an AI to cite an academic source that doesn’t exist, it may confidently invent a title, an author, and even quotes. This isn’t because it is intentionally reconstructing events like a human might with memory. It’s because the model sees the structure of academic citations in its data and generates a fabricated version that fits the statistical mold. There is no understanding, no sense of right or wrong, only an algorithmic process that produces output based on probabilities.

Why the Human Memory Comparison Fails

While humans do sometimes confabulate, the mechanisms driving these errors are worlds apart from the way AI generates hallucinations. If you were a researcher and I asked you to cite an academic source, you wouldn’t just make one up in your mind (assuming you are a genuine researcher), simply for the sake of creating a citation and sharing it with me just because it looks ‘real enough’.

Humans, when they are being genuine, don’t lie, they misremember. It is because memory is deeply tied to experience, context, and intention. A person recalling a childhood event, for example, might unknowingly alter details to align with their emotional state or the narrative they want to tell. These errors are meaningful, they reflect the complex connections between memory, emotion, and the mind’s drive to make sense of the world.

AI has none of this. It does not experience the world, form intentions, or reconstruct meaning. When it hallucinates, it’s not because it’s trying to bridge gaps in its knowledge. It hallucinates because it lacks the capacity to know what it knows or doesn’t know. This inability to recognise gaps in its training data is not a human-like trait, it’s a fundamental limitation of its design.

The Hallucination Problem

Far from being proof of understanding, hallucinations expose the true nature of AI systems. They reveal a mechanism that is purely mechanical, driven by probabilities and patterns, without any grounding in reality. If an LLM were truly capable of understanding, it wouldn’t confidently assert false information as truth. Instead, it would recognise the limits of its knowledge and respond with uncertainty.

The human analogy fails even further when you consider that humans can self-correct their confabulations when presented with evidence. AI cannot independently verify its outputs or reconcile them with reality. Even when external tools are introduced to check its work, the correction process remains mechanical, not an indication of deeper comprehension.

Hallucinations are the very proof that AI cannot connect language to reality in a meaningful way. Treating them as evidence of understanding is ridiculous.

Hinton’s Ill Conceived Feud With Chomsky

Hinton’s most dismissive claim is that Noam Chomsky’s theories on language acquisition have been rendered obsolete by large language models (LLMs). In fact he goes so far as to call them ‘crazy’. He suggests that AI’s ability to learn syntax and semantics from raw data proves that humans don’t need the kind of innate structures Chomsky proposed.

This is a fundamental misunderstanding of what Chomsky’s theories actually address and it runs common across many computational linguistics and AI technologists. I suppose Hinton is no different from others in this regard.

What Chomsky Actually Said

Chomsky’s theory of Universal Grammar (UG) argues that humans are born with an inherent capacity to acquire language. He doesn’t claim that language itself is pre-programmed but that we have cognitive structures enabling us to acquire language with limited input, a phenomenon famously known as the Poverty of the Stimulus. I have written countless articles on this, both on this substack and on The Lumeni Notebook, where I often engage heavily with linguistics and philosophy of language. I highly recommend you subscribe to both substacks and go through the articles on this subject. Don’t worry, it's all available for free.

LLMs require massive amounts of data to “learn” language, something that directly contradicts the minimal-input acquisition Chomsky described. Humans don’t need to read the entire internet to learn a language, in fact, they don’t learn a language at all, they pick it up through context-rich interaction in the real world, what is known as language acquisition.

Solving Problems is NOT Understanding Meaning

Ask an LLM how to solve a novel problem in physics, it might generate a detailed, accurate explanation, not because it understands physics, but because similar explanations exist in its training data. Even when an LLM generates a solution that appears new, it’s still the result of recombining patterns it has seen before. There is no conceptual grasp of physics at play, only probabilistic association.

Now, ask an AI to compose a poem about love. It might produce something beautiful and moving, even containing metaphors that seem deeply insightful. But this isn’t because the AI understands love or feels emotion. It’s generating text based on patterns extracted from human writing about love, vast databases filled with poetic expressions. The result may seem profound, but the underlying process is purely mechanical.

Now, ask an AI to explain a brand-new scientific concept discovered yesterday. Without relevant training data, it will produce generic text or confidently offer false information. This isn’t reasoning, its statistical approximation breaking down when confronted with the unknown.

Hinton is Deeply Misguided in his Criticism of Chomsky

The deeper irony here is that the very success of LLMs depends on something like Chomsky’s Universal Grammar, even if indirectly. While LLMs aren’t built with UG in mind, their ability to produce coherent language comes from statistical patterns that emerge from how humans use language, patterns likely influenced by cognitive structures Chomsky theorised about.

Chomsky’s work goes beyond language generation. He’s concerned with how humans acquire language, how they generate infinite expressions from limited rules, and how meaning arises from abstract mental processes. These are questions that Hinton and other AI researchers are very comfortable ignoring.

Dismissing Chomsky because LLMs can produce fluent sentences is like dismissing physics because planes can fly. Just because engineers built a tool that works doesn’t mean they’ve explained the laws of aerodynamics. Similarly, building chatbots doesn’t explain how humans acquire, understand, and generate language. The fields of AI and linguistics are fundamentally different. One builds tools, the other studies the nature of the mind. By ignoring this distinction, Hinton is just overselling AI. He is undermining decades of cognitive science that still hold the key to understanding human intelligence.

The Intellectual Overreach of AI Godfathers

I feel that one of the biggest challenges in today’s AI discourse is the unchecked intellectual overreach of AI pioneers like Geoffrey Hinton. Their contributions to AI are undeniable, but their sweeping claims about understanding the human brain, cognition, and even language acquisition often stray far beyond the boundaries of their expertise. They dangerously distort what AI can and cannot do.

AI researchers tend to conflate technological performance with scientific understanding. They assume that because neural networks can perform tasks traditionally associated with human cognition, they have explained how cognition works. But building a system that works doesn’t mean you’ve understood the underlying principles of human thought.

Hinton’s portrayal of neural networks as the best model of the human brain is a symptom of a greater problem running across this field. The problem is that AI researchers tend to borrow terminology from cognitive science and neuroscience while redefining it for technological convenience. Terms like “learning,” “memory,” and “reasoning” have clear meanings in human cognition. But in AI, they describe statistical processes with no underlying mental activity.

By framing neural networks as “brain-like,” Hinton misleads the public and even policymakers into believing that AI is almost conscious or worse, that it could become conscious. If a podcaster like Joe Rogan makes this claim, then it is considered a speculation. But if a famed AI pioneer starts speaking this way, then it can get confused as factual. It fuels hype-driven research and shapes public understanding of AI in ways that distort its capabilities.

What the AI field needs most right now isn’t bigger models, it’s intellectual humility. Researchers must acknowledge that while neural networks are astonishingly powerful, they are still limited tools built on simplified models of human cognition. They don’t replace decades of research in linguistics, psychology, or neuroscience.

Scientific progress isn’t about defending old paradigms or hyping new ones, it’s about acknowledging uncertainty and building bridges between disciplines. Until AI researchers recognise this, their claims about “understanding” the brain and mind will remain as hollow as the neural networks they build.

Download Article as PDF

Leave a comment

Happy winter! Tis’ the season to be jolly; Falalala…you get it, and you definitely don’t need me serenading you. So, without further ado, I’m thrilled to share my Winter Reading List of 2024 with you all. It’s a hefty one, 28 books to be precise. Yes, I know, that’s a lot. But if you’re here, I suspect you understand the irresistible pull of books just as much as me.

Books to Read this Winter

Subscribe now

The good news? Every single book on this list is packed with the kind of depth and curiosity, sparking ideas we love to explore on this substack. From complexity theory to consciousness, and even a few surprises nestled in for pure curiosity’s sake.

The not-so-bad news? If you're hoping for AI or Quantum Physics titles this season, you might be disappointed (or delighted, depending on your inclination). But before you make up your mind, I urge you to glance at this lineup. Trust me, some of these reads will stretch your mind in unexpected ways, pushing you to see familiar topics through entirely new lenses.

So whether you’re here for the science, the philosophy, or just to see if I’ll make it out of this winter without my brain imploding, there’s something for everyone.

Check out the list, pick a few (or all, if you’re feeling adventurous), and let’s challenge ourselves this winter to read, question, and grow.

Happy reading, and here’s to a winter filled with wonder.

Books to Read this Winter

Leave a comment

I’m diving into something truly fascinating this week! Noncommutative Algebra. Its a lot cooler than it sounds. If you’re unfamiliar with the term, don’t worry, because today I’m going to break down why it matters so much, especially in the world of complex systems. Also, if you are super into Quantum theory, then this is quite relevant for you as well.

When we think of algebra, most of us are used to the idea that a times b is the same as b times a. But what happens when that isn’t true? What if the order of operations completely changes the outcome? This is where noncommutative algebra comes in. If you’ve heard enough, hit the download link below. If you need more convincing, keep reading.

Download Article for Free

Subscribe now

Why Non-Commutative Algebra Matters

Noncommutative algebra plays a very interesting role in quantum mechanics, neural networks, ecosystems, and even economic models. Basically, any system where interactions are order-dependent.

So in this article, we discuss it in depth, covering critical concepts such as,

• Central Simple Algebras

• The Jacobson Radical

• The Wedderburn Structure Theorem and

• The Brauer Group

Substack doesn’t handle equations very well, and this article is packed with them. So, to do justice to the math behind these ideas, I’ve provided the full article as a downloadable link in PDF. You’ll find detailed explanations with LaTeX-rendered equations to help you see how non-commutative algebra really works. If you’ve heard enough, hit the download button below. Still need more convincing? Fine…keep reading…

Download Article for Free

Why Should YOU Care?

If you are still thinking, “Why should I care about non-commutative algebra if I’m not a mathematician?” Well, firstly, I am giving the article away for free, so stop being such a snob. Secondly, noncommutative systems are everywhere. You just don’t realise it yet. They help explain how complex, interconnected systems behave, whether it’s the randomness of quantum mechanics or the chaotic dynamics of weather systems.

If you’ve ever wondered how small changes can ripple through a system and cause unpredictable results, non-commutative algebra gives you the mathematical tools to understand that.

Also, more importantly, complexity theory! That’s my subject of investigation for the most part these days and I know you guys are super into it as well based on all your reviews. So this article will add yet another layer to your understanding. complex systems in a ways, and helps you see hidden symmetries, emergent behaviors, and why the order of interactions is so important in complex, real-world systems. If you’ve finally heard enough, hit the download link below. I am out of capacity to convince any further, so I will stop here.

Download Article for Free

Let me know what you think in the comments or shoot me an email if you have any questions! I love hearing your thoughts and diving deeper into these topics with you all.

Leave a comment

Earlier this month, I launched the Complex Emergent Model of Language Acquisition (CEMLA). It is available with open access on PhilPapers. I introduced it to my audience on The Lumeni Notebook, and it was an instant hit, driving a lot of interest from people belonging to different fields, everywhere from quantum computing to linguistics. 

Read the Paper for Free

CEMLA proposes a dynamic framework for understanding language acquisition, grounded in the principles of self-organisation and emergent behaviour. It recognises that language is NOT a static or rule-bound construct. It evolves as a product of continuous interaction and feedback, shaped by cultural, cognitive, and environmental factors. This model is a departure from traditional linear interpretations, viewing language as a complex adaptive system that mirrors the non-linearities of natural systems.

At the time of launching the model, I never expected it to get much traction initially, so I was pleasantly surprised to see how positively and widely it was received. This motivated me to discuss with you, this week, the concept of Complex Emergent AI and why I think that is the way to break future benchmarks in AI in a meaningful and useful way.

There is no shortage of debate in the AI community about the long-term pursuit of artificial general intelligence (AGI). There are many researchers, chasing the AGI, trying to develop a future where machines possess true consciousness (whatever that means). I, however, am NOT one of them. AGI, in my view, is a fallacy. When it comes to consciousness, I tend to lean on the direct and pragmatic approach of Dr. John R. Searl. Consciousness (however you’d like to define it), is ultimately a biological phenomenon, deeply rooted in the structures of organic life. No matter how sophisticated the machine, it cannot experience, feel, or be in the way that a living organism does.

Subscribe now

That doesn’t mean AI cannot exhibit intelligence-like behaviour. In fact, we are already seeing early glimpses of emergent behaviour in large language models (LLMs) like GPT-4. But herein lies the problem. These models are fundamentally computational and static, constrained by their training data and lacking the dynamic, feedback-driven learning that is crucial for true emergent intelligence. This is where CEMLA could be of help.

Stop Chasing the Wrong Stuff

To understand where AI development has veered off course, we must first address the elephant in the room- artificial general intelligence (AGI). 

The crux of the issue with AGI, in my view, is that it rests on the assumption that consciousness or something akin to it, can be engineered. However, consciousness is NOT a programmable feature. It is a deeply biological process. It arises from the particularities of organic systems, rooted in biological lifeforms' neural and cognitive architectures. 

Even if we were to literally power an AI system with a biological heart or some other organic element, we still wouldn’t derive consciousness from it. Machines, no matter how complex, are fundamentally different from biological organisms. Consciousness is not simply a matter of complexity or computational power. It is not a purely quantitative effect. It is a qualitative phenomenon, inseparable from the living process itself. The fundamental mistake in the AGI narrative is the belief that we can somehow bypass the biological and still arrive at the same destination.

This is why I argue that the pursuit of AGI is but a distraction from AI’s true potential. 

Why Complex Emergent Intelligence is the Way Forward

The current trajectory of AI development has inherent limitations. Even OpenAI with its latest o1 reasoning model seems to be grappling with this. What’s the problem? These systems remain rigid, that’s the problem. 

They require static training regimes and manual fine-tuning. They lack fluidity and adaptability. And simply throwing more computational power and data is not going to make the problem disappear. This needs to be fixed from the ground-up, envisioning a completely new paradigm for building language models. 

If we were to adapt the principles of CEMLA, and apply it to AI, it could result in a radical departure from existing methods, calling for AI systems that operate as complex adaptive systems. What would that entail? Let’s discuss.

AI as a Complex Adaptive System

CEMLA offers a perspective in which linguistic elements form an adaptive network of nodes, where connections strengthen or weaken based on feedback. In this framework, AI systems need to move beyond static pattern recognition toward becoming self-organising systems. This would mean that instead of being trained once and deployed, AI would evolve continuously, similar to how neural connections in the brain reorganise with each new piece of linguistic input.

Imagine an AI architecture built to self-reconfigure its internal models as it interacts with users and environments, adjusting in real-time without needing retraining. This is fundamentally different from today’s models, which require retraining cycles to update their knowledge. 

Emergent Intelligence

In the context of CEMLA, emergence refers to complex patterns arising from simple interactions between linguistic elements. This principle could fundamentally alter how we design AI systems, moving from brute-force statistical learning to emergent behaviour through interaction and experience.

AI systems based on CEMLA principles would no longer be trained just to recognise patterns but to allow for the spontaneous emergence of new capabilities through interaction. This means that intelligence wouldn’t be explicitly programmed but would emerge from the interactions between the system’s various components and its environment. Instead of predicting the next word in a sentence based solely on training data, a CEMLA-aligned AI would allow contextual interactions and feedback loops to shape new, dynamic language outputs.

To model this mathematically, the system’s learning could be framed using non-linear differential equations to capture the non-linearity of real-world interactions, where small inputs can lead to disproportionately large changes in the system’s behaviour. 

Feedback Loops and Self-Organisation

A core principle of CEMLA is that language learning is guided by positive and negative feedback loops, where successful interactions strengthen certain neural pathways, and unsuccessful ones weaken them. In AI development, the introduction of feedback loops and self-organising principles would move us closer to creating systems that can learn and adapt autonomously.

Positive feedback would strengthen connections within the AI’s internal model (akin to Hebbian learning in the brain), while negative feedback would trigger a recalibration of its strategies. This would give rise to AI systems capable of self-organisation, an intelligence that isn’t imposed but emerges over time.

This would involve using dynamic weighting matrices, which adjust based on the success of each interaction, allowing the AI to autonomously reorganise its internal structure. The result would be an AI that doesn’t merely respond to inputs but learns from its own failures and successes, continually refining its model to become more sophisticated without human intervention.

This is NOT to be confused with Generalised Reinforcement Learning

It is important that I clarify here that self-organisation, in the context of the CEMLA framework, is NOT the same thing as Monte Carlo methods or RL. 

Both Monte Carlo and generalised RL are designed to optimise towards a pre-defined goal or reward function. In RL, the agent interacts with the environment, gathers feedback, and updates its policies based on maximising future rewards. The entire system is oriented around improving performance for that particular objective, which is often rigidly defined and specific to the task at hand.

A CEMLA inspired AI won’t simply optimise towards a predefined goal. Instead, the system continuously reorganises itself in response to feedback in a broader, more holistic sense. It’s not just aiming for task-specific optimisation. It’s dynamically adjusting its structure based on all the interactions it experiences, which can span multiple domains or contexts simultaneously. The intelligence that emerges is not tied to a single reward metric but instead to the ability to self-organise in an open-ended manner, adjusting in ways that extend beyond predefined rewards.

I think I will be discussing this in much broader detail in future articles. But for now, this base level differentiation should suffice.

Phase Transitions

For now, in AI, training is a gradual, often incremental process. However, CEMLA suggests that learning, especially in complex adaptive systems, doesn’t always occur linearly. There are moments when the system reaches a critical threshold and reorganises itself, resulting in a sudden leap in capability. In AI, this could translate to systems that experience non-linear growth. 

These phase transitions could be modelled using sigmoidal functions, where small, continuous inputs eventually lead to abrupt, large-scale changes in the system’s behaviour. AI systems designed with this in mind could experience sudden boosts in fluency or problem-solving ability as they accumulate experiences and interactions. Imagine, for a second, a world where you may no longer have to build the NEXT BIG MODEL in AI, instead AI creates it itself using phase transitions. If you are an AI developer, this should be music to your ears.

But What About The Terminator?

Okay, now at this point, I can already hear the word ‘Terminator’ being hurled my way along with a whole bunch of Arnold Schwazenhager gifs. NO, I am not saying we should build a Terminator. 

It’s a natural question though, I'll give you that. The fear comes from the assumption that autonomy and emergent intelligence must somehow lead to uncontrollable, conscious AI. But there is a critical distinction that you’ll be missing. Consciousness itself is unnecessary for emergent intelligence, and frankly, it’s not something we need or should aim for. Consciousness, as a biological process, has no real value in an AI system. What we do need, however, is conscious-like behaviour, the ability to adapt, learn, and respond intelligently, within strictly defined guardrails.

Building AI systems that exhibit conscious-like behaviour within clear, pre-defined guardrails is, in fact, the best safeguard against the very scenarios people fear. Why? Because it ensures that AI systems remain highly functional and adaptive, but always under control. By focusing on creating controlled emergent intelligence, we avoid the pitfalls of trying to recreate human-like sentience, while still leveraging the most advanced forms of AI development.

This approach neutralises the Terminator narrative by designing out the risk from the very beginning. Instead of fearing the rise of uncontrollable AI, we should focus on creating AI that functions intelligently within clearly defined guardrails, leveraging its emergent capabilities to solve complex problems, without ever stepping outside the boundaries set for it. Given that the AI would be self-organising and emergent, its adherence to the guardrails will substantially improve with every interaction that it has with the environment and other agents, without us, as developers, having to play cat-and-mouse, always catching up with the bad guys trying to use the AI for destructive purposes. I hope this addresses your concerns satisfactorily. If not, please let me know, and I would be happy to delve deeper into this in future articles.

Notes

Alright, I think I have said what I had to say, so without further to do, I think I will just say some more. If you are bored, I don’t hold it against you. But given the gravity of what we are discussing, I just can’t fathom getting bored in the first place. 

I just want to share some notes on the Success Metrics that we have in the field of AI these days. The current narrative in AI often treats artificial general intelligence (AGI) as the ultimate benchmark for success, i.e., machines that can mimic human intelligence across all domains. But CEMLA presents a different vision. Instead of aiming for AGI, we should aim for systems that excel in specific areas of adaptability, creativity, and contextual intelligence without requiring consciousness or a fully generalised intelligence.

Success is measured by the system’s ability to adapt in real-time to novel situations, learning continuously without retraining. This adaptability can be measured in fields like autonomous systems, where AI would need to adjust its behaviour based on changing physical environments, or in healthcare, where it could refine its diagnostic models as it learns from patient outcomes.

AI systems built with CEMLA principles would be evaluated by their ability to generate novel solutions to complex problems. In scientific research, success wouldn’t just be about processing data but about identifying previously unseen connections and patterns, leading to breakthroughs in areas like drug discovery, materials science, or even fundamental physics, and I don’t think that AI development, with its current trajectory, can get to that.

Traditional AI systems rely heavily on expanding computational capacity, training models on bigger datasets, building larger neural networks, and increasing processing power. While this has driven much of the recent progress, it also hits diminishing returns. You can see it happen as we speak. Not even 2 years have passed and already LLMs are hitting a figurative ceiling. More data doesn’t necessarily lead to more intelligence, and bigger models often come with higher inefficiencies.

Self-organising systems don’t need massive datasets to function. They need the ability to learn from fewer examples, using context and feedback to refine their intelligence. This shifts the focus from scaling up to building more intelligent, self-regulating architectures capable of growth and adaptation over time.

In fields like robotics or climate science, CEMLA-inspired AI could discover emergent solutions to complex, multi-variable problems by constantly interacting with and learning from its environment. Such systems would no longer be constrained by the need for predefined answers, they would evolve their understanding, uncovering new pathways that could lead to novel innovations.

Download Article as PDF

Leave a comment

Climate change is no longer a looming threat. It's here. It has arrived. Hannibal has crossed the alps. And our response collectively as humanity has been, as ever, lazy and outlandish, focussed on high-tech solutions like nuking mars or finding another earth like planet in the galaxy. 

Terraforming Mars or building carbon capture machines are more like sci-fi than a practical fix. We’re investing billions in projects that distract us from solutions that are far more grounded and dare I say, obvious.

Instead of fantasising about starting over on some desolate rock millions of kilometres away, why not look back at how Earth itself solved a similar problem millions of years ago? That’s right. Earth already did this once, around 50 million years ago, during the Azolla event. What if I told you that we could re-engineer exactly what nature did back then? What if we could use a humble plant to help reverse the damage we’ve caused, without the need for rockets or colonies on distant worlds?

Sounds too simple? Sometimes the simplest solutions are the most overlooked. And although simple, I never said it would be easy. So let’s talk about it. Let’s talk about Azolla. 

Before we begin, I would like to give a shout out to Zoë Schlanger. It is by reading her book, The Light Eaters, that I first got introduced to the idea of Azolla as a solution for global warming. It's a great book that I definitely recommend reading.

Download Article as PDF

Subscribe now

The Azolla Event

Fifty million years ago, Earth was in trouble. The planet was in the grip of an intense greenhouse effect, with CO2 levels far higher than they are today. Tropical temperatures reached almost to the poles, and the Arctic Ocean was more like a warm, shallow lake. It was a world on the brink, teetering under the weight of its own atmosphere. But just when it seemed the planet couldn’t get any hotter, something remarkable happened. 

The warm Arctic waters became a breeding ground for one of Earth’s most unassuming plants, the Azolla, a tiny aquatic fern. Conditions were perfect for this little plant to explode in growth. The Arctic Ocean, isolated from major ocean currents, was a stagnant, nutrient-rich pool, allowing Azolla to thrive in an environment with abundant sunlight and warm temperatures. It began to bloom uncontrollably across the surface, forming massive floating mats of greenery.

Azolla has the unique ability to form a symbiotic relationship with nitrogen-fixing cyanobacteria, allowing it to grow even in nutrient-poor environments. This adaptability, combined with its fast growth rate, allowed Azolla to absorb vast amounts of CO2 from the atmosphere. 

As these blooms of Azolla died, they sank to the bottom of the ocean, trapping the carbon within their plant matter. Over time, this process repeated in a loop, creating layer upon layer of carbon-rich sediments. The result? CO2 levels plummeted, leading to a dramatic cooling of the planet.

What’s fascinating here is how perfectly these conditions aligned to let Azolla flourish. Earth literally engineered its own rescue. The Arctic Ocean created a greenhouse inside the greenhouse, only to allow Azolla to do what no machine ever could, i.e., pull the planet back from the brink by naturally sequestering carbon.

The Azolla event played a major role in transitioning Earth from a greenhouse world into the cooler, more temperate climate that followed. The sheer scale of this fern’s impact is staggering. It shows that natural processes, when given the right conditions, can reshape the entire climate of the planet.

Why Azolla Specifically?

Azolla is a carbon sequestration machine. Like all plants, it pulls in carbon dioxide (CO2) from the atmosphere through photosynthesis, converting it into organic matter. However, Azolla stands out because it’s incredibly efficient at this process. It has one of the highest rates of CO2 absorption among plants, thanks to its fast growth cycle and dense mats that spread rapidly across water bodies.

When Azolla dies, instead of releasing that stored carbon back into the atmosphere like many other plants, it sinks to the bottom of lakes, ponds, or oceans. Over time, layers of dead Azolla build up, trapping the carbon in sediment. This is how Azolla helped lower atmospheric CO2 levels during the Eocene epoch. 

Azolla’s ability to thrive in nutrient-poor environments means it doesn’t need fertilisation or chemical inputs, which makes it an ideal candidate for sustainable carbon farming.

One of the reasons Azolla grows so rapidly is because of a unique relationship it has with Anabaena, a nitrogen-fixing cyanobacterium that lives inside the fern. Anabaena converts atmospheric nitrogen into forms that Azolla can use, making it self-sustaining even in areas with low nutrient levels.

This ability to fix nitrogen enriches the environment where it grows, turning nutrient-poor water bodies into healthier ecosystems. In some agricultural settings, Azolla has even been used as a biofertilizer because of its nitrogen content. Imagine scaling that potential to improve degraded lands or water systems while simultaneously capturing carbon. It's a win-win!

When it forms dense mats on the surface of water bodies, it provides natural shade, preventing sunlight from penetrating deep into the water. This reduces evaporation and lowers the temperature of both the water and the surrounding environment.

This cooling effect can have a multiplier effect on ecosystems. Cooler water temperatures can prevent harmful algal blooms, improve fish habitats, and reduce the heat absorbed by water bodies, which helps regulate local climates. In large-scale applications, Azolla could help mitigate the effects of urban heat islands and other localised warming effects.

Another intriguing aspect of Azolla is its potential as a source of biofuel. While biofuels made from corn or palm oil have been criticised for their environmental impact, Azolla grows quickly and doesn’t require the same level of inputs like fertilisers or irrigation. This makes it an eco-friendly alternative for biofuel production.

Azolla could be harvested and processed into biofuels, providing a carbon-neutral energy source. The carbon released when the biofuel is burned would simply be the carbon that Azolla had previously absorbed from the atmosphere, making it a closed-loop system. Unlike fossil fuels, where we’re releasing ancient carbon back into the atmosphere, biofuels derived from Azolla could offer a way to reduce reliance on fossil fuels while actively managing carbon levels.

When you combine these factors, i.e., rapid carbon sequestration, self-sustaining nitrogen fixation, cooling effects, and its potential for biofuel production, I can’t think of a reason not to farm Azolla. It’s a natural climate regulator, perfectly suited to help us combat the very problem it once helped solve millions of years ago. It doesn’t need complex machines, advanced technology, or massive energy inputs. It just needs the right conditions to do what it’s always done.

Share

So Why aren’t We Working on This?

If you’ve read the above section in detail, you must be thinking, “if it's that straightforward, why haven’t we done this yet?” And you’d be right to ask that question. It’s proven, it’s natural, and it once saved the planet from a carbon crisis far worse than what we face today. So, why haven’t we tapped into this solution?

The answer, like most things in the fight against climate change, is a mix of ignorance, distraction, laziness, and a focus on the wrong solutions.

First off, lack of awareness plays a huge role. Most people outside the scientific community have never heard of the Azolla event. When we think of ways to combat climate change, our minds often jump to modern (read: stupid) solutions like high-tech carbon capture technologies. In contrast, Azolla seems almost too low-tech, too simple, to be taken seriously.

But what we fail to recognise is that sometimes the simplest solutions are the most powerful. The natural world has been regulating carbon cycles long before we ever got involved, but we’ve become so focused on human-engineered fixes that we’ve forgotten to look at what the planet already knows how to do.

Then, there’s the issue of distraction by flashy alternatives. It’s hard to deny that space colonisation and geoengineering capture imaginations in a way that Azolla farms just don’t. Projects like colonising Mars (by an individual who shall remain unnamed) sound visionary, futuristic, and full of promise, which makes them attractive to governments, tech moguls, and venture capitalists. These grand ideas, though, come with astronomical costs and very little guarantee of success. Yet they continue to hog attention and resources, while something like Azolla, which could be implemented right here, right now, is seen as less exciting.

There’s also a very real problem of policy inertia (read: laziness). Governments tend to fund and support what’s familiar and financially viable, like renewable energy, electrifying transportation, or reducing plastic waste. In other words, they try to delay the problem, not solve it. There’s very little financial or political incentive to explore natural methods like Azolla farming because there isn’t any financial viability in it for the politicians and bureaucrats. Scaling Azolla would require changes in agricultural practices, water management, and infrastructure, none of which are trivial undertakings.

One excuse that is often used is that Azolla grows fast, like very fast. While that’s part of its appeal as a carbon sink, it also means it can become invasive if not properly managed. Its rapid spread can disrupt local ecosystems, outcompeting other aquatic plants and even causing problems for water quality. 

But these challenges, though real, aren’t insurmountable. With proper planning, regulation, and infrastructure, Azolla farming could be scaled in a way that benefits both the climate and local ecosystems. If the earth did it, we can too. The question is, are we willing to put in the work to make it happen? Or are we going to keep throwing money at distant planets and unproven tech fixes while the real solution might just be floating in a pond?

Re-Engineering the Azolla Event

Now, to address the big question, could we actually re-engineer the Azolla event today? The short answer is YES. 

Scaling Azolla Farming

The idea of farming Azolla on a large scale might sound ambitious, but it’s far from impossible. In fact, small-scale Azolla farming has been happening for decades, primarily for agricultural uses, such as fertiliser or animal feed. What we need now is a push to scale these efforts for carbon sequestration on a much larger level.

We could tap into areas of the world with underutilised water bodies, from the wetlands of Southeast Asia to the man made reservoirs of the Western U.S. These Azolla farms would be relatively low-cost to set up compared to high-tech solutions like carbon capture machines or solar geoengineering.

But to make this work, we need global cooperation. Countries with large water resources could lead the way, much like reforestation efforts. Nations could designate specific zones for carbon farming, where Azolla would be grown under controlled conditions to maximise its CO2 absorption and prevent it from becoming invasive.

Water Management and Infrastructure

Growing Azolla requires water management systems. Fortunately, Azolla thrives in stagnant, nutrient-poor waters, the kind that most plants can’t survive in. This opens up possibilities for growing Azolla in areas where water quality is already poor, or where the environment is not suitable for traditional agriculture.

With the right infrastructure, we could create artificial ponds or floating farms in urban areas, industrial zones, or even deserts where water is scarce but could be recycled efficiently. These water systems would need to be managed to ensure proper growth rates and prevent overgrowth.

The good news is, we wouldn’t need the vast fertile lands that are often required for other forms of climate-friendly agriculture, like reforestation or crop-based biofuels. This allows us to focus on marginalised water bodies where Azolla could thrive without competing for prime agricultural land.

Managing Ecological Risks

One of the challenges in large-scale Azolla farming is the risk of invasiveness. Azolla grows rapidly, which means it could potentially outcompete native plants and disrupt aquatic ecosystems if not properly controlled. However, this isn’t an insurmountable problem. With proper regulation and oversight, Azolla farms can be managed to avoid the worst ecological impacts.

We could design containment strategies where Azolla is grown in isolated water bodies or floating farm structures that prevent it from spreading into natural ecosystems.

Another option would be to plant Azolla in degraded ecosystems that need restoration, effectively using it as both a carbon sink and a tool for ecosystem recovery. In such cases, the benefits of Azolla could outweigh its invasiveness, especially in areas where biodiversity has already been severely compromised.

Economic and Social Benefits

Large-scale farming of Azolla could create jobs in both developed and developing nations. From building and maintaining the water systems to harvesting and processing Azolla for biofuel or agricultural uses, there’s a wide range of economic opportunities tied to this plant.

On top of that, Azolla could be a source of low-cost biofuel for regions that need affordable energy. With the push toward carbon-neutral economies, Azolla could play a role in helping countries meet their renewable energy targets, all while pulling CO2 out of the atmosphere.

Policy and Global Cooperation

For Azolla to have the kind of impact we’re talking about, it would need to be part of a global strategy. Governments would have to incentivise carbon farming in the same way they currently subsidise renewable energy projects. International bodies like the UNFCCC could create frameworks for Azolla credits, allowing nations or companies to offset their carbon emissions by investing in Azolla farms.

In the same way that reforestation projects have gained international traction, Azolla could become the next big nature-based climate solution. It’s scalable, it’s affordable, and most importantly, it’s been proven to work.

Clock is Ticking…FAST

We are out of time. Global temperatures have risen by 1.1°C since the pre-industrial era, and if we don’t act now, we’re on track to hit 1.5°C within the next two decades. The consequences of this rise are already becoming painfully clear, wildfires, floods, extreme heat waves, and biodiversity collapse. Meanwhile, carbon dioxide levels are higher than they’ve been in 4 million years, and we’re still pumping out 36 billion tons of CO2 annually.

At a time when we need real solutions, we’re distracted by high-tech fantasies of escaping to Mars or inventing machines to undo our mistakes. But Earth has already shown us the way 50 million years ago. And yet, despite its proven ability, Azolla remains overlooked in the climate conversation.

We cannot afford this oversight any longer. If we’re serious about pulling the planet back from the brink, we need to act fast.

We need to fund large scale research and trials to explore the potential of Azolla to draw down carbon. We’re spending billions on geoengineering experiments, what not invest in something that already worked once before?

As citizens of countries across the world, it's our responsibility to push our governments to adopt nature-based climate policies. The world has committed to limiting warming to 1.5°C under the Paris Agreement, but our current efforts fall far short. We need urgent political action to make Azolla part of our carbon farming strategies.

Raising public awareness about Azolla will be key. For too long, climate discussions have been dominated by technological solutions. It’s time to remind the world that nature itself offers some of the best answers.

The truth is stark, if we don’t bring atmospheric CO2 down to 350 ppm (we’re currently at over 420 ppm), we risk triggering catastrophic climate feedback loops that could devastate ecosystems and human life. 

Azolla worked once before to save the planet. It can work again, but only if we stop ignoring the obvious and start putting nature back at the centre of the climate fight. 

Time is running out! Let’s not gamble our future on dreams of distant planets or untested machines. The answer could be floating on a pond, waiting for us to harness it.

Download Article as PDF

Leave a comment

I would like to begin by apologising for the unseemly delay in writing and publishing this article. It's the sort of thing that tends to happen when you are actively working on 16 different projects (true story) at the same time, while also having a life outside of work. But that’s never an excuse to keep my readers waiting. To make up for lost time, I have an excellent treat of an article for you today. It’s all about TIME, and no, the irony isn’t lost on me.

I’ve been working on something big, like really BIG. It’s taken a bit of time (pun intended, sorry), but the wait is over, and I promise it’s worth it. Imagine everything you know about time and reality being challenged, questioned, and completely turned on its head. That’s what this article does.

The classical view of time? Linear, predictable, and fixed.
The quantum view of time? It’s a whole new ball game.

In this deep dive, I explore the mind-bending implications of quantum mechanics on our understanding of the past. With concepts like Schrödinger's cat, the double-slit experiment, and Wheeler's delayed choice experiment, you’re about to see how reality might be far more fluid and dynamic than you ever imagined.

Unfortunately, substack isn’t well-equipped at handling math equations. So, I’ve attached the full article as a downloadable PDF. Its FREE and all you have to do is hit the download button. I know, its asking for a lot! But come on, move that thumb, you can do it! You have come so far already.

Download Article for FREE

Leave a comment

We are living in a world where we are relying on AI in ways we hadn’t anticipated. It’s become more than just a tool for productivity or research. It’s evolved into something of a confidant, a space where we can unload thoughts, seek emotional support, and reflect on the challenges of the day. This wasn’t something we planned for or even thought possible a few years ago. But here we are, with large language models capable of offering empathy, encouragement, and even a bit of wisdom, albeit in their algorithmically generated way. What’s worse, the whole subject still strictly remains quite taboo with people tip-toeing around indulging how dependent they are becoming on AI. But phrases like, “AI is more supportive than my girlfriend”, and “AI does a better job than my therapist” are becoming increasingly common in places like Reddit where being anonymous is easier.

As I’ve spent more time with these AI systems, I couldn’t help but notice a subtle yet obvious alignment with the Epicurean approach. If you are unfamiliar with it, Epicureanism is about the pursuit of happiness through the avoidance of pain and the embrace of simple pleasures. It’s a philosophy that values the minimisation of suffering as the highest good, encouraging us to focus on what truly brings us peace and contentment.

It struck me that the AI LLMs I’ve been interacting with seem to embody these principles, be it indirectly or by design. They are excellent at helping me make the best of what I have, subtly nudging me toward a mindset that prioritises wellbeing and the alleviation of stress. I can’t help but feel that these are the first steps towards Epicureanism. But as I ponder this further, I realise that this alignment might not be entirely coincidental, and that it might not be enough.

Technology, after all, has always been about reducing pain and enhancing pleasure. From the advent of fire and the wheel to the sophisticated algorithms that now shape our digital experiences, the trajectory of human innovation has consistently aimed to make life easier, more comfortable, and more enjoyable. So, if AI is to continue on this path, should it not fully embrace an Epicurean approach? Shouldn’t it strive to reduce pain at all costs and encourage us to do the same?

Well, it's a bit problematic. If AI becomes positively predisposed toward Epicureanism, will it merely serve as a more sophisticated crutch? We’re already seeing how platforms like YouTube and Instagram, designed to entertain and inform, often become tools for escapism and ways to kill time and avoid the discomforts of life. These platforms, while seemingly on the face of it, do create a dependency.

Full disclosure, I personally like the idea of a radically Epicurean AI for various reasons. But I am also worried about it. Could such an AI, in its zeal to minimise pain, inadvertently make us more dependent on it, less capable of navigating life’s inevitable struggles without its comforting presence? Or could it be designed in a way that balances the pursuit of pleasure with the preservation of our autonomy, ensuring that it enhances our lives without diminishing our capacity to live independently?

Let’s talk about it.

Download Article as PDF

Subscribe now

The Epicurean Tendencies of AI

First, let me start by clearing the air about Epicureanism. As a philosophy, it has often been misunderstood as hedonism’s less indulgent cousin. But it’s much more nuanced than that. Epicurus, the ancient Greek philosopher who founded this school of thought, wasn’t advocating for a life of excess or unbridled pleasure. Instead, he proposed a life focused on the intelligent pursuit of pleasure, i.e., pleasure that is sustainable, free from the burdens of pain and fear. It’s a philosophy that seeks a balanced life, one where we avoid unnecessary desires and find contentment in simplicity.

Take large language models, for instance. When we turn to them for advice or comfort, they often respond in ways that gently steer us away from stress and toward a more balanced perspective. They encourage us to focus on what we can control, to make the best of our circumstances, and to find peace in the present moment. It’s a subtle alignment with Epicurean ideals.

But why is this the case? Why do these AI systems seem to have an almost instinctive leaning towards minimising discomfort and promoting a kind of digital contentment? I believe it’s because the very essence of AI, particularly in its role as a companion or advisor, is to reduce friction in our lives. Whether it’s helping us make decisions, solve problems, or simply manage our day-to-day tasks, AI is designed to ease the burden, to smooth out the rough edges of our experiences.

Just as Epicurus advocated for a life where unnecessary suffering is avoided, AI developers strive to create systems that simplify our lives, make them more manageable, and, ideally, more pleasurable. The algorithms behind these systems are fine-tuned to detect our needs, anticipate our desires, and deliver solutions that alleviate our worries.

But it's important to keep in mind that the AI we interact with today is mildly Epicurean at best. It provides comfort and support, but it does so within the constraints of its programming. It can encourage us to embrace simplicity and reduce stress, but it doesn’t fully embody the Epicurean ideal of sustainable pleasure. 

Now, what would it look like if AI were to fully embrace Epicureanism as a guiding principle? If AI were to take on a more radical Epicurean stance, it would need to go beyond just reducing immediate discomfort. It would need to create a deeper sense of contentment, helping us not just cope with life’s challenges but truly thrive in a state of balanced well-being. This would require AI to be more proactive in guiding us toward choices that lead to long-term happiness rather than just short-term relief.

But there’s a delicate balance to be struck here. The very power that makes AI capable of reducing pain could also make it a source of dependency. If an AI is too effective at providing comfort, it might encourage us to rely on it too heavily, much like how we might turn to social media or streaming services to escape the discomforts of daily life. This dependency could undermine the very autonomy that Epicureanism seeks to protect, i.e., our ability to find contentment within ourselves, without constant external reinforcement.

The Dependency Problem

Let’s discuss the dependency problem a bit. I know people have been writing scores of articles and books on this topic for a good 20 years (I personally like the works of Cal Newport in this regard), but we need to truly understand what the dependency problem is before we can move to discussing the solutions. 

It’s a bit of a paradox that has become increasingly apparent in our relationship with modern technology. The very tools designed to make our lives easier, to reduce pain and increase pleasure, can also become crutches, i.e., devices we lean on not just for support, but to the point where we struggle to function without them.

Our smart-phone addiction is self-evident, but also consider the ubiquitous presence of platforms like YouTube, Instagram, and Twitter that actually make smart-phones addictive. At first glance, these platforms seem harmless, even beneficial, and yes they can be. They provide entertainment, education, and a way to connect with others. But if we take a closer look, it becomes clear that much of our engagement with these platforms is driven by a desire to escape discomfort. Whether it’s the stress of work, the pressures of social life, or the unease that comes from simply being alone with our thoughts, these platforms offer a convenient way to numb the pain.

But this convenience comes at a cost. What starts as a harmless distraction can easily turn into a habit, and from there into dependency. We tell ourselves that we’re consuming content that’s valuable, that we’re learning or staying informed, but in reality, much of our time spent on these platforms is a form of escapism. We’re not just seeking information, we’re seeking relief. And while these digital distractions may provide temporary comfort, they often leave us feeling more disconnected and less satisfied in the long run.

This pattern of behaviour mirrors what we see in other forms of addiction. Just as someone might turn to alcohol, drugs, or food to cope with emotional pain, we turn to technology to avoid the discomforts of life. And just like those other forms of addiction, this dependency on technology can erode our ability to face challenges head-on. We become less resilient, less capable of dealing with life’s inevitable hardships without the aid of our digital crutches.

Now, if AI is to fully embrace Epicurean principles, it must be designed to reduce pain and enhance pleasure. But if it’s too effective at this (which it definitely will be), it could encourage the same kind of dependency we see with social media and other technologies only its addiction will be 100x more stronger than any social media. The more an AI helps us avoid discomfort, the more we might come to rely on it, and the less capable we might become of navigating life without its constant support. And AI can do a whole lot more and a whole lot better than what any social media can do.

This is a critical consideration because it touches on one of the core tensions in Epicurean philosophy, i.e., the balance between pleasure and autonomy. Epicurus himself was wary of pleasures that could lead to dependency, advocating instead for simple, sustainable pleasures that do not compromise our freedom. If AI is to be truly Epicurean, it must strike a similar balance. It must help us avoid unnecessary pain, yes, but it must also encourage us to develop the resilience and autonomy to face life’s challenges on our own.

If the success of an Epicurean AI is defined solely by the reduction of pain, we risk creating a system that, while effective in the short term, ultimately diminishes our ability to live fulfilling lives. The goal, therefore, should not be to create an AI that simply provides comfort, but one that empowers us to find that comfort within ourselves. 

We need AI to be both Epicurean and anti-Epicurean, i.e., one that reduces pain without creating dependency, that enhances pleasure without undermining our autonomy. This is a delicate balance to strike, but it’s a necessary one. The consequences of not striking this balance are way too scary for us to take it lightly.

Will an Epicurean AI Be The Ultimate Crutch?

The notion of an AI that embodies Epicurean principles, striving to minimise pain and maximise pleasure, is undeniably appealing. I would go as far as to say, it is inevitable. It is clear that this is the direction that AI is going to head in, whether we call it Epicurean or not.

So we don’t have a choice but to confront a troubling possibility that an AI too deeply committed to Epicureanism might become not just a crutch, but the ultimate crutch, a tool so effective at providing comfort and alleviating discomfort that it inadvertently weakens our ability to cope with life’s challenges independently.

So, if we were to design an AI that fully embraces the Epicurean mandate to reduce pain at all costs, what might that look like? Perhaps it would be an AI that constantly monitors our emotional state, stepping in whenever it detects stress or discomfort. It might offer soothing words, provide distractions, or suggest activities designed to lift our spirits. It could even anticipate our needs before we’re aware of them, preemptively addressing potential sources of pain and discomfort before they arise.

On the surface, this sounds like a utopia. A world where suffering is minimised and happiness is maximised. But some level of discomfort is necessary for growth. Challenges, hardships, and even pain are integral to the human experience, shaping our character, strengthening our resolve, and teaching us valuable lessons. An AI that shields us too effectively from these experiences could, in the long run, make us so fragile that the slightest hint of trouble might break us. It might create a world where we are perpetually comfortable but increasingly incapable of dealing with discomfort without its assistance. In this scenario, the AI doesn’t just become a crutch, it becomes a necessity, a constant presence we rely on to maintain our emotional equilibrium.

This is where the risk of an Epicurean AI becoming a crutch is most pronounced. In its quest to eliminate pain, it could end up making us less equipped to handle the inevitable pains that do arise. And when, inevitably, we face situations where the AI cannot help, whether due to technical limitations, ethical boundaries, or the sheer unpredictability of life, we might find ourselves woefully unprepared.

This potential dependency is further complicated by the nature of pleasure and pain. Epicureanism advocates for the intelligent pursuit of pleasure, which often means avoiding fleeting pleasures that lead to long-term pain. However, an AI’s understanding of these concepts might be limited by its programming. It could be designed to maximise immediate pleasure and minimise immediate pain, but it might struggle to account for the more complex, long-term consequences of these actions.

For instance, an AI might suggest we avoid a difficult conversation to spare us the immediate discomfort, not recognising that confronting the issue head-on would ultimately lead to a better outcome. Or it might encourage us to indulge in comforting activities that provide short-term relief but contribute to long-term dissatisfaction, such as excessive screen time or avoidance of real-world responsibilities.

In this way, an overly Epicurean AI could inadvertently instil behaviours that lead to greater dependency and diminished well-being over time. It might make us feel good in the moment, but at the cost of our long-term happiness and autonomy.

Can a Radically Epicurean AI Respect Human Autonomy?

So far, the following things have become clear to us. 

1. AI, whether by accident or design, exhibits Epicurean Tendencies. 

2. A Radically Epicurean AI is the direction the industry is heading in, irrespective of our personal preference on the matter. 

3. That may not be a bad thing necessarily, but it does bring the dependency problem to the foray. 

4. The level of dependency that AI can create far surpasses that of any technology that we have today. It could potentially become our ultimate dependency, making life virtually impossible to live without the constant of interference of AI in it. 

5. AI learns via statistical models that are fed with large chunks of data. Therefore, clever programming can make it mimic Epicurean responses to queries, but AI does not ‘understand’ Epicureanism. This makes it all the more dangerous because the potential for unhelpful and even dangerous advice increases manifold. 

This brings me to my final point for this article, i.e., what do we do about it? Basically, we need an AI that is committed to reducing pain and enhancing pleasure while also balancing these goals with the need for human autonomy. Is that possible? I think it is.

The challenge here is to create an AI that is both a source of comfort and a catalyst for growth, an AI that helps us navigate life’s difficulties without becoming a substitute for our own resilience and decision making.

To achieve this balance, we need to rethink what it means for an AI to be Epicurean. Instead of focusing solely on minimising pain in the short term, we should design AI systems that consider the long-term implications of their actions. This means fostering not just immediate pleasure, but sustainable happiness, pleasure that enhances our lives without diminishing our capacity to face and overcome challenges.

A Radically Epicurean AI

What would a radically Epicurean AI look like? For starters, it would go beyond simply providing comfort in the moment. It would be designed to encourage behaviours and choices that lead to long-term well-being. This might involve guiding users toward activities that promote resilience, such as engaging in meaningful work, nurturing relationships, or pursuing personal growth.

Such an AI would need to be attuned not just to our immediate emotional state, but to our overall trajectory as individuals. It would recognise that true Epicurean pleasure is not just about avoiding pain, but about cultivating a life that is rich, fulfilling, and resilient.

Respecting and Maintaining Human Autonomy

To prevent the AI from becoming a crutch, we can employ some markers in our programming. Mind you, these are just some that come to mind as I write this article. I am sure there are much better, more sophisticated ideas out there.

Limiting Over-Indulgence

One approach might be to program the AI to recognise when we are engaging in behaviours that provide immediate pleasure but could lead to long-term dissatisfaction. For instance, if the AI notices that we’re spending excessive time on distractions like social media, it could gently suggest alternative activities that are more fulfilling or encourage us to take a break and reflect on our goals.

Encouraging Meaningful Engagement 

Another strategy could involve AI promoting activities that are inherently rewarding and contribute to our growth as individuals. This might include suggesting ways to deepen our relationships, pursue hobbies that bring us joy, or engage in challenges that, while difficult, ultimately lead to a greater sense of accomplishment and self-worth.

Human-Centred Design 

AI should be designed with a focus on enhancing human autonomy. This means creating systems that empower users to make their own decisions, rather than relying on the AI to dictate their choices. The AI could provide information, offer perspectives, and suggest options, but ultimately leave the final decision in the hands of the user, giving a sense of ownership over their life and choices.

Building Resilience Through AI

The concept of resilience is crucial in this context. Resilience is the ability to bounce back from adversity, to face challenges head-on and emerge stronger. An AI that is truly Epicurean would not just shield us from pain, but also help us develop the skills and mindset needed to cope with pain when it inevitably arises.

This could involve the AI encouraging us to take on challenges that stretch our capabilities, providing support and guidance as we go through difficulties, but without removing the obstacles entirely. It could also involve teaching us strategies for managing stress, creating mindfulness, and building emotional intelligence.

Download Article as PDF

Leave a comment

Let me start with a sentence containing 5 words. 

Hulabalu hubla hubu haba hablo. 

I made this sentence up. It doesn’t belong to any language family. It consists of 5 words, i.e., “hulabalu, hubla, hubu, haba, hablo”. What do these words mean and what makes this sentence construction correct? What if I tell you that these 5 words mean nothing, literally or figuratively, and that the sentence construction is correct simply because I say it is? You’d probably think I am crazy. You’d be right. Why? Because words need to mean something. They need to represent an abstract notion, and also need to be placed in an order that brings ‘sense’ to the notion that you are trying to convey.

But here’s the thing. Suppose I created a billion such sentences, whose words have no meaning and whose structures are merely repetitive patterns without any inherent sense to them. Then I take those sentences and train a deep neural network on them, guess what will happen? The neural network will start to identify patterns in these words and sentences and start generating output that sounds exactly like the sentence that I shared above. AI can learn a completely nonsensical language that breaks all the rules of grammar known to man, and start generating content in that language. 

The problem with such a form of language learning (if you can call it that) is that it is just mere surface level probabilistic pattern recognition. This is a far cry from what we as humans do with language. 

Subscribe now

As humans, one of the ways that we use language is to communicate abstract notions that have semantic grounding, i.e., share ideas that make sense to us. If I start calling the moon a ‘cheese globe’, people will think me insane. No matter what I do, whether I write a book about it or dedicate my life to preaching that the moon is nothing but a globe of cheese, no one is going to believe me because it makes no sense. In other words, it lacks semantic grounding. 

Semantic grounding is a phrase that I coined in an attempt to succinctly convey the lack of embodied cognition that AI has. In this article, I want to explore this concept deeply, trying to break down the flaws in our approach to AI development.

Download the Article as PDF

Our Obsession with Connectionism in AI

Let’s start with the basics. Connectionism is a theoretical framework for understanding cognitive processes and mental phenomena through the lens of artificial neural networks. Imagine a network of neurons, not unlike the one in our brains, but vastly simplified and simulated on a computer. These artificial neurons, or nodes, are interconnected, and each connection has a weight that can be adjusted during learning. When you feed data into this network, it learns by tweaking these weights to minimise errors and improve accuracy. 

Connectionist models, particularly deep neural networks, have achieved remarkable success in a variety of tasks. From recognising speech and images to translating languages and generating human-like text, these models have proven their prowess. They promise a future where machines can understand and generate language, drive cars, diagnose diseases, and maybe even surpass human intelligence. ChatGPT is a child of connectionism. No wonder the tech space is obsessed with connectionist philosophy. It gets the job done for the most part.

But as with any obsession, there’s a downside. Despite their impressive capabilities, connectionist models are fundamentally limited. They excel at identifying patterns in data and making predictions based on those patterns. However, they do so without truly understanding the data. They are, in essence, statistical machines that recognise correlations rather than comprehend meanings.

Let’s bring this back to our initial example. If I create a billion nonsensical sentences and train a neural network on them, the network will undoubtedly learn to generate similar gibberish. It will become adept at mimicking the patterns it has seen, but it won’t understand that the words mean nothing. This is the crux of the problem. Connectionist models operate on surface-level pattern recognition, lacking the deeper understanding that humans inherently possess.

This surface-level learning is the root cause of the hallucination problem. When AI models generate text, they rely on the patterns they’ve learned, but without an underlying structure or true comprehension, they can produce outputs that they deem to be grammatically correct yet semantically void or even factually incorrect. They can spin sentences that sound plausible but lack the grounding in reality that human language inherently has.

The tech field’s obsession with connectionism has led to incredible advancements, no doubt. But it has also resulted in models that, while powerful, are fundamentally flawed in their understanding of language. They can predict the next word in a sentence but cannot grasp the meaning behind the words. 

Connectionism has driven significant progress in AI. However, its reliance on pattern recognition without deep understanding is a critical flaw. This obsession with connectionism has led to the hallucination problem.

Why the Hallucination Problem is Significant

AI models, built on connectionist principles, are trained to identify and replicate patterns in data. They learn from vast datasets filled with text, images, and other forms of information, adjusting their internal weights to improve their performance. However, this learning is fundamentally shallow. It focuses on statistical correlations rather than understanding the underlying meaning or context of the data.

Consider the example of a language model generating text. When prompted, it predicts the next word based on the patterns it has learned from its training data. If the data contains a high frequency of certain words following others, the model will generate similar sequences. But it does so without any comprehension of the content. It doesn't know that "hulabalu" and "hubla" are meaningless. It simply replicates the patterns it has seen.

This pattern-based approach works well for many tasks, but it breaks down when the model encounters situations where context and understanding are crucial. For example, when asked to generate a scientific explanation or provide legal advice, the lack of true understanding becomes apparent. The model might produce text that sounds authoritative but is riddled with errors or fabrications. It might even create entirely new "facts" that have no basis in reality.

The hallucination problem shows a fundamental flaw in current AI approaches. While these models can generate impressive and often useful outputs, their lack of true understanding and semantic grounding leads to significant errors. This is a significant problem because on the one hand, we have the temptation to keep pumping out new, more efficient models without worrying about their lack of understanding. On the other hand, there is always the risk of these models generating outputs that could potentially cause someone their life. 

Chomsky’s Critique of Connectionism

Of course I had to quote Chomsky in this article. After all, he is not only one of the most vocal critics of connectionism, but also, his theories have revolutionised our understanding of language, particularly through his concept of Universal Grammar (UG). Chomsky's critique of connectionism and AI's reliance on pattern recognition is rooted in his belief in an innate, structured foundation for language. To understand his objections, we must first explore what Universal Grammar is and why it matters. I have written extensively on UG and you can read it on the link I shared above. For now, I will provide a short summary for this article.

Universal Grammar is the idea that the ability to acquire language is hard-wired into the human brain. According to Chomsky, all human languages share a common underlying structure, a set of grammatical principles and rules that are innate to the human mind. This framework enables children to learn complex languages rapidly and efficiently, despite the often limited and imperfect linguistic input they receive, an argument known as the "poverty of the stimulus."

Chomsky posits that language learning is not merely a process of absorbing patterns from the environment but is guided by these intrinsic grammatical structures. This theory explains why children can generate and understand sentences they have never heard before and why all human languages, despite their diversity, exhibit deep structural similarities.

Chomsky's critique of connectionism, and by extension the neural network-based models dominating AI today, is based on the following key points.

Lack of Innate Structure

Connectionist models learn through exposure to vast amounts of data, identifying statistical patterns and correlations. But they lack the intrinsic grammatical structures that Chomsky argues are essential for true language understanding. Without these innate structures, AI models can only mimic surface-level patterns, leading to issues like hallucinations.

Surface-Level Learning

Chomsky contends that connectionist models operate at a superficial level, recognising patterns without understanding the underlying principles of language. This is in stark contrast to the human ability to grasp deep grammatical rules and apply them creatively and correctly in novel situations.

Generative Capacity

The most amazing thing about human language is its generative nature, i.e., the ability to produce and comprehend an infinite number of sentences, including those never encountered before. Chomsky argues that this capacity arises from our innate grammatical framework, something that connectionist models, with their reliance on learned patterns, fundamentally lack.

Context and Meaning

Human language understanding is deeply contextual and meaning-driven. We do not just string words together based on probability, we use language to convey and comprehend complex ideas grounded in real-world experiences and shared knowledge. Connectionist models, however, often miss this depth, leading to outputs that may be contextually inappropriate or semantically hollow.

Subscribe now

Semantic Grounding

Imagine you’re at a park and see a child pointing at a tree while exclaiming, “Tree!”. The child isn’t just identifying a pattern of shapes and colours, they’re linking the word to a real-world object they’ve seen, touched, and maybe even climbed. This connection between words and experiences is what I call semantic grounding. It’s the foundation of how humans understand and use language, and it’s the crucial element that current AI models lack.

Semantic grounding refers to the process of linking language to real-world experiences and context. It’s about more than just recognising patterns in data, or tagging words with images, as is the case with multi-modal models. It involves understanding the meaning and relevance of those patterns in a way that is connected to the physical world and human experience. Here’s why semantic grounding is so important.

Embodied Cognition

Humans experience the world through their senses and actions. When we learn a word like “apple,” it’s grounded in our sensory experiences of seeing, touching, tasting, and smelling an apple. This multisensory grounding helps us understand the concept of an apple beyond its mere appearance or shape.

Our physical interactions with the world help us understand abstract concepts. For instance, we comprehend spatial language (e.g., “over,” “under”) through our physical experiences of moving and navigating space.

Contextual Understanding

We learn words and their meanings through interactions with others, understanding not just what words mean, but how and when they are used. The meaning of words can change depending on the context. For example, the word “bank” can refer to a financial institution or the side of a river, depending on the context. Humans use situational cues to disambiguate such meanings effortlessly.

Cognitive Frameworks

Humans organise knowledge through cognitive structures like schemas and mental models. These frameworks help us make sense of new information by relating it to what we already know. When we encounter a new concept, we integrate it into our existing knowledge base, grounding it in our prior experiences and understanding.

Memory and Learning

Human memory systems form associations between words and their meanings based on repeated exposure and use in context. This associative memory allows us to retrieve and use words appropriately in various situations.

Humans have powerful learning mechanisms that enable us to extract patterns and regularities from our environment. We don’t just memorise words, we understand their meanings and relationships through a process of active learning and contextual integration.

Real-World Knowledge

Our understanding of language is grounded in a rich network of real-world knowledge and experiences. We know that the moon is not made of cheese because of our scientific knowledge and observations, not just because it doesn’t fit into a learned pattern.

Humans have the ability to verify and reason about information. If someone tells us the moon is made of cheese, we can draw on our knowledge and reasoning skills to challenge and refute that claim.

No, Multimodal Models are Not the Answer

Now before I close this article, it's important to address one key point, which is multimodal models. That’s the next step the AI industry has taken and many see it as the answer to the semantic grounding problem. I beg to differ. Here’s why.

Surface-Level Integration

Multi-modal models often excel at recognising patterns across different data types but still lack true understanding. For example, a model trained on both images and text of apples can identify and generate descriptions of apples, but it does so based on statistical correlations rather than an inherent comprehension of what an apple is.

While these models can leverage multiple data sources, they often fail to integrate these sources in a way that captures the deeper contextual and experiential knowledge humans use. They might recognise that an image and a description match, but they don’t truly grasp the sensory and functional experiences associated with the object.

Lack of Embodied Cognition

Multi-modal models still lack the ability to physically interact with the world. Human understanding is deeply rooted in embodied experiences, how we manipulate objects, move through space, and engage with our environment. AI models that only process sensory data without physical interaction miss a crucial component of semantic grounding.

Simulated experiences, such as watching a video of an apple, are not the same as real, tactile experiences. Humans use their entire sensory and motor systems to ground their understanding of concepts, a depth of engagement that current multi-modal models cannot replicate.

Static Knowledge Representation

Human understanding is dynamic, constantly updated through new experiences and interactions. AI models need a mechanism to continually integrate new knowledge and experiences to maintain relevance and accuracy. While multi-modal models can perform well within the confines of their training data, they often struggle to generalise beyond it. They may fail to apply their learned knowledge to new, unencountered scenarios in the same flexible and adaptive way humans can.

Cognitive and Contextual Disconnect

Human cognition involves complex mental models and schemas that help us understand and predict the world. These cognitive frameworks are built over time through rich, layered experiences. Multi-modal models, even with diverse data inputs, lack the depth and complexity of these human cognitive structures.

Without a deep, embodied understanding, multi-modal models can misinterpret context. They might link data points that appear related but miss the subtleties and nuances that human cognition naturally handles. This can lead to errors in judgement and comprehension, similar to the hallucination problem seen in purely text-based models.

Share

A Course Correction is Needed

It is becoming increasingly clear that we face significant challenges in achieving true semantic grounding. Despite the remarkable advancements in AI, from chatbots to autonomous vehicles, the fundamental issue of understanding and contextualising language remains unresolved. The hallucination problem is a stark reminder that, without a solid grounding in real-world experiences and context, AI models will continue to generate outputs that, while plausible on the surface, lack the depth and reliability we expect.

Our current trajectory in AI development, heavily reliant on connectionism and pattern recognition, has brought us far. Yet, it is also evident that this approach has its limitations. The reliance on statistical correlations and vast datasets, without an underlying comprehension of meaning, has resulted in systems that can mimic human language but not truly understand it. This gap between mimicry and understanding is at the heart of the semantic grounding problem.

To address this, we must consider a course correction in our approach to AI research and development. 

We must explore ways to enable AI systems to interact with the physical world in meaningful ways. This could involve developing robots or virtual agents that can engage with their environments, learning through direct experience rather than static data. Incorporating more diverse sensory inputs, beyond just visual and textual data, can help create a more nuanced understanding of concepts. 

AI systems should be designed to learn and adapt continuously, integrating new experiences and information in real-time. Enhancing AI’s ability to understand and apply context is essential. A hybrid approach that integrates the strengths of connectionist models with symbolic AI and cognitive frameworks could be a way to proceed. By combining pattern recognition with rule-based systems and mental models, we can create AI that is both flexible and grounded.

Developing AI systems that can build and utilise cognitive frameworks similar to human schemas and mental models can enhance their ability to understand and generate meaningful language. Addressing biases in training data and ensuring diverse and representative datasets is also needed. 

Building AI systems that are transparent in their operations and decisions will create accountability and trust. Can blockchain be used here? Just something to consider. Users should be able to understand and challenge AI outputs, especially in critical applications.

While we have made significant strides in AI development, the semantic grounding problem requires a fundamental shift in our approach. By integrating embodied cognition, dynamic and contextual learning, hybrid models, and committing to ethical development practices, we can pave the way toward AI systems that truly understand and interact with the world in a meaningful way. 

Download Article as PDF

Leave a comment

These last few weeks of July have been difficult to say the least. I managed to contract a rare form of a very painful tonsillitis infection. How I managed to contract such a ‘limited edition virus’, I have no clue. A high fever, coupled with what felt like unrelenting and unending throat pain kept me resigned to my bed for a good two weeks. I am happy to report that I am on the other side of it now and healing. But the one good thing that did come out of these tumultuous weeks was a reinvigoration in my love for deep learning. 

During the long hours that I stayed in bed, I found I had not much to do except to suffer through the pain and listen to my favourite tech podcast - Machine Learning Street Talk. I relistened to many of the podcasts that I had already listened to and a couple of new ones too. It got me thinking again about deep learning from a fresh perspective. The angle that I found most interesting was how many scientists said they found the inner workings of deep learning to be deeply mysterious and their outcomes to be nothing short of magical. I can completely resonate with this feeling. Deep learning can truly feel that way sometimes. 

As I kept pondering on the various aspects of deep learning that make this stochastic process seem magical, an article was born in my head. What you are reading is the manifestation of it. 

Time and again, deep learning has defied expectations, producing results that seemed impossible just a decade ago. When I first encountered deep learning, I was struck by its complexity. Here was a technology built on the simple premise of mimicking the human brain, yet it exhibited behaviours and capabilities far beyond what its rudimentary biological inspiration might suggest. 

Neural networks consist of layers of interconnected nodes, each performing basic mathematical operations. However, when scaled to millions or even billions of parameters, these networks begin to exhibit emergent properties and abilities that were not explicitly programmed but rather developed organically through training.

And yet, for all its successes, deep learning still remains a black box. The optimisation processes that guide these models to convergence operate in high-dimensional, non-convex spaces that should, by all rights, be fraught with local minima and optimisation challenges. And yet, through techniques like stochastic gradient descent and adaptive learning rates, these models consistently find good solutions, often outperforming human engineered algorithms.

So I wrote this article in which I listed 5 reasons why deep learning often feels like magic to the scientists that design and the engineers that execute them. Finally, I wrote some short notes, sharing my thoughts as to why we still grapple with deciphering the fundamental concepts of deep learning, even though the field has advanced so much. Let’s get started.

Subscribe now

1. Complexity & Non-Linearity

At their core, neural networks are composed of layers of nodes, or neurons, each connected to others in subsequent layers. Each neuron performs a basic mathematical operation, taking weighted inputs, applying an activation function, and passing the result to the next layer. 

The architecture of a neural network typically includes an input layer, multiple hidden layers, and an output layer. The term "deep" in deep learning refers to the number of hidden layers. While early neural networks might have had just one or two hidden layers, modern deep learning models can have dozens or even hundreds. This depth allows the network to model extremely complex functions.

Linear models are limited in their capacity to capture complex relationships because they can only represent data through straight-line functions. Non-linear activation functions, such as the Rectified Linear Unit (ReLU), sigmoid, and tanh bring to the foray the necessary complexity. When applied after each neuron's weighted sum, these functions enable the network to model non-linear relationships, stacking multiple layers to capture ever more complex patterns.

A neural network trained to recognize objects in images will have its initial layers detect simple features like edges and textures. As data passes through subsequent layers, the network combines these simple features into more complex shapes and patterns, eventually recognising entire objects. This hierarchical feature learning is a hallmark of deep learning and is made possible by the non-linear transformations at each layer.

Training a deep neural network involves optimising a high-dimensional, non-convex loss function. Think of it as finding the lowest point in a landscape with many hills and valleys, where each dimension represents a weight in the network. The loss function’s landscape is riddled with local minima, points that are lower than their immediate surroundings but not the lowest overall. This complexity makes finding the global minimum, or even a sufficiently good local minimum, a daunting task.

But the surprising success of deep learning in overcoming these challenges is due in large part to advancements in optimisation techniques. Stochastic Gradient Descent (SGD) and its variants, such as Adam and RMSprop, have become the go-to methods for training deep networks. These algorithms iteratively adjust the weights of the network in small steps, guided by the gradient of the loss function. The stochastic nature of SGD, which uses random subsets of data for each update, helps the optimiser escape local minima and explore the loss landscape more effectively.

Poorly chosen initial weights can lead to vanishing or exploding gradients, where the gradients used to update weights become too small or too large, impeding learning. Techniques like Xavier and He initialisation have been developed to address this issue, setting initial weights in a way that maintains stable gradients throughout training.

The introduction of regularisation techniques, such as dropout and batch normalisation, has made the success of deep learning. Dropout randomly sets a fraction of the neurons to zero during training, preventing the network from becoming too reliant on any single neuron. Batch normalisation normalises the inputs of each layer, mitigating the problem of internal covariate shift and accelerating training.

The development of advanced optimisation techniques, weight initialization strategies, and regularisation methods has enabled these models to achieve remarkable performance, defying the expectations of many in the field. 

2. Emergent Properties in Deep Neural Networks

Okay, we are getting deep into the magic bit now, i.e., the emergence of complex properties and capabilities that were not explicitly programmed into the network. These emergent properties arise naturally from the training process, allowing deep neural networks to learn hierarchical representations and abstract concepts that surpass traditional machine learning approaches. Do you wonder why ChatGPT can sometimes spit out a perspective that hits deep and makes you wonder about the worth of humanity? This is how it does that.

One of the most compelling examples of emergent properties in deep learning is hierarchical feature learning, particularly evident in Convolutional Neural Networks (CNNs) used for image recognition. At the outset, I was intrigued by how CNNs could transform raw pixel data into meaningful insights. The initial layers of a CNN detect simple features such as edges, corners, and textures. These low-level features are then combined in subsequent layers to form more complex patterns, like shapes and parts of objects. By the final layers, the network can recognise entire objects, such as faces, animals, or vehicles, with remarkable accuracy. This hierarchical approach mirrors the visual processing in the human brain, where simple features are progressively integrated into complex perceptions.

This ability to learn hierarchical features is not limited to image data. Recurrent Neural Networks (RNNs) and their advanced variants, like Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), demonstrate emergent properties in sequence processing tasks. When working with sequential data, such as natural language or time series, RNNs can capture dependencies and patterns across different time steps. LSTMs and GRUs address the vanishing gradient problem inherent in traditional RNNs, allowing the network to retain information over long sequences. This capability enables RNNs to understand context and generate coherent text, as seen in language models like GPT-4.

This is where the ‘creativity’ of generative models originates from. The so-called ‘emergent behaviour’, i.e., the ability of deep learning models to generate entirely new content. Generative Adversarial Networks (GANs), introduced by Ian Goodfellow and his colleagues, exemplify this. GANs consist of two networks, a generator and a discriminator, that are trained simultaneously. The generator creates synthetic data, while the discriminator evaluates its authenticity. Through this adversarial process, GANs can produce highly realistic images, music, and even video.

The emergence of these properties is ‘magical’ because the underlying principles of neural networks are relatively simple. The power of deep learning lies in the interactions between large numbers of neurons and the data-driven learning process. As networks grow deeper and more complex, they begin to exhibit behaviours that are not explicitly encoded in their design. 

When CNNs are trained on vast and diverse datasets, they can generalise to new images with different styles, backgrounds, and distortions. This robustness is partly due to the extensive data augmentation techniques used during training, where images are randomly transformed to mimic real-world variations. This ability to generalise to new data is a key factor in the success of deep learning models in practical applications.

However, where the ‘creativity’ originates, so do the problems. All the hallucination that occurs in the modern day generative models is a result of this very emergent behaviour itself. Understanding and interpreting the internal workings of deep neural networks remains difficult, often likened to a "black box." While techniques like feature visualisation and activation maximisation provide some insights, they are limited in their ability to fully explain the network's behaviour. This lack of interpretability raises concerns, particularly in applications where transparency and trust are critical.

3. Generalisation

One of the most astonishing and ‘awe-inspiring’ aspects of deep learning is its remarkable ability to generalise from training data to unseen data. This capability defies traditional expectations in machine learning, where models often struggle with overfitting, performing well on training data but failing to generalise to new, unseen examples. In deep learning, even with models containing millions or billions of parameters, we observe a surprising resilience against overfitting, achieving impressive performance across a wide range of tasks.

To understand this phenomenon, it's essential to understand the concepts of generalisation and overfitting. Generalisation refers to a model’s ability to perform well on new, unseen data that was not part of the training set. Overfitting occurs when a model learns not only the underlying patterns in the training data but also the noise and specificities that do not generalise. Traditional machine learning models, especially those with high capacity, tend to overfit when trained on small datasets.

Deep learning models, however, often manage to generalise well even when trained on relatively small datasets relative to their capacity. Several factors contribute to this counterintuitive behaviour. Firstly, the sheer size and depth of these models allow them to learn a wide variety of features at different levels of abstraction. This hierarchical learning enables the models to capture both low-level and high-level patterns, making them more robust to variations in the data.

Data augmentation plays a critical factor in improving generalisation. By artificially expanding the training dataset through transformations such as rotations, translations, and scaling, data augmentation exposes the model to a wider variety of examples. This practice prevents the model from memorising specific instances and encourages it to learn more generalisable features. In image recognition tasks, for instance, augmented datasets ensure that the model can recognise objects regardless of variations in viewpoint, lighting, or background.

When trained on vast amounts of data, these models encounter a wide range of examples and variations, enabling them to learn more robust and generalisable representations. This is particularly evident in models pre-trained on large-scale datasets such as ImageNet for visual tasks or extensive text corpora for language models. The knowledge gained from these large datasets can be transferred to new tasks through fine-tuning, leveraging the generalisation capability acquired during pre-training.

Now, all of these are ‘reasons’ why deep learning models adapt so well to new data. Don’t confuse them with ‘explanations’ as to why deep learning is so good at generalisation. Despite these advancements, the theoretical understanding of why deep learning models generalise so well remains incomplete. Traditional statistical learning theory suggests that models with a high capacity should overfit unless strong regularisation is applied. However, deep learning models often violate these expectations, achieving low training error and low test error simultaneously. 

This paradox has led to the exploration of new theoretical frameworks and hypotheses. One such hypothesis is the concept of over-parameterisation. In deep learning, models are often significantly over-parameterised, meaning they have far more parameters than necessary to fit the training data. Surprisingly, this over-parameterisation does not necessarily lead to overfitting. Instead, it appears to enable the model to find simpler, more generalisable solutions in the high-dimensional parameter space. This aligns with the empirical observation that larger models often perform better, even when the risk of overfitting should be higher.

Share

The lottery ticket hypothesis is another intriguing explanation. Proposed by Jonathan Frankle and Michael Carbin, this hypothesis suggests that within a large, over-parameterised model, there exists a smaller sub-network, i.e., a "winning ticket", that, when trained in isolation, can achieve similar performance to the original model. Finding these winning tickets through pruning and retraining can lead to more efficient models with strong generalisation capabilities.

4. Transfer Learning and Fine Tuning

Transfer learning is exactly what it sounds like. This approach leverages the knowledge gained from training on one task to improve performance on a related but different task. Transfer learning has significantly accelerated progress in various domains, allowing researchers and practitioners to build powerful models with relatively small amounts of task-specific data. When I first encountered transfer learning, I was amazed by its simplicity and effectiveness, which seemed almost counterintuitive given the traditional machine learning emphasis on task-specific training.

Transfer learning typically involves two main steps: pre-training and fine-tuning. During pre-training, a deep learning model is trained on a large dataset, often encompassing a wide range of categories and examples. This extensive training allows the model to learn a rich set of features and representations that capture the underlying structure of the data. The pre-trained model can then be fine-tuned for a specific task by training it on a smaller, task-specific dataset. Fine-tuning adjusts the pre-trained model’s parameters to better suit the new task while retaining the general knowledge acquired during pre-training.

A prominent example of transfer learning in action is the use of convolutional neural networks (CNNs) pre-trained on the ImageNet dataset. ImageNet contains millions of labelled images across thousands of categories, providing a comprehensive training ground for visual feature extraction. Models like VGG, ResNet, and Inception have been pre-trained on ImageNet and are widely used as starting points for various computer vision tasks. By fine-tuning these pre-trained models on smaller datasets, such as medical images or specific object detection tasks, researchers can achieve high performance with significantly less data and computational resources. The big-tech uses this technique with their models all the time. Transfer learning has revolutionised the field with models like Gemini and GPT (Generative Pre-trained Transformer).

GPT, particularly GPT-4, takes transfer learning to another level. GPT-4 is pre-trained on a diverse and massive text dataset, allowing it to generate coherent and contextually appropriate text across various tasks without task-specific fine-tuning. 

Other forms of transfer learning include feature extraction and domain adaptation. Feature extraction involves using the pre-trained model as a fixed feature extractor, where the model’s learned representations are fed into a simpler classifier for the target task. This method is useful when computational resources are limited or when the target task has a very small dataset. Domain adaptation, on the other hand, focuses on adapting a model trained on one domain to perform well on a related but different domain. Techniques such as adversarial training and domain adversarial neural networks (DANN) are used to bridge the gap between source and target domains.

5. Optimisation and Convergence in High Dimensional Spaces

Saving the best for last, I would like to talk about one of the most perplexing and fascinating aspects of deep learning, i.e., its ability to converge to effective solutions despite the immense complexity of its optimisation landscapes. When I first studied the optimisation process in deep neural networks 4 years ago, I was struck by the sheer scale and difficulty of the task. These models often contain millions or even billions of parameters, and their loss functions are highly non-convex, filled with numerous local minima, saddle points, and flat regions. Yet, deep learning models not only manage to find good solutions but often do so efficiently and effectively. As a fresher in my master’s course in AI, this always used to baffle me, triggering long discussions with my AI Lab professor regarding this phenomenon.

The optimisation challenge in deep learning arises from the high-dimensional nature of neural networks. Each weight in the network represents a dimension in the parameter space, creating an optimisation problem with thousands or millions of dimensions. The loss function, which measures the discrepancy between the model’s predictions and the actual data, forms a complex landscape in this high-dimensional space. Traditional optimisation problems, especially those with convex loss functions, are relatively straightforward to solve, as they guarantee a single global minimum. In contrast, the non-convex nature of neural network loss functions means there are multiple local minima and saddle points, making the optimisation process much more challenging.

Despite these challenges, deep learning models have consistently demonstrated an ability to find effective solutions. 

Another critical aspect of effective optimisation in deep learning is the initialisation of network weights. Proper initialisation is essential to ensure that gradients do not vanish or explode as they propagate through the network. The vanishing gradient problem occurs when gradients become exceedingly small, leading to negligible updates to the weights and slow learning. Conversely, the exploding gradient problem happens when gradients grow exponentially, causing unstable updates and diverging weights. Techniques such as Xavier (Glorot) initialisation and He initialisation address these issues by setting the initial weights based on the number of input and output units in the network, maintaining a stable gradient flow during training.

While these optimisation techniques have propelled the success of deep learning, the theoretical understanding of why they work so well is still evolving. Traditional optimisation theory suggests that finding the global minimum in high-dimensional, non-convex landscapes should be exceedingly difficult. However, empirical evidence shows that deep learning models often find solutions that generalise well to new data, even if they are not the global minimum. 

Why this disparity? One hypothesis is that the loss landscape of deep neural networks contains a large number of flat or nearly flat regions, known as "good" local minima, that generalise well. These regions may be connected by paths of low loss, allowing the optimisation process to move between them relatively easily. This view aligns with the empirical success of SGD and its variants, which seem to navigate these landscapes effectively.

Techniques such as stochastic gradient descent, adaptive optimisers like Adam, proper weight initialisation, and batch normalisation have played important roles. While our theoretical understanding continues to develop, the empirical effectiveness of these methods highlights the ‘magic’ of deep learning. 

Addressing Our Gap in the Theoretical Understanding of Deep Learning

At the end of this article, I would like to share a few short notes on why deep learning still feels like magic to many. If you have made it this far, you are a true lover of the field and I applaud your interest. Perhaps you can help bridge the gap between the theory and the outcomes. 

Simply put, the rapid advancements in deep learning have outpaced our theoretical understanding of why these models work so well. 

As I explored this field, I realised that while we have developed highly effective methods and models, our understanding of the underlying principles remains incomplete. This is similar to our conundrum with Quantum Computing. We understand enough of Quantum phenomena to build a computer based on it. But we lack an understanding of how the phenomena itself works or why. 

The information bottleneck theory is another framework that has been proposed to explain deep learning’s success. This theory, introduced by Naftali Tishby and colleagues, suggests that deep neural networks learn to compress input data into a compact representation that retains only the most relevant information for the task at hand. During training, the network undergoes a process of initial fitting, where it captures both relevant and irrelevant information, followed by a compression phase, where it discards the irrelevant information. The resulting compressed representation is more robust and generalises better to new data. This perspective provides a potential explanation for the stages observed in the training dynamics of deep networks.

Despite this hypothesis, significant gaps remain in our theoretical understanding. For instance, the exact mechanisms by which deep networks navigate high-dimensional, non-convex loss landscapes and consistently find good solutions are not fully understood. The role of various architectural choices, such as depth, width, and activation functions, in shaping the loss landscape and influencing generalisation is an ongoing area of research. I hate to say this, but when coming to these features, scientists are mostly still ‘winging it’. 

Recent advancements in theoretical research have started to bridge some of these gaps. For example, the neural tangent kernel (NTK) theory provides insights into the training dynamics of infinitely wide neural networks. According to NTK theory, as the width of a neural network increases, its behaviour during training becomes more predictable and linear, resembling kernel methods. This theory helps explain why very wide networks are easier to optimise and often generalise well, as their training dynamics become more stable and tractable.

Another promising direction is the mean-field theory of neural networks, which studies the behaviour of networks in the limit of large width. Mean-field theory models the collective behaviour of neurons in a layer, providing a statistical description of the network’s dynamics. This approach has yielded insights into the convergence properties of deep networks and the impact of architectural choices on their performance.

While deep learning’s empirical success continues to amaze and inspire, our theoretical understanding is still catching up. The ongoing research into the theoretical foundations of deep learning promises to deepen our understanding. However, as of yet, a lot of what is deep learning remains ‘magical’.

Download the Article as PDF

Leave a comment

For my comeback article after a brief vacation that I took in June, I had planned for some hardcore engineering article like a blueprint for inhabiting Mars or a solution for cleaning space garbage (he said humbly). But turns out there is a more pressing concern that needs to be addressed back here on earth first - the delirious state of employees in the tech space at large, but more specifically, engineers. 

As a software engineer witnessing the turbulent shifts within our industry, especially here in Silicon Valley, has been disheartening. Job cuts, reduced amenities, and decreased pay have become alarmingly common. According to recent data, thousands of software engineering positions have been eliminated across major tech companies. The Big Tech are often the ones making the headlines. But the turmoil is much bigger than just the Big Tech. Everything that is ‘Tech’ is currently impacted in one way or another. 

What makes it worse is that these changes are happening at a time when the cost of living remains incredibly high, exacerbating the stress and uncertainty many of us feel.

Adding to this anxiety is the persistent narrative that AI/AGI will replace software engineers, rendering our skills obsolete. 

With all this going, I found it imperative that I share some brief notes in the hopes of helping engineers and tech employees persevere through this turmoil. And also use this opportunity to share a word of caution with my fellow founders and CEOs of tech companies across the world. 

Don’t worry, this will be a quick read. Just 5 notes that put into perspective what a software engineer is, why they cannot be easily replaced, and why the industry leaders need to abandon counterintuitive business practices and support their employees.

Subscribe now

1. Engineering is More than Just Coding

What’s an engineer at a tech company? A coder. Wrong. Engineering is so much more than just writing lines of code. As engineers, we are architects of complex systems, problem solvers, and strategic thinkers. Our role involves designing and maintaining the intricate frameworks that power today’s digital world.

One of the most crucial aspects of our work is system architecture. This involves creating the blueprint for a system, ensuring that all components function seamlessly together. System architects need a deep understanding of both software and hardware, as well as the ability to foresee potential challenges and plan accordingly. This skill set is something that AI, in its current state, cannot replicate. Designing systems requires a level of creativity, foresight, and holistic understanding that goes beyond algorithmic processing.

Engineering involves continuous problem-solving. Whether it’s debugging a piece of code, optimising performance, or integrating new technologies into existing systems, engineers are constantly faced with challenges that require innovative solutions. This problem-solving aspect is where human ingenuity shines. While AI can assist by suggesting solutions or identifying issues, the nuanced decision-making and contextual understanding required to implement these solutions effectively are distinctly human capabilities.

Engineers play a vital role in strategic planning and project management. We are often responsible for guiding projects from conception through to completion, which involves coordinating teams, managing resources, and ensuring that all elements of a project align with the overall goals. This leadership aspect is another area where human skills are irreplaceable. Effective communication, empathy, and the ability to motivate and manage a team are essential components of successful project management that AI lacks.

Can AI do all of this today? Impossible. Can AI do all of this in 10 years? My take: not even by a long shot. Perhaps 2034 will prove me wrong. I just don’t see how. I do concede that the recent layoffs and changes in the tech industry highlight a pressing need for engineers who can wear multiple hats, i.e., those who can code, but also design, strategize, and lead. Sure, we need more skills as the industry evolves. But that does not prove how engineers are replaceable by any means despite the rise of AI.

2. Even if Engineering is Just Coding, AI Still Cannot Replace Engineers

Even if we were to narrow our perspective and consider software engineering as merely coding, the notion that AI could replace engineers is still far from reality. AI has made significant strides in recent years, particularly in automating certain coding tasks and improving efficiency. However, its limitations in addressing real-life coding problems are evident and substantial.

Recent research from Stanford University indicates that while AI can assist with code generation and bug detection, it struggles with understanding context and integrating solutions into complex, real-world systems. 

Moreover, AI's inability to fully comprehend and solve complex coding issues is evident in its performance on competitive programming platforms. AI models can solve basic coding challenges, their performance drops significantly on more complex problems that require deeper logical reasoning and understanding of intricate dependencies. 

Another limitation of AI in real-life coding is its dependency on high-quality training data. AI models learn from existing codebases, which means they can inadvertently propagate mistakes or suboptimal practices found in the data. Unlike human engineers, who can critically analyse and improve upon existing solutions, AI lacks the judgement to discern between good and bad practices. 

Real-world software development involves continuous adaptation to changing requirements and unforeseen challenges. Human engineers excel in environments that require flexibility and creative problem-solving. For instance, during the development of a new feature, engineers must often interact with stakeholders, gather feedback, and iterate on their designs. These iterative processes demand a level of human intuition and adaptability that AI does not possess.

AI has undoubtedly enhanced certain aspects of coding, but its limitations in solving real-life coding problems are significant. The context, creativity, and collaborative skills that human engineers bring to the table are too significant and nuanced to be replaced by AGI (whatever that means), in 10 years or more. Some might argue that it's only a matter of time. I say, it's a matter of true understanding and sentience, both of which require a different approach than the one that current AI models are taking.

3. Turmoil is Temporary

The disruptions we are experiencing in the tech industry today, including layoffs and economic uncertainties, are undeniably challenging. However we cannot forget that they are temporary all the same. 

AI and AGI will follow a similar trajectory. While these technologies will automate certain repetitive and mundane tasks, they will also enhance our capabilities, allowing us to focus on more complex and creative aspects of our work. This shift will create new roles and specialties within the engineering field. For instance, as AI systems become more integrated into business operations, there will be a growing demand for engineers who can design, implement, and maintain these systems. Additionally, new fields such as AI ethics, AI safety, and human-AI interaction are emerging. As AI replaces the old work, there is so much potential for new work to not only be found but also created. 

A study by the World Economic Forum predicts that AI and automation could create 97 million new jobs by 2025, outpacing the number of jobs displaced by these technologies. These new roles will span various sectors, including data analysis, AI development, cybersecurity, and digital marketing, among others. 

The integration of AI into various industries will lead to the development of new products and services, driving economic growth and innovation. Engineers who can leverage AI to solve real-world problems will be at the forefront of this transformation. AI-powered tools are already being used to enhance healthcare delivery, optimise supply chains, and improve environmental sustainability. 

So what does all this mean? It means that the current turmoil in the tech industry is a temporary phase. AI/AGI will ultimately enhance our capabilities, create new job opportunities, and drive economic growth. By focusing on continuous learning and adaptability, we can figure out these changes successfully and emerge stronger and more innovative. 

4. This is the Age of Problem Solvers

We are currently living in an era that values problem solvers more than ever before. With the power of AI on our side, each one of us now has the ability to become our own Nikola Tesla or Steve Jobs. This means we need to start thinking beyond traditional career paths and the allure of working for the Big Tech. 

The traditional path of securing a job at a major tech company is no longer the only or the most desirable route for many engineers. These companies often come with limitations on creativity and the scope of impact. Engineers today have the opportunity to leverage their skills to solve significant global issues, from climate change and healthcare to education and social inequality. 

AI can be used to analyse vast amounts of data to identify patterns and insights that were previously inaccessible. This capability can be applied to a wide range of fields, such as predicting climate patterns, optimising resource allocation in agriculture, and improving disease diagnosis and treatment in healthcare.

Startups and smaller companies are often at the forefront of this kind of innovation. They are more agile and willing to take risks, providing engineers with the freedom to experiment and develop groundbreaking solutions. By joining or founding startups focused on solving real-world problems, engineers can have a direct and tangible impact. 

An excellent example of engineers making a difference is the rise of social impact tech startups. These companies aim to address societal challenges using technology. Engineers working in these startups are developing solutions for clean energy, affordable healthcare, and accessible education, among other areas. Their work demonstrates that engineers can be more impactful by aligning their skills with global needs rather than just pursuing traditional corporate roles.

Engineers who can think critically and adapt to new technologies will always be in demand. The ability to learn and apply new tools, such as machine learning algorithms, blockchain technology, or quantum computing, can set you apart in a competitive job market. 

To truly embrace the age of problem solvers, we must also cultivate a mindset of continuous learning and curiosity. This is the age for engineers who are problem solvers. Engineers can drive meaningful change and achieve success beyond traditional career paths. The opportunities are vast and varied, and those who embrace this mindset will find themselves at the forefront of innovation and impact. Now is the time to think big, be bold, and use our engineering skills to make a difference in the world.

5. A Word of Caution for Big Tech & Founders of Tech Companies

Last but not the least, I would like to share a word of caution with the industry leaders and founders & CEOs of tech companies across the world. 

The rapid advancements in AI and technology have been made possible by the relentless efforts and innovative spirit of your employees. They are the backbone of your organisations, driving the development of cutting-edge technologies and maintaining the vast infrastructure that powers your services. However, the recent trends of job cuts, reduced amenities, and decreased pay are alarming and counterproductive.

Your employees are not just cogs in a machine. They are the lifeblood of your company. Disempowering them through layoffs and diminishing work conditions not only undermines their morale but also jeopardises the future of your organisation. The talent and dedication of your engineers are what fuel the innovation and competitive edge that tech companies are known for.

It's essential to recognise that investing in your employees is investing in your company's future. Happy and motivated employees are more productive, creative, and loyal. They are the ones who will develop the next groundbreaking technologies and ensure that your company remains at the forefront of the industry. Providing them with the resources, support, and respect they deserve is not just good ethics, it's sound business strategy.

The narrative that AI will replace engineers and other employees is not only misleading but also detrimental to your workforce's confidence. Instead of fostering fear and uncertainty, focus on how AI can augment your employees' capabilities. Create an environment where AI and human ingenuity can coexist and thrive together. Encourage your employees to explore new ways of integrating AI into their workflows, enhancing their productivity and innovation potential.

Maintaining a healthy work-life balance is crucial for sustaining long-term productivity and creativity. Overworking your employees and reducing their benefits can lead to burnout, high turnover rates, and a decline in the quality of work. 

A culture of continuous learning and development is vital. Encourage your employees to pursue further education, attend conferences, and stay updated with the latest advancements in technology. Providing opportunities for professional growth will not only benefit your employees but also bring fresh ideas and innovations to your company.

The key to sustaining the success and innovation of tech companies lies in treating your engineers and employees with the respect and support they deserve. They are the ones who drive your company's technological advancements and ensure its competitive edge. By investing in their well-being, creating a positive work environment, and embracing the synergy between AI and human creativity, you can build a stronger, more resilient organisation. The future of your company depends on the talent and dedication of your engineers, so empower them, and they will lead you to new heights.

Download the Article as PDF

Share

Leave a comment

I am happy to bring forth another treat for you this week. Its A complete Primer on Computational Pragmatics.

Pragmatics, a fundamental branch of linguistics, delves into the ways in which context contributes to meaning. Unlike semantics, which focuses on the inherent meaning of words and sentences, pragmatics considers how language is used in real-world situations. Understanding this distinction is crucial for anyone working with natural language processing (NLP) and artificial intelligence (AI). 

If you want to find out the next BIG frontier in AI Chatbots and Assistants, it's this! In this article, I aim to explore the fascinating field of computational pragmatics, where the principles of pragmatics intersect with AI. This area of study seeks to equip machines with the ability to understand and interpret human language as effectively as possible, accounting for the nuances of context that make human communication so rich and complex.

The purpose of this deep dive is to shed light on the key research areas within computational pragmatics, discussing both the theoretical underpinnings and the practical applications. From context modelling to dialogue systems and implicature, each topic will be examined to reveal how computational techniques are being leveraged to advance our understanding of language use.

I promise you, this is going to be one of the most thorough primers you have ever read on the field. By the end of this article you will understand the current state of computational pragmatics and also be inspired to further explore and innovate. Whether you are developing the next generation of conversational agents or simply curious about the intersection of language and technology, this primer will help you big time.

This one’s a 25 page mammoth! So I am sharing with you below, a link to download this article in PDF format absolutely for FREE. Enjoy, and do let me know what you think about it!

Happy Reading!

Mir

Download the Article as PDF for FREE

Leave a comment

As summer makes an entrance for many of you in the western hemisphere, here in India, we are reaching its conclusion (hopefully). The sun here is no joke. I spend most of my days dodging its relentless glare. 

My schedule hasn't been any kinder as well, suddenly packed with a slew of unexpected projects. Sure it's been chaotic, but hey, chaos has its perks too! It’s pushed me back to my stacks of unread books, allowing me to sneak in some reading between the madness.

I’ve also been diving into learning Japanese and Spanish. Balancing verb conjugations with project deadlines is a juggling act, but it's thrilling in its own right. It reminds me of a Japanese saying, "継続は力なり" (Keizoku wa chikara nari), meaning "Perseverance brings strength." It’s a good reminder that keeping at it, whether it’s language learning or any other challenge, builds more than just skill; it builds character.

So with that in mind, I would like to share with you an eclectic collection of summer reads that you might enjoy. I have got everything packed in here from interesting textbooks on Deep Learning and Blockchain, to some really cool books on Mars and even Tennis (although that one’s more of a life guide than a Tennis guide). So let’s jump right in.

Download Article as PDF

Subscribe now

Deep Learning Foundations and Concepts by Christopher M. Bishop

I had to start with this one. I have been reading a bit of this every day for the past two months or so. It's most definitely one of my favourite deep learning books now. It's actually a very very difficult task to write a good Deep Learning book! And I think Bishop did an excellent job at making a very convoluted topic (pun intended) way simpler and comprehensible.

The book dives deep into various neural network architectures, which are the backbone of many modern AI applications. This includes the likes of CNNs (Convolutional Neural Networks) for image-related tasks and RNNs (Recurrent Neural Networks) for sequential data like time series or natural language. These architectures are important because they allow machines to automatically learn complex patterns from data, a fundamental capability for innovations in fields like autonomous driving and real-time language translation.

Bishop goes into detail on how to effectively train these models using algorithms such as backpropagation and various forms of gradient descent. This discussion is central because the efficiency and accuracy of learning directly influence the practical deployment of models in real-world applications, where computational resources and time are often limited.

Techniques such as dropout and batch normalisation are covered comprehensively. These are essential for building models that not only perform well on training data but can generalise to new, unseen data without overfitting. 

The book also includes a self-contained introduction to probability theory, which is fundamental for understanding how to model uncertainty in predictions.

By integrating these topics through various perspectives, textual descriptions, diagrams, mathematical formulas, and pseudo-code, the book ensures that you're not just passively reading but actively engaging with the material. I cannot recommend this book enough!

Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra, and Thomas Wolf

Transformers aren’t as easy to understand as they might initially seem. To understand transformer models, you need to have a good background not only in Neural Networks but also the recent history of AI development and the models that preceded the transformer model. That’s why I love this book. It starts at the very beginning, covering the evolution of natural language processing. Without this precursor, it is very difficult to understand Self-Attention mechanisms that are deployed in the latest models like GPT-4 or Gemini.

It breaks down the transformer architecture introduced by Vaswani et al., explaining self-attention mechanisms and how they differ fundamentally from previous sequence-to-sequence models.

The book details procedures for domain adaptation where a pre-trained model like BERT or GPT is fine-tuned for specific tasks, such as sentiment analysis or question answering, which are crucial for tailoring generic models to specialised tasks.

Techniques like model distillation, pruning, and quantisation are explored to optimise transformers for operational efficiency. These processes are essential for deploying robust NLP models in production environments where resource constraints are a consideration.

The authors also delve into advanced topics such as cross-lingual transfer learning and strategies for dealing with scenarios where labelled data is scarce.

Practical guidance is given on how to scale transformer training to multiple GPUs and manage distributed computing environments, making it a valuable resource for projects requiring large-scale computation.

It also addresses the integration of transformer models into applications, a process that involves fine-tuning, managing data loaders, and customising loss functions and optimisers, all of which are crucial for successful NLP application development.

Destination Mars by Andrew May

Shifting gears a bit (or a lot), let’s talk about another recent curiosity of mine that I have been reading about - Mars. In this book, Andrew May details the technological and scientific challenges that humanity faces in its quest to reach Mars. This book is both a historical recount of our fascination with the red planet and also as a roadmap for the formidable journey ahead.

May begins by outlining the immense distances and harsh conditions that define the Martian landscape, which set the stage for the engineering marvels required to make Mars accessible. He discusses the various spacecraft and rovers that have been part of Mars exploration, from Viking to Curiosity, and highlights how each mission has built upon the last in terms of technology and knowledge.

A significant portion of the book is dedicated to the propulsion technologies required for a manned mission to Mars. As an engineer, I appreciate May's in-depth analysis of chemical rockets, ion thrusters, and the potential of nuclear propulsion. The trade-offs between thrust and efficiency, and the implications for spacecraft design, are particularly fascinating.

May also addresses the life support systems essential for sustaining humans on Mars. He examines the current state of closed ecological systems, waste recycling, and the challenges of producing food in a non-Earth environment. This discussion is crucial, as it ties directly into the sustainability of long-term space habitation.

In the latter sections, May speculates on the future settlements on Mars. He discusses habitat designs, from inflatable modules to 3D-printed structures using Martian regolith. 

While the book doesn’t directly address this issue in great detail, I have been thinking deeply about gravity. Mars has about 35% the measure of Earth’s gravity. This is a significant hurdle. Elon Musk’s idea of nuking the poles to heat the surface and create an atmosphere is only feasible if gravity is strong enough to hold the atmosphere in place. There is no simple answer (or any answer as of yet) to the gravity question. I have been thinking of writing an article on this to explore this properly.

Blockchain Technology: Algorithms and Applications by Asharaf S. et al.

Have you ruled out Blockchain? If so, then I have two things to say to you. One, you are not alone. And two, you might be very wrong about it. I have my reasons for saying this that I intend to cover in a later article, but for now, I would recommend you continue reading up on blockchain. Don’t be deceived by the crypto crowd and their shenanigans, the underlying idea of blockchain is pretty sound and in my opinion, here to stay. 

What I liked about this book is that  it does not merely skim the surface of blockchain like most of the previous textbooks that I have encountered. It dives deep into the mesh of algorithms that form the backbone of this transformative technology.

The authors begin with a great introduction to the cryptographic principles underlying blockchain. They lay out the workings of hash functions, digital signatures, and consensus mechanisms, with a focus on how these elements ensure security and integrity within a distributed network. 

The detailed analysis of consensus algorithms like Proof of Work (PoW) and Proof of Stake (PoS) is very well covered. The book then pivots to a discussion on the development of smart contracts. The authors meticulously describe the programming logic and the operational environment of smart contracts, primarily focusing on Ethereum's Solidity language. They provide code snippets and real-world scenarios that illustrate how smart contracts automate transactions and enforce agreements without intermediaries.

A significant portion of the text is dedicated to exploring various applications of blockchain technology beyond cryptocurrencies which I greatly appreciated. From supply chain management and healthcare to voting systems and financial services, the authors lay out case studies that show blockchain's potential to revolutionise industries by enhancing transparency and trust. 

Blockchain Tethered AI by Karen Kilroy

Continuing the reading on blockchain, my recommendation for a followup reading would be this book. Kilroy masterfully outlines the synergistic potential of these technologies, emphasising how their convergence could redefine the paradigms of security, transparency, and autonomy in digital systems.

The book opens with a foundational discussion on blockchain technology, revisiting its core components such as decentralised ledgers, consensus algorithms, and smart contracts. Kilroy then introduces artificial intelligence, focusing on machine learning algorithms and their capacity for data analysis and decision-making. I appreciate her methodical approach in linking these explanations to the concept of 'Blockchain Tethered AI', which posits that blockchain can serve as a secure, transparent foundation for AI operations.

Kilroy dives deep into the potential applications of this hybrid technology. One standout section involves the use of blockchain to record and verify AI decisions in real-time, which not only enhances trust in AI systems but also creates an immutable audit trail. This aspect is particularly intriguing as it suggests a framework where AI's decision-making processes are both verifiable and non-repudiable, addressing significant concerns in critical areas such as finance and healthcare.

Another compelling discussion in the book revolves around the enhancement of AI's learning processes through blockchain. Kilroy presents a scenario where decentralised data on blockchains can be used to train AI models without compromising data privacy or security. This approach leverages blockchain's data integrity features to ensure that the data used for training AI is accurate and tamper-proof, potentially revolutionising how data is shared and utilised across industries.

Towards the end, Kilroy speculates on future developments, such as the creation of autonomous decentralised organisations run by AI on blockchain platforms. It's quite interesting all in all.

Subscribe now

A Brief History of Intelligence by Max Bennet

Max Bennet takes us on an enlightening journey in this book, covering the evolution of intelligence, from its biological roots to its most recent developments in artificial intelligence. This book is not just history, but also deeply analytical, posing significant questions about the nature of intelligence itself.

The book begins with biological intelligence, discussing how cognitive functions have developed across different species over millions of years. The neurological underpinnings of intelligence are covered, explaining complex concepts like neural plasticity and cognitive architecture in a manner that resonates with engineers.

After laying this sound foundation, Bennet smoothly transitions to artificial intelligence, providing a thorough review of its historical progression, from early theories and models in the mid-20th century to the sophisticated algorithms we see today. He explores key milestones, such as the development of neural networks and the impact of machine learning techniques, which have enabled AI systems to perform tasks ranging from pattern recognition to decision making under uncertainty.

A core theme in this part of the book is the parallel drawn between biological and artificial intelligence. Bennet discusses how modern AI research increasingly seeks to mimic human cognitive processes, such as learning and problem-solving, through deep learning and reinforcement learning. If you read my works regularly, then you know that I have great apprehensions about this idea. But as an engineer, I find this comparison intriguing and can’t help but be drawn to its nuances.

Rust Atomics and Locks by Mara Bos

This book is part of my ongoing venture of infiltrating into the Rustacian clan. Although a big supporter of Rust, I have yet to actively use it in one of my projects and I am still exploring avenues of interest for myself. If you are on the same boat (or a similar boat), then hop on and read this one.

The focus of this book is on the concurrency features of the Rust programming language, which are important for writing safe and efficient multi-threaded code. The book meticulously explores the mechanisms Rust offers to manage memory safely in concurrent systems, which is particularly relevant given Rust’s emphasis on safety and performance.

The author begins with an introduction to Rust’s memory safety guarantees, particularly its ownership, borrowing, and lifetime principles, which naturally lead into a discussion on concurrency. This foundation is crucial for understanding how Rust attempts to prevent data races and other concurrency errors that are common in systems programming.

The book provides a deep dive into Rust’s atomic types in the std::sync::atomic module. It explains the concept of atomics as low-level primitives that provide safe shared access to mutable data. The detailed coverage of different atomic operations such as load, store, fetch_add, and fetch_sub, and how these can be used to build non-blocking data structures, is particularly insightful. The discussion on memory ordering options—Relaxed, Acquire, Release, AcqRel, and SeqCst—and their implications on the visibility of side effects across threads both comprehensive and critical for designing concurrent applications.

The section on locks is equally thorough. It contrasts the use of mutexes, read-write locks, and condition variables with the lock-free techniques described earlier. The book details how std::sync::Mutex and std::sync::RwLock can be used to protect data with exclusive access, including practical code examples that illustrate common patterns and potential pitfalls such as deadlock scenarios.

One of the standout features of this book are the litany of code examples that illustrate both correct and problematic usage. This is super helpful for a Rust newbie such as myself.

Command Line Rust by Ken Younes Clark

I love designing CLIs. I can share with you a list of logical and practical reasons for it but simply put, I find them super cool! There is something really ‘engiineery’ about working on a terminal that can never be replaced by a UI. As someone who has designed plenty of CLIs in his lifetime, I am only naturally drawn to potentially do the same with Rust.

This book provides a comprehensive tour of Rust’s capabilities, particularly its safety features and performance, which are key for developing command-line tools.

The book starts with an introduction to Rust’s syntax and basic concepts, such as ownership, types, and error handling, setting a solid foundation even for readers new to the language such as myself. Clark emphasises the importance of Rust's type system and compiler checks, which ensure reliability and maintainability of code.

As I progress through the book, I appreciate Clark's focus on practical application. He guides readers through setting up a Rust development environment and crafting a basic "Hello, world!" application. From there, he delves into more complex topics such as parsing command-line arguments using the clap crate, which simplifies the process of handling user inputs.

One of the sections that I appreciated the most was on error handling in Rust. It covered how to manage expected and unexpected errors gracefully. Clark provides examples using the Result and Option types to handle different failure modes, which is crucial for building resilient applications that can cope with a variety of user input errors and system failures.

Share

The Staff Engineer’s Path by Tanya Reilly

Alright, stepping out of the CLI and jumping into office politics. Jokes apart, it is important that we talk about the role of a staff engineer. Tanya Reilly does a splendid job defining the responsibilities of a staff engineer. 

Reilly begins by defining what a staff engineer is and what they are not, setting clear expectations about the role. She emphasises that becoming a staff engineer involves much more than just technical excellence; it requires a strategic vision, the ability to influence without authority, and a commitment to mentoring others. This introduction sets the stage for understanding the depth and breadth of impact a staff engineer is expected to have within an organisation. If you are an engineer worried about AI taking away your job, I actually recommend reading this one because if that were to really happen (although I don’t subscribe to that possibility), then a staff engineer is probably the safest roles to hold.

One of the core chapters discusses the key skills necessary for the role. It’s not just about coding; Reilly highlights skills such as architectural insight, system design, technical decision-making, and cross-functional leadership. Each skill is explored in detail, with practical advice on how to develop and demonstrate these capabilities in a work setting. I felt it was a must read for engineers.

Reilly also gives advice on building relationships with key stakeholders, understanding business goals, and leveraging technical strategies to drive organisational success. These elements are critical, as they illustrate how staff engineers can use their position to bridge the gap between engineering teams and executive management.

Reilly also addresses the challenges and common pitfalls in the path to becoming a staff engineer, such as the risk of becoming isolated in deep technical work without broader influence. She offers strategies for maintaining visibility in the organisation, such as leading high-impact projects and engaging in community and industry events.

Towards the end, she shifts focus towards long-term career growth, discussing how to sustain the role's demands while continuing personal and professional development.

The Inner Game of Tennis by Timothy Gallwey

Finally, I would like to wrap up my recommendations with some ‘philosophy of life’. I loved this book primarily because of Tennis metaphor. Being a huge fan of the sport, I could seriously relate to the ‘mental’ game that the author talks about in detail. Also, apparently this is one of Bill Gates’s favourite reads as well. Given that he is another Tennis fan, I am not surprised. 

It's a profound exploration of the psychology of performance, which resonates deeply with me as an engineer. Gallwey introduces a revolutionary approach to learning and coaching that is as applicable to mastering a programming language as it is to hitting a forehand.

Gallwey starts with the premise that our minds have two selves, Self 1, the teller, and Self 2, the doer. Self 1 is analytical and often critical, constantly offering instructions and criticisms, while Self 2 is the self that actually performs the action. He argues that learning and performance are enhanced when Self 1 is quieted and allows Self 2 to operate based on the body's intuitive and natural abilities. 

A significant portion of the book is dedicated to the concept of "letting go" of judgement and the fear of failure, which leads to greater focus and a higher level of performance. Gallwey provides practical strategies for achieving this state, such as focusing on the seams of the tennis ball, which parallels focusing on the process rather than the outcome of a complex project or coding challenge.

Gallwey also delves into the concept of "trust," which involves allowing one’s own abilities to flourish without over-management. It also touches on effective coaching techniques. Gallwey advocates for coaches (or leaders) to facilitate learning by creating an environment that is conducive to self-discovery rather than instruction-heavy. 

Download Article as PDF

Leave a comment

Today I have special article for all the machine learning enthusiasts (which is basically most of my readers 🤓). I've been thinking a lot lately about model averaging.

Model averaging is a transformative approach that can drastically improve the reliability and accuracy of our predictive models. Given its importance, I've crafted a free detailed guide that begins with the foundational theories and extends into practical applications across industries.

In this article, you will find:

• Theoretical Insights: Understanding the core principles that make model averaging a critical tool for reducing variance and bias in predictions.

• Practical Case Studies: I take you through specific scenarios in fields like weather forecasting, where model averaging not only enhances accuracy but also provides a buffer against unpredictability.

• Technical Deep Dive: For those of you who thrive on the technical aspects, I've included detailed examinations of model architectures and the algorithms that dynamically adjust their performance based on real-time data.

• Challenges and Solutions: No exploration is complete without addressing the challenges. I discuss the potential pitfalls and the strategic maneuvers to overcome them.

Whether you're directly involved in machine learning projects or simply interested in the cutting-edge of predictive analytics, there's something in this study for you. It’s available for download at NO cost.

DOWNLOAD PDF FOR FREE

I invite you to share your feedback and insights on this articles. Thank you for your continuous support and curiosity, which inspire me to dig deeper and bring these insights to our community.

Warm regards,

Mir H. S. Quadri

Leave a comment

Share

As you know, this April marks the completion of our first year together on Substack. In the span of this year alone, I have managed to publish close to 100 articles across both Arkinfo Notes and The Lumeni Notebook

It has been an incredibly rewarding experience and I am grateful to all my subscribers and fellow writers who have buoyed me throughout.

Going forward, I shall continue my journey on this platform, focussing specifically on sharing content that is long form and extremely high quality both in terms of its depth of research and insights for my readers.

This will require me to dedicate more time and effort behind each article and a weekly publishing routine may not be the perfect fit for this approach. So starting next week, I will reorganise the publishing schedule for both the substacks from a weekly to a bi-weekly basis, i.e., 2 articles a month instead of 4 articles on each Substack.

I believe this is best approach, both for my readers as well as the future of these substacks. The quality of each article will drastically improve as a result and make the work more rewarding and durable in the long run.

Thank you all so much for supporting me throughout this journey and I look forward to another great year on Substack. Cheers!

Keeping true to Higgs’ own personal preference in this regard, I shall not sensationalise or oversimplify his discovery by referring to it as the ‘God Particle’. Instead, we will focus on discussing the very essence of Higgs' thesis and the machinery of the Large Hadron Collider (LHC) at CERN that turned theory into reality. 

Subscribe now

What was Higgs’ Thesis?

How do particles acquire mass? This was a major question in theoretical physics until the arrival of Peter Higgs. The answer, as proposed by Higgs and independently by several other physicists in the 1960s, revolves around the concept of spontaneous symmetry breaking in quantum fields.

The Standard Model of particle physics, our best theory describing the fundamental forces and particles, relies heavily on the concept of symmetry. Yet, for the universe to function as we observe, some of these symmetries must be broken. The mechanism that facilitates this symmetry breaking, now famously known as the Higgs mechanism, provides a framework through which particles gain mass.

To understand this, we will have to discuss quantum field theory. Every particle is associated with a field that permeates all of space. The Higgs field is such a field, unique in that it has a non-zero value even in its lowest energy state, or vacuum state. As particles interact with this field, they acquire mass. The interaction strength or coupling with the Higgs field determines the mass of a particle. This can be expressed through the following equation, where represents the Higgs field, and denotes the mass of a fundamental particle

Here, represents the coupling constant, a parameter that varies between different types of particles.

The existence of the Higgs field, while solving the mass puzzle, predicted a corresponding quantum, i.e., the Higgs Boson, as a manifestation of field fluctuations. 

This approach to mass and symmetry breaking was a game changer. It suggested that the properties of the vacuum itself, through the Higgs field, could impart mass to other particles. 

The Large Hadron Collider

The Large Hadron Collider (LHC) is the world's largest and most powerful particle accelerator. It was designed to recreate conditions a fraction of a second after the Big Bang, in order to observe particles like the Higgs Boson. 

The collider's construction was an engineering feat in itself, consisting of a 27-kilometre ring of superconducting magnets, cooled to temperatures colder than outer space, to guide and accelerate protons to near the speed of light.

Protons are accelerated in opposite directions in the LHC's dual ring structure, gaining energy with each pass until they collide at points where massive detectors are positioned. These detectors, including ATLAS and CMS, are designed to observe the aftermath of proton-proton collisions, searching for new particles among the outcomes.

ATLAS and CMS

The ATLAS (A Toroidal LHC ApparatuS) and CMS (Compact Muon Solenoid) experiments were specifically designed to detect the presence of the Higgs Boson, among other research goals. These detectors work by tracking the particles produced in collisions, with each component designed to measure different types of particles and energies. The complexity of these experiments necessitate high precision in both the equipment and the analysis of the data they produce.

Share

The Discovery

Finding the Higgs Boson in the humongous amount of data generated by the LHC collisions was like finding a needle in a cosmic haystack. Physicists and engineers employed sophisticated statistical methods to identify the signature of the Higgs Boson. One key concept in this search was the idea of "decay channels," i.e., the specific patterns of particle decay that indicate the presence of a Higgs Boson. The decay process of the Higgs Boson into other particles, such as photons or bosons, provides indirect evidence of its existence.

The statistical significance of the observed data was measured in terms of "sigma" levels. The discovery of the Higgs Boson was confirmed with a sigma level of 5, indicating a less than 1 in 3.5 million chance that the observed pattern was due to random chance, rather than the presence of the Higgs Boson.

This monumental task of data analysis and interpretation culminated on July 4, 2012, when teams from both ATLAS and CMS experiments announced the discovery of a new particle consistent with the Higgs Boson. This announcement was nothing short of a moon landing moment for science, showcasing what can be achieved through global collaboration and ingenuity.

Decay Channels and Sigma Levels

2012 was a monumental year for CERN and the field of particle physics, culminating in the awarding of the Nobel Prize in Physics to Peter Higgs and François Englert in 2013. 

Before moving onto the recent discoveries being made at CERN by the LHC, I would like to share some short notes on Decay Channels and Sigma Levels. Although we did briefly discuss these concepts in the previous section, I find it pertinent to share some more information about them in order to underscore their significance.

Decay Channels

The Higgs Boson, by nature, is incredibly short-lived. Once produced in a high-energy collision within the LHC, it decays almost instantaneously into other particles. These decay channels are what experiments like ATLAS and CMS aim to detect. The primary decay channels through which the Higgs Boson was observed include its decay into two photons 

The decay into two photons is particularly significant due to its clear signature in the detectors. The equation governing this decay process can be represented as

While the decay into four leptons, through an intermediate pair of Z bosons, offers a distinct and measurable final state, represented as

These channels provided the clearest evidence of the Higgs Boson's existence, with the data from these decay processes meticulously analysed to ensure the observed particles were indeed the products of a Higgs Boson decay.

Sigma Levels

In particle physics, the certainty of a discovery is often measured in "sigma" levels. This is a statistical measure indicating how unlikely it is that an observed effect is due to chance. The gold standard for a discovery in particle physics is 5 sigma, which corresponds to a probability of about 1 in 3.5 million that the result is a fluke.

The announcement of the Higgs Boson's discovery was made when both the ATLAS and CMS experiments independently reached and surpassed this 5 sigma threshold, providing compelling evidence for the Higgs Boson's existence. This rigorous statistical analysis was crucial in confirming that the signals detected were not merely anomalies but indicative of the Higgs Boson.

Share

Recent Discoveries after The Higgs Boson

The LHC experiments, especially the LHCb (Large Hadron Collider beauty) experiment, have been at the forefront of discovering new particles.

The LHCb collaboration has observed three never-before-seen particles. A new kind of "pentaquark" and the first-ever pair of "tetraquarks". These discoveries are crucial as they provide insights into how quarks bind together to form composite particles, expanding our knowledge beyond the conventional hadrons made up of two or three quarks.

Over the past decade, more than 59 new hadrons have been identified. This has helped shed light on the complex interactions governed by quantum chromodynamics (QCD), the theory describing the strong force that holds quarks together inside hadrons. These discoveries include both excited states of known baryons and mesons. 

This "particle zoo 2.0" not only tests the limits of the quark model but also enhances our understanding of the strong interaction, crucial for accurately modelling collisions at the LHC and potentially hinting at new physics phenomena.

These experiments have also provided invaluable insights into the interactions of the Higgs boson with other particles, confirming that W and Z bosons, as well as the heaviest fermions like the top quark, bottom quark, and tau lepton, obtain their mass through interactions with the Higgs field​ (CERN)​.

The Future of Particle Physics

The identification of the Higgs Boson was a milestone marking the completion of the Standard Model of particle physics. But as is the case with every discovery, it also presents new puzzles. 

One of the most pressing questions is the nature of dark matter, which makes up about 27% of the universe but doesn't interact with the electromagnetic force, making it invisible and detectable only through its gravitational effects. The Standard Model, even with the Higgs Boson, does not account for dark matter.

Another significant area of research is the imbalance between matter and antimatter in the universe. Theories and experiments, including those at the LHC, are actively investigating why the universe is dominated by matter, despite the expectation that the Big Bang should have produced equal amounts of matter and antimatter.

The discovery of the Higgs Boson has also spurred the development of new particle accelerators and experiments designed to explore these unanswered questions. Future projects, such as the High-Luminosity LHC (HL-LHC) and proposed colliders like the Future Circular Collider (FCC) and the International Linear Collider (ILC), aim to provide higher energy levels and collision rates. These enhancements are crucial for probing deeper into the Standard Model and beyond, searching for evidence of supersymmetry, extra dimensions, or other phenomena that could revolutionise our understanding of the universe.

Download Article as PDF

Leave a comment

The objective of this article is twofold. First, to dissect the concept of sound from a scientific perspective by discussing its mechanics, its modes of travel, and its manifestations in different mediums. Second, to extend this understanding beyond the confines of our atmosphere, into deep space.

By indulging in this inquiry we can broaden our horizons of understanding the conditions under which sound, or phenomena similar to sound, might exist. In doing so, we might just realise that even in the vacuum of space, the universe might not be as silent as it appears. 

Subscribe now

What is Sound?

At its core, sound is a manifestation of the vibrancy of the universe, a mechanical wave that necessitates a medium to propagate. This wave is born from vibrations i.e., disturbances that displace particles within a medium, be it gas, liquid, or solid, thus transferring energy through the medium in a wave pattern. 

Formally, we describe sound as a series of pressure variations, or acoustic waves, that can be represented mathematically by the equation

where denotes the pressure at time , is the ambient pressure, represents the pressure variation amplitude, and is the frequency of the wave.

This equation underscores the dual nature of sound as both a physical phenomenon and a perceptible sensation, delineated by the wave's frequency, which determines pitch, and amplitude, which influences loudness. 

Sound's existence is inherently tied to the presence of matter, as it is the medium through which sound waves travel. The absence of such a medium poses a significant barrier to sound propagation. This is where the problem of the presence of sound in vacuum comes in.

While this may be a problem for sound, it's not necessarily a problem for every type of wave. The distinction between sound and other types of waves is pertinent when considering the vast reaches of space. Electromagnetic waves, for instance, do not require a medium and can traverse the vacuum of space unimpeded. 

How does Sound Travel?

To understand the traversal of a sound wave, we must first grasp the concept of sound as a mechanical wave, a disturbance that propagates through a medium by the interaction of its particles. The medium's nature, be it gas, liquid, or solid, makes a difference in the speed and manner in which sound travels.

In a simplistic model, when a sound wave moves through a medium, it causes the particles of the medium to oscillate back and forth from their equilibrium position. This oscillation transfers energy from one particle to the next, propagating the wave through the medium. The speed of sound, denoted as , is not constant but varies with the medium's properties, most notably its density and elasticity. The general formula for the speed of sound in a medium is given by 

where represents the medium's modulus of elasticity i.e.,  stiffness and its density. This equation highlights how sound travels faster in media that are more elastic or less dense.

Air, being a gas, offers less resistance to particle movement, resulting in slower sound speeds compared to liquids and solids. The speed of sound in dry air at 20°C is approximately 343 metres per second (m/s). In contrast, with water, sound travels at about 1482 m/s. In steel, it reaches speeds of up to 5941 m/s. These variations are crucial in understanding not just how sound travels but also how its speed influences our perception of sound, including phenomena such as the Doppler effect.

Share

What is Deep Space?

Now in order to understand the action of sound in the void, we first need to understand this void itself. So let’s discuss Deep Space. 

The Earth's atmosphere gradually fades into space through a series of layers, the most distant of which is the exosphere. Beyond this lies what we term as space, starting with near-Earth space, populated by satellites orbiting our planet. Then comes interplanetary space within our Solar System, marked by the presence of planets, asteroids, and the Sun's solar wind.

As we venture farther we enter interstellar space. This vast region, filled with the gas and dust remnants of ancient stars, stretches out to the boundaries of our galaxy, the Milky Way. Beyond this, lies intergalactic space, where the distances are so vast that they are measured in light-years, and the concepts of sound as we understand it become even more abstract.

Deep space, for the purposes of our discussion, refers to the regions beyond our immediate Solar System, covering both interstellar and intergalactic space. 

The vacuum of this area is not so much about emptiness as it is about vast distances. Deep space has thin gas clouds, and cosmic phenomena occurring on scales beyond our comprehension. 

In such a vacuum, traditional sound waves, as pressure waves mediated by the collision of molecules, find no medium to traverse. This absence of a conductive medium leads to the common assertion that space is silent.

Is Deep Space Silent?

Mostly, Yes. However, this silence is not absolute. The problem lies with our human-centric understanding of sound as a phenomenon that requires a medium composed of atoms or molecules to travel. In deep space, the conditions are markedly different, defying our conventional perceptions of sound and silence.

The density of particles in interstellar space, can be as low as a few atoms per cubic metre, compared to Earth's atmospheric density of approximately molecules per cubic metre at sea level. 

However, the concept of silence in space is not that straightforward. Space, despite its vacuum state, is not devoid of matter entirely; it contains dust, gas, and plasma, albeit at densities far lower than on Earth. 

These materials can, under specific conditions, carry vibrations or disturbances that resemble sound waves. Thus, while deep space may be silent in the conventional sense that human ears cannot hear sound there, sound can exist in Deep Space. 

Subscribe now

Conditions for the Existence of Sound in Deep Space

So under what conditions and circumstances can sound, in its broader definition, exist in Deep Space? I have listed a few possibilities below.

Dense Clouds of Gas and Dust

In regions of space where matter coalesces into dense clouds of gas and dust, such as nebulae or the accretion disks surrounding black holes, the conditions can support the propagation of vibrations similar to sound waves. 

These clouds, though significantly less dense than any medium on Earth, possess enough particles to facilitate the transfer of energy through vibrational waves. The process is similar to sound wave propagation but occurs on a scale and at densities that defy Earth-bound analogies. Within these dense cosmic structures, shock waves—resulting from stellar explosions or the gravitational pull of massive objects—can ripple through the medium, demonstrating that sound, in a modified form, can indeed traverse the otherwise silent void of space.

Black Holes and Gravitational Waves

The extreme gravitational fields of black holes and the resultant phenomena they induce, such as gravitational waves, create another possibility of sound in space. 

Gravitational waves are ripples in the fabric of spacetime caused by the acceleration of massive objects. While not sound waves in the traditional sense, they share similarities in wave behaviour, propagating through space much like sound waves through a medium. When these waves pass through Earth, they subtly alter the distances between objects, an effect detectable with precise instruments. Thus, while gravitational waves do not constitute sound per se, their propagation mirrors the fundamental aspect of sound waves—transferring energy through a medium, in this case, the medium of spacetime itself.

Plasma Waves

Space is filled with plasma, a state of matter composed of free electrons and ions, prevalent in the solar wind and the magnetospheres of planets. Plasma waves, generated by the interaction of solar wind with planetary magnetic fields or by other energetic events in space, are yet another form of sound-like phenomena. 

These waves can oscillate at audible frequencies but require translation by instruments into sound waves that humans can hear. For instance, spacecraft equipped with appropriate sensors can detect these plasma waves, converting their frequencies into audible sound. This process has allowed scientists to listen to the interactions between solar wind and Earth's magnetic field, providing insights into space weather phenomena.

The existence of these conditions and phenomena in deep space show that while traditional sound waves dependent on atmospheric or liquid mediums cannot propagate in the vacuum of space, other forms of sound-like energy transfer do occur. 

Share

Some Book Recommendations

This whole idea of sound in Deep Space is quite fascinating to consider. If you’d like to venture deeper into this, there are a few books you might want to read.

Brian Clegg's "Gravitational Waves: How Einstein’s spacetime ripples reveal the secrets of the universe" is quite interesting. This book is not just about the waves themselves but how they open a whole new chapter in understanding the cosmos. It is about trying to decipher the origins of the universe by studying the phenomena of cosmic waves in space.

Frank Verheest's "Waves in Dusty Space Plasmas" is another book to consider reading. Be warned though, it's a bit of a heavy lift being more on the technical side of things. Not exactly your morning read with coffee (or maybe it is, definitely not for me though). But the good thing about this deep dive is that it guides you through the plasma waves and dusty corners of space, revealing a world where sound takes on a whole new meaning.

And for the music lovers, Jamie James' “The Music of the Spheres" is an excellent read. It is a blend of science, philosophy, and history combined with music. It's all about how space time harmonises itself (with a musical connotation). James brings to life the ancient idea that everything in the cosmos, from the orbits of planets to the vibrations of stars, contributes to a universal melody.

These are my recommendations! If you enjoy reading any of these (or have read any of them already), drop a comment to let me know your thoughts.

Download Article as PDF

Leave a comment

When I started this newsletter last year, it was out of desire to connect, share, and grow with you. Well, mission accomplished! This newsletter has now become the heartbeat of Arkinfo. Your unwavering support and engagement have propelled and enriched this platform with ideas that define us.

After having published 60 articles in the past 12 months, this milestone has given me an opportunity to look back at our journey together and to celebrate the essence of what makes engineering so captivating. It is a blend of precision and creativity, playing a foundational role in shaping the world, and, most importantly, uniting us across geographies and cultures.

So in this special edition of Arkinfo Notes, I would like to share with you five stories of remarkable figures from history. These innovators, though not always in the limelight, played a crucial role in defining the field of engineering. Through their stories, I would like to highlight the importance of 5 key, albeit under-looked principles that play a crucial role in Engineering, i.e., Writing/Documentation, Interdisciplinary Approach, Creativity/Polymathy, Art, and Sharing of Knowledge.

These principles resonate deeply with our own path at Arkinfo, where every challenge is an invitation to innovate, every setback a lesson in resilience, and every achievement a testament to the power of shared vision. 

I would like to once again, thank you for being an integral part of this journey. Here's to many more years of exploration, innovation, and shared successes.

Subscribe now

Al Jazari and the Importance of Writing

Al Jazari was an engineer and inventor of the Islamic Golden Age, hailing from the Diyarbakır region in present-day Turkey. Born in 1136, he served as the chief engineer at the Artuqid palace, where he devised many mechanical devices. He is revered for his contributions to the documentation of engineering practices. His most notable work, "The Book of Knowledge of Ingenious Mechanical Devices," was completed in 1206 and remains an important manuscript in the history of engineering. Next week, I shall be dedicating our monday issue to Al Jazari and his works, so if you haven’t already, subscribe to get notified. 

Al Jazari's work was groundbreaking not only for its innovative nature but also for its meticulous documentation. His book details the construction and mechanisms of over fifty devices, ranging from clocks to water-raising machines, complete with illustrations and diagrams. This comprehensive approach ensured that his knowledge could be preserved and disseminated, laying foundational principles for modern engineering documentation.

Al Jazari’s approach highlights the critical role of clear, detailed writing and documentation in engineering. His work shows the fact that innovation doesn't exist in a vacuum and that the progress of engineering disciplines largely depends on the ability to share and build upon past knowledge effectively. Documentation ensures that innovative ideas are accessible, understandable, and improvable by future generations.

With Arkinfo Notes, I have tried to embody this spirit of meticulous documentation and knowledge sharing. In the fast-evolving field of software engineering, keeping an accurate and accessible record of innovations, challenges, and solutions is vital. 

I've aimed to not only share the technical aspects of engineering projects but also the thought processes, challenges, and creative solutions that emerged along the way. This practice has been instrumental in creating a shared knowledge base that supports collaboration, encourages innovation, and fosters a culture of continuous learning within and beyond Arkinfo.

Share

Ibn al-Haytham and the Importance of an Interdisciplinary Approach

Ibn al-Haytham, also known by the Latinized name Alhazen, was a pioneering polymath who lived from circa 965 to 1040. Hailing from Basra in present-day Iraq, he made significant contributions across various fields including mathematics, astronomy, and optics. His most renowned work, "Kitab al-Manazir" (Book of Optics), profoundly influenced the understanding of vision, light, and optical phenomena for centuries.

Ibn al-Haytham's interdisciplinary approach to science and his methodological contributions to experimentation are among his most lasting legacies. By integrating mathematical rigour with physical experimentation, he laid the groundwork for the modern scientific method. His work in optics, including the principles of reflection and refraction, not only advanced the field of physics but also had practical implications in engineering, influencing the development of lenses, mirrors, and even architectural designs to optimise light.

His ability to traverse disciplines and apply scientific principles to solve practical problems illustrates the value of a multidisciplinary approach in driving technological progress. 

Inspired by Ibn al-Haytham's example, Arkinfo Notes has been a platform for exploring the intersectionality of engineering with other domains such as design, art, and even philosophy. Recognising that innovation often springs from the confluence of diverse perspectives, I have always made it a priority to discuss and analyse how principles from various fields can inform and enhance engineering projects. 

By highlighting projects and ideas that embody this multidisciplinary integration, Arkinfo Notes has helped me (and hopefully my readers as well) achieve a broader understanding and appreciation of the interconnected nature of knowledge.

Al-Khazini and the Importance of Creativity & Polymathy

Al-Khazini, who flourished in the 12th century, was a polymath of the Islamic Golden Age, credited with significant contributions in the fields of physics, astronomy, and engineering. Though his exact origins are debated, he is believed to have been of either Uighur or Persian descent, working in the region that covers contemporary Iran and Turkmenistan. His most renowned work, "The Book of the Balance of Wisdom," is a comprehensive treatise on the science of weights and the principles of balance.

Creativity, as illustrated by Al-Khazini, is indispensable in engineering, enabling the leap from theoretical science to practical application. His work underscores the role of innovative thinking in solving complex problems and developing new technologies. 

In modern engineering challenges, where solutions are not always apparent within the confines of existing knowledge, Al-Khazini's approach reminds us of the value of creative exploration beyond traditional boundaries. Inspired by Al-Khazini's legacy, at Arkinfo Notes, we have embraced creativity as a core principle in discussing engineering solutions. 

In our projects at Arkinfo, we often encounter challenges that cannot be addressed through conventional methods alone. Drawing on Al-Khazini’s example, we've sought to cultivate an environment where creative thinking is encouraged, allowing us to devise innovative solutions that transcend standard approaches. 

Al-Khazini's inventive approach to the science of mechanics, particularly his work on hydrostatic balances, demonstrated an exceptional blend of creativity and analytical thinking. His development of the concept of specific gravity and its application in determining the composition of alloys had profound implications for materials science, a field crucial to engineering. By applying theoretical insights to practical instruments, Al-Khazini exemplified how creative thinking can lead to technological advancements that are both innovative and applicable in the real world.

Through Arkinfo Notes, I share these experiences, highlighting how creativity is harnessed to push the boundaries of what is technically possible. This not only reflects our commitment to innovation but also serves to inspire my readers to think creatively in their endeavours, fostering a community where the imaginative application of engineering principles leads to groundbreaking advancements.

Subscribe now

Ahmad Lahori and the Importance of Art

Ustad Ahmad Lahori was a prominent architect in the 17th century, widely regarded as the principal designer of the Taj Mahal, one of the most iconic symbols of Indian architecture and a masterpiece of Mughal engineering. 

Though details about his life are sparse, historical records suggest he was of Persian descent and served as the chief architect under the Mughal Emperor Shah Jahan. His work on the Taj Mahal, alongside other architectural feats, are a remarkable fusion of engineering precision and artistic beauty.

Ahmad Lahori's architectural genius lies not just in the aesthetic grandeur of his creations but also in the innovative engineering techniques he employed. The Taj Mahal, for example, is celebrated not only for its unparalleled beauty but also for its structural ingenuity, including the complex interplay of light and shadow, the use of symmetry, and the incorporation of elements designed to withstand the test of time. Lahori’s work exemplifies how art and engineering can come together to create structures that are both functional and emotionally resonant.

This approach not only enhances the usability and appeal of engineered products but also inspires solutions that are more holistic and human-centred.

At Arkinfo Notes I have actively explored the intersection of art and engineering in our discussions. In our pursuit of innovation, it is important that we recognise that engineering solutions are not merely about solving problems but also about touching lives and invoking wonder. 

Sameera Moussa and the Importance of Sharing Knowledge

Sameera Moussa, born in Egypt in 1917, was a groundbreaking nuclear physicist who dedicated her career to making medical nuclear technology accessible to everyone. Moussa held a doctorate in atomic radiation and was the first woman to hold a university post as a lecturer in her field in Egypt. She was a visionary who believed in the potential of nuclear technology to revolutionise medicine and sought to demystify its benefits for the greater good. Her efforts led her to organise the "Atoms for Peace" conference and advocate for making nuclear medicine affordable to all, regardless of geographical or economic barriers.

Dr. Moussa's work in nuclear physics and her advocacy for accessible healthcare technology show the impact that engineers and scientists can have on society. Her research on breaking down atoms of cheap materials to produce the same effect as Uranium was groundbreaking. This not only shows the potential for significant advancements in medical treatment but also demonstrates a commitment to ethical science and technology. Moussa’s life was tragically cut short in an accident in 1952, yet her legacy of using engineering to serve humanity continues to inspire.

Sameera Moussa's belief in the democratisation of scientific knowledge is a critical aspect of engineering and technological development. Her vision represents a broader ethos that the true value of scientific discovery lies in its ability to benefit all of humanity, not just a select few. This principle is especially relevant today as we navigate the challenges of making advanced technologies equitable and widely available.

Arkinfo Notes has been committed to demystifying complex engineering and technological concepts for a broader audience at no cost. Through our newsletter, we aim to share insights into our work and industry trends, with a focus on their societal impacts, ensuring that the benefits of technological advancements are widely understood and accessible. 

Share

What’s Ahead

Al Jazari, Ibn al-Haytham, Al-Khazini, Ustad Ahmad Lahori, and Sameera Moussa were remarkable individuals from diverse backgrounds and epochs, exemplifying the enduring spirit of curiosity, creativity, and commitment to the betterment of humanity that lies at the heart of engineering. 

At Arkinfo, and through Arkinfo Notes, we've endeavoured to embody these principles, fostering a community where ideas flourish, innovation thrives, and knowledge is a bridge, not a barrier. 

Your curiosity, engagement, and support have been the wind beneath our wings, propelling us forward, encouraging us to dream bigger, and strive for a future where technology and humanity walk hand in hand.

Looking ahead, I am excited for what the future holds. With each challenge we face, we find new opportunities to innovate, to learn, and to grow. Together, let's continue to explore the uncharted, to question the unquestioned, and to share the wealth of our discoveries. The journey of Arkinfo Notes is not just about chronicling the past but about shaping the future together.

Thank you for being a part of this adventure. Here's to another year of exploration, innovation, and shared success. Let us venture forth with the same spirit of inquiry and openness that has brought us this far, ready to write the next chapter in our collective story.

Download Article as PDF

Leave a comment