Summer Reads for Nerds, Engineers, & the Curious

Book Recommendations for Summer 2024 from AI & Blockchain to Mars and Tennis

Fellow Nerds,

As summer makes an entrance for many of you in the western hemisphere, here in India, we are reaching its conclusion (hopefully). The sun here is no joke. I spend most of my days dodging its relentless glare. 

My schedule hasn't been any kinder as well, suddenly packed with a slew of unexpected projects. Sure it's been chaotic, but hey, chaos has its perks too! It’s pushed me back to my stacks of unread books, allowing me to sneak in some reading between the madness.

I’ve also been diving into learning Japanese and Spanish. Balancing verb conjugations with project deadlines is a juggling act, but it's thrilling in its own right. It reminds me of a Japanese saying, "継続は力なり" (Keizoku wa chikara nari), meaning "Perseverance brings strength." It’s a good reminder that keeping at it, whether it’s language learning or any other challenge, builds more than just skill; it builds character.

So with that in mind, I would like to share with you an eclectic collection of summer reads that you might enjoy. I have got everything packed in here from interesting textbooks on Deep Learning and Blockchain, to some really cool books on Mars and even Tennis (although that one’s more of a life guide than a Tennis guide). So let’s jump right in.

Artwork - Summer Reading

Deep Learning Foundations and Concepts by Christopher M. Bishop

I had to start with this one. I have been reading a bit of this every day for the past two months or so. It's most definitely one of my favourite deep learning books now. It's actually a very very difficult task to write a good Deep Learning book! And I think Bishop did an excellent job at making a very convoluted topic (pun intended) way simpler and comprehensible.

The book dives deep into various neural network architectures, which are the backbone of many modern AI applications. This includes the likes of CNNs (Convolutional Neural Networks) for image-related tasks and RNNs (Recurrent Neural Networks) for sequential data like time series or natural language. These architectures are important because they allow machines to automatically learn complex patterns from data, a fundamental capability for innovations in fields like autonomous driving and real-time language translation.

Bishop goes into detail on how to effectively train these models using algorithms such as backpropagation and various forms of gradient descent. This discussion is central because the efficiency and accuracy of learning directly influence the practical deployment of models in real-world applications, where computational resources and time are often limited.

Techniques such as dropout and batch normalisation are covered comprehensively. These are essential for building models that not only perform well on training data but can generalise to new, unseen data without overfitting. 

The book also includes a self-contained introduction to probability theory, which is fundamental for understanding how to model uncertainty in predictions.

By integrating these topics through various perspectives, textual descriptions, diagrams, mathematical formulas, and pseudo-code, the book ensures that you're not just passively reading but actively engaging with the material. I cannot recommend this book enough!

Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra, and Thomas Wolf

Transformers aren’t as easy to understand as they might initially seem. To understand transformer models, you need to have a good background not only in Neural Networks but also the recent history of AI development and the models that preceded the transformer model. That’s why I love this book. It starts at the very beginning, covering the evolution of natural language processing. Without this precursor, it is very difficult to understand Self-Attention mechanisms that are deployed in the latest models like GPT-4 or Gemini.

It breaks down the transformer architecture introduced by Vaswani et al., explaining self-attention mechanisms and how they differ fundamentally from previous sequence-to-sequence models.

The book details procedures for domain adaptation where a pre-trained model like BERT or GPT is fine-tuned for specific tasks, such as sentiment analysis or question answering, which are crucial for tailoring generic models to specialised tasks.

Techniques like model distillation, pruning, and quantisation are explored to optimise transformers for operational efficiency. These processes are essential for deploying robust NLP models in production environments where resource constraints are a consideration.

The authors also delve into advanced topics such as cross-lingual transfer learning and strategies for dealing with scenarios where labelled data is scarce.

Practical guidance is given on how to scale transformer training to multiple GPUs and manage distributed computing environments, making it a valuable resource for projects requiring large-scale computation.

It also addresses the integration of transformer models into applications, a process that involves fine-tuning, managing data loaders, and customising loss functions and optimisers, all of which are crucial for successful NLP application development.

Destination Mars by Andrew May

Shifting gears a bit (or a lot), let’s talk about another recent curiosity of mine that I have been reading about - Mars. In this book, Andrew May details the technological and scientific challenges that humanity faces in its quest to reach Mars. This book is both a historical recount of our fascination with the red planet and also as a roadmap for the formidable journey ahead.

May begins by outlining the immense distances and harsh conditions that define the Martian landscape, which set the stage for the engineering marvels required to make Mars accessible. He discusses the various spacecraft and rovers that have been part of Mars exploration, from Viking to Curiosity, and highlights how each mission has built upon the last in terms of technology and knowledge.

A significant portion of the book is dedicated to the propulsion technologies required for a manned mission to Mars. As an engineer, I appreciate May's in-depth analysis of chemical rockets, ion thrusters, and the potential of nuclear propulsion. The trade-offs between thrust and efficiency, and the implications for spacecraft design, are particularly fascinating.

May also addresses the life support systems essential for sustaining humans on Mars. He examines the current state of closed ecological systems, waste recycling, and the challenges of producing food in a non-Earth environment. This discussion is crucial, as it ties directly into the sustainability of long-term space habitation.

In the latter sections, May speculates on the future settlements on Mars. He discusses habitat designs, from inflatable modules to 3D-printed structures using Martian regolith. 

While the book doesn’t directly address this issue in great detail, I have been thinking deeply about gravity. Mars has about 35% the measure of Earth’s gravity. This is a significant hurdle. Elon Musk’s idea of nuking the poles to heat the surface and create an atmosphere is only feasible if gravity is strong enough to hold the atmosphere in place. There is no simple answer (or any answer as of yet) to the gravity question. I have been thinking of writing an article on this to explore this properly.

Blockchain Technology: Algorithms and Applications by Asharaf S. et al.

Have you ruled out Blockchain? If so, then I have two things to say to you. One, you are not alone. And two, you might be very wrong about it. I have my reasons for saying this that I intend to cover in a later article, but for now, I would recommend you continue reading up on blockchain. Don’t be deceived by the crypto crowd and their shenanigans, the underlying idea of blockchain is pretty sound and in my opinion, here to stay. 

What I liked about this book is that  it does not merely skim the surface of blockchain like most of the previous textbooks that I have encountered. It dives deep into the mesh of algorithms that form the backbone of this transformative technology.

The authors begin with a great introduction to the cryptographic principles underlying blockchain. They lay out the workings of hash functions, digital signatures, and consensus mechanisms, with a focus on how these elements ensure security and integrity within a distributed network. 

The detailed analysis of consensus algorithms like Proof of Work (PoW) and Proof of Stake (PoS) is very well covered. The book then pivots to a discussion on the development of smart contracts. The authors meticulously describe the programming logic and the operational environment of smart contracts, primarily focusing on Ethereum's Solidity language. They provide code snippets and real-world scenarios that illustrate how smart contracts automate transactions and enforce agreements without intermediaries.

A significant portion of the text is dedicated to exploring various applications of blockchain technology beyond cryptocurrencies which I greatly appreciated. From supply chain management and healthcare to voting systems and financial services, the authors lay out case studies that show blockchain's potential to revolutionise industries by enhancing transparency and trust. 

Blockchain Tethered AI by Karen Kilroy

Continuing the reading on blockchain, my recommendation for a followup reading would be this book. Kilroy masterfully outlines the synergistic potential of these technologies, emphasising how their convergence could redefine the paradigms of security, transparency, and autonomy in digital systems.

The book opens with a foundational discussion on blockchain technology, revisiting its core components such as decentralised ledgers, consensus algorithms, and smart contracts. Kilroy then introduces artificial intelligence, focusing on machine learning algorithms and their capacity for data analysis and decision-making. I appreciate her methodical approach in linking these explanations to the concept of 'Blockchain Tethered AI', which posits that blockchain can serve as a secure, transparent foundation for AI operations.

Kilroy dives deep into the potential applications of this hybrid technology. One standout section involves the use of blockchain to record and verify AI decisions in real-time, which not only enhances trust in AI systems but also creates an immutable audit trail. This aspect is particularly intriguing as it suggests a framework where AI's decision-making processes are both verifiable and non-repudiable, addressing significant concerns in critical areas such as finance and healthcare.

Another compelling discussion in the book revolves around the enhancement of AI's learning processes through blockchain. Kilroy presents a scenario where decentralised data on blockchains can be used to train AI models without compromising data privacy or security. This approach leverages blockchain's data integrity features to ensure that the data used for training AI is accurate and tamper-proof, potentially revolutionising how data is shared and utilised across industries.

Towards the end, Kilroy speculates on future developments, such as the creation of autonomous decentralised organisations run by AI on blockchain platforms. It's quite interesting all in all.

A Brief History of Intelligence by Max Bennet

Max Bennet takes us on an enlightening journey in this book, covering the evolution of intelligence, from its biological roots to its most recent developments in artificial intelligence. This book is not just history, but also deeply analytical, posing significant questions about the nature of intelligence itself.

The book begins with biological intelligence, discussing how cognitive functions have developed across different species over millions of years. The neurological underpinnings of intelligence are covered, explaining complex concepts like neural plasticity and cognitive architecture in a manner that resonates with engineers.

After laying this sound foundation, Bennet smoothly transitions to artificial intelligence, providing a thorough review of its historical progression, from early theories and models in the mid-20th century to the sophisticated algorithms we see today. He explores key milestones, such as the development of neural networks and the impact of machine learning techniques, which have enabled AI systems to perform tasks ranging from pattern recognition to decision making under uncertainty.

A core theme in this part of the book is the parallel drawn between biological and artificial intelligence. Bennet discusses how modern AI research increasingly seeks to mimic human cognitive processes, such as learning and problem-solving, through deep learning and reinforcement learning. If you read my works regularly, then you know that I have great apprehensions about this idea. But as an engineer, I find this comparison intriguing and can’t help but be drawn to its nuances.

Rust Atomics and Locks by Mara Bos

This book is part of my ongoing venture of infiltrating into the Rustacian clan. Although a big supporter of Rust, I have yet to actively use it in one of my projects and I am still exploring avenues of interest for myself. If you are on the same boat (or a similar boat), then hop on and read this one.

The focus of this book is on the concurrency features of the Rust programming language, which are important for writing safe and efficient multi-threaded code. The book meticulously explores the mechanisms Rust offers to manage memory safely in concurrent systems, which is particularly relevant given Rust’s emphasis on safety and performance.

The author begins with an introduction to Rust’s memory safety guarantees, particularly its ownership, borrowing, and lifetime principles, which naturally lead into a discussion on concurrency. This foundation is crucial for understanding how Rust attempts to prevent data races and other concurrency errors that are common in systems programming.

The book provides a deep dive into Rust’s atomic types in the std::sync::atomic module. It explains the concept of atomics as low-level primitives that provide safe shared access to mutable data. The detailed coverage of different atomic operations such as load, store, fetch_add, and fetch_sub, and how these can be used to build non-blocking data structures, is particularly insightful. The discussion on memory ordering options—Relaxed, Acquire, Release, AcqRel, and SeqCst—and their implications on the visibility of side effects across threads both comprehensive and critical for designing concurrent applications.

The section on locks is equally thorough. It contrasts the use of mutexes, read-write locks, and condition variables with the lock-free techniques described earlier. The book details how std::sync::Mutex and std::sync::RwLock can be used to protect data with exclusive access, including practical code examples that illustrate common patterns and potential pitfalls such as deadlock scenarios.

One of the standout features of this book are the litany of code examples that illustrate both correct and problematic usage. This is super helpful for a Rust newbie such as myself.

Command Line Rust by Ken Younes Clark

I love designing CLIs. I can share with you a list of logical and practical reasons for it but simply put, I find them super cool! There is something really ‘engiineery’ about working on a terminal that can never be replaced by a UI. As someone who has designed plenty of CLIs in his lifetime, I am only naturally drawn to potentially do the same with Rust.

This book provides a comprehensive tour of Rust’s capabilities, particularly its safety features and performance, which are key for developing command-line tools.

The book starts with an introduction to Rust’s syntax and basic concepts, such as ownership, types, and error handling, setting a solid foundation even for readers new to the language such as myself. Clark emphasises the importance of Rust's type system and compiler checks, which ensure reliability and maintainability of code.

As I progress through the book, I appreciate Clark's focus on practical application. He guides readers through setting up a Rust development environment and crafting a basic "Hello, world!" application. From there, he delves into more complex topics such as parsing command-line arguments using the clap crate, which simplifies the process of handling user inputs.

One of the sections that I appreciated the most was on error handling in Rust. It covered how to manage expected and unexpected errors gracefully. Clark provides examples using the Result and Option types to handle different failure modes, which is crucial for building resilient applications that can cope with a variety of user input errors and system failures.

The Staff Engineer’s Path by Tanya Reilly

Alright, stepping out of the CLI and jumping into office politics. Jokes apart, it is important that we talk about the role of a staff engineer. Tanya Reilly does a splendid job defining the responsibilities of a staff engineer. 

Reilly begins by defining what a staff engineer is and what they are not, setting clear expectations about the role. She emphasises that becoming a staff engineer involves much more than just technical excellence; it requires a strategic vision, the ability to influence without authority, and a commitment to mentoring others. This introduction sets the stage for understanding the depth and breadth of impact a staff engineer is expected to have within an organisation. If you are an engineer worried about AI taking away your job, I actually recommend reading this one because if that were to really happen (although I don’t subscribe to that possibility), then a staff engineer is probably the safest roles to hold.

One of the core chapters discusses the key skills necessary for the role. It’s not just about coding; Reilly highlights skills such as architectural insight, system design, technical decision-making, and cross-functional leadership. Each skill is explored in detail, with practical advice on how to develop and demonstrate these capabilities in a work setting. I felt it was a must read for engineers.

Reilly also gives advice on building relationships with key stakeholders, understanding business goals, and leveraging technical strategies to drive organisational success. These elements are critical, as they illustrate how staff engineers can use their position to bridge the gap between engineering teams and executive management.

Reilly also addresses the challenges and common pitfalls in the path to becoming a staff engineer, such as the risk of becoming isolated in deep technical work without broader influence. She offers strategies for maintaining visibility in the organisation, such as leading high-impact projects and engaging in community and industry events.

Towards the end, she shifts focus towards long-term career growth, discussing how to sustain the role's demands while continuing personal and professional development.

The Inner Game of Tennis by Timothy Gallwey

Finally, I would like to wrap up my recommendations with some ‘philosophy of life’. I loved this book primarily because of Tennis metaphor. Being a huge fan of the sport, I could seriously relate to the ‘mental’ game that the author talks about in detail. Also, apparently this is one of Bill Gates’s favourite reads as well. Given that he is another Tennis fan, I am not surprised. 

It's a profound exploration of the psychology of performance, which resonates deeply with me as an engineer. Gallwey introduces a revolutionary approach to learning and coaching that is as applicable to mastering a programming language as it is to hitting a forehand.

Gallwey starts with the premise that our minds have two selves, Self 1, the teller, and Self 2, the doer. Self 1 is analytical and often critical, constantly offering instructions and criticisms, while Self 2 is the self that actually performs the action. He argues that learning and performance are enhanced when Self 1 is quieted and allows Self 2 to operate based on the body's intuitive and natural abilities. 

A significant portion of the book is dedicated to the concept of "letting go" of judgement and the fear of failure, which leads to greater focus and a higher level of performance. Gallwey provides practical strategies for achieving this state, such as focusing on the seams of the tennis ball, which parallels focusing on the process rather than the outcome of a complex project or coding challenge.

Gallwey also delves into the concept of "trust," which involves allowing one’s own abilities to flourish without over-management. It also touches on effective coaching techniques. Gallwey advocates for coaches (or leaders) to facilitate learning by creating an environment that is conducive to self-discovery rather than instruction-heavy. 

---