Categories
Blog

The Fish and The Cat

“The Fish and The Cat” is a parable for why bad explanations of reality are so reliable at generating blindspots and why this makes it hard to understand the functions that are the good explanations of reality.


To begin, let’s start in 2011. Physicist David Deutsch published a book called “The Beginning of Infinity1”. It is a very long and challenging book. Below is Naval Ravikant’s2 take on it. 

I was pleasantly surprised a couple of years back when I opened an old book that I’d read a decade ago called The Beginning of Infinity by David Deutsch.

Sometimes you read a book and it makes a difference right away. Sometimes you read a book and you don’t understand it; then you read it at the right time and it makes a difference.

People throw around the phrase “mental models” a lot. Most mental models aren’t worth reading or thinking about or listening to because they’re trivial.

But the concepts that came out of The Beginning of Infinity are transformational because they very convincingly change the way that you look at what is true and what is not.

The important thing to understand is getting to what is true and what is not. Building from this context, we now layer in the idea of the Cargo Cult3, introduced by Nobel Prize Physicist Richard Feynman. This is a concept describing the perils of bad explanations and how we fool ourselves into believing how the bad explanation is the best way to understand reality. 

I would like to call Cargo Cult Science.  In the South Seas there is a Cargo Cult of people.  During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now.  So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land.  They’re doing everything right.  The form is perfect.  It looks exactly the way it looked before.  But it doesn’t work.  No airplanes land.  

So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.” But this long history of learning how to not fool ourselves—of having utter scientific integrity—is, I’m sorry to say, something that we haven’t specifically included in any particular course that I know of.  We just hope you’ve caught on by osmosis.

The first principle is that you must not fool yourself—and you are the easiest person to fool.  So you have to be very careful about that.  After you’ve not fooled yourself, it’s easy not to fool other scientists.  You just have to be honest in a conventional way after that.  One example of the principle is this: If you’ve made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out.  If we only publish results of a certain kind, we can make the argument look good. 

We must publish both kinds of result.  For example—let’s take advertising again—suppose some particular cigarette has some particular property, like low nicotine.  It’s published widely by the company that this means it is good for you—they don’t say, for instance, that the tars are a different proportion, or that something else is the matter with the cigarette.  In other words, publication probability depends upon the answer.  That should not be done.

What we bring from here is that while X (less nicotine is good for you) may be true, if Y (more tar has a more severe impact on lifespan) is also true, than Z. (it is likely that it is less healthy for you in reality)

X is a dangerous and bad explanation. That makes it harder to get to Y and to understand Z.


Let’s simplify and reduce to “The Fish and The Cat”.

In Dr. Seuss’s book “the Cat in the Hat4”, there is a scene where the Fish tells the Cat why it is not a cat. The explanation of the Fish is that it has never heard of a six foot cat, (X) thus that the Cat is not a cat. This is a bad explanation. The truth is simple. If the cat has whiskers, a tail, a cat heart, a cat brain… it is empirically a cat. This is an infinitely better explanation. However, due to the bad explanation, the Fish fails to understand the reality. 

Fish: “You are not a Cat. I’ve Never Heard of a Six foot Cat.”

Cat: “But I am a Cat. Technically I am a Cat in a Hat”

Fish: “You are not a Cat in a Hat. That’s not a Hat and you are not a Cat.”

Cat: “I am indeed a Cat. And this is indeed a Hat.”

It seems simple, but I find myself grappling with these types of blindspots5 all the time. Where there is a bad explanation that in and of itself stands in the way of the good explanation and requires a great deal of effort to get past. 


Honorable mention!

As we near the 20th anniversary of the iPod we look to an Internet form where 20 years ago intelligent people greatly underestimated the importance of the qualitative for the success of the pre-cursor to the most important invention6 of the 21st century. 

Take a look at the explanations of why it would fail for yourself. https://forums.macrumors.com/threads/apples-new-thing-ipod.500/7

One bad explanation was that the competitors had more memory at a cheaper price. While this may be true (X), the better explanation (Y) on the impact of reality (Z) was that the product gave more value through the qualitative aspects of the user experience and user perception of the product. (underestimated factors from the forum commenters, in part because these are harder to quantify vs the reality of the mass market… also known as people and how they feel about the product)


https://en.wikipedia.org/wiki/The_Beginning_of_Infinity1

https://nav.al/infinity2

https://calteches.library.caltech.edu/51/2/CargoCult.htm 3

https://en.wikipedia.org/wiki/The_Cat_in_the_Hat 4

https://upskillcoach.com/blog/what-is-the-johari-window/ 5

https://www.itv.com/news/2016-01-27/smartphone-is-most-important-consumer-invention-of-21st-century-so-far 6

Categories
Blog Review

The 3 Points Series #1

In this series, I edit podcasts into 3 points.

This first installment is based on Practical AI podcast #138

Episode hosts Dan and Chris have a conversation with William Falcon, creator of PyTorch Lightning & CEO of Grid AI

The Three Points Series #1 – Why Using The World’s Top Research Lab Will Make Your Life Better

P1 Situation: Current private and academic research systems hinder AI and ML progress


a)  Dissemination of research: It usually costs thousands of dollars for researchers to publish on Elsevier-managed journals… To read the entire paper, their colleagues have to access these journals such as one large scientific publisher, Springer Nature, which has online-only prices that average about $2,020 per year in 2021.

b) Registration of precedence: It make(s) it a lot more expensive and painful to look for prior art so that you don’t end up repeating someone else’s work by accident.

c) Providing a fixed archival version for future reference: When privately-owned journals go under, their archives can simply vanish from the internet. 

d) Certification of quality: The replication crisis that first emerged in the early 2010’s has since spread to numerous other fields. Multiple studies from the past decade show that well over half the surveyed papers in various disciplines failed to replicate.

– Balaji Srinivasan’s, https://1729.com/crypto-sci-hub

P2 PyTorch Lighting: Framework advances research and application


As existing private and academic research systems become diffuse, PyTorch Lighting concentrates resources and information in one place. 

William Falcon created PyTorch Lighting along with an open source community of people grappling with the systemic and technical challenges already present with new research. Inspired through the frustrations of AI research during his Ph.D. work at NYU and later on at FAIR. (Facebook Artificial Intelligence Research)

“My vision was
Can we build the world’s research lab?
Can we all have access to top researchers and resources?”

PyTorch Lighting is doing just that. The open source project is approaching 500 contributors. Top researchers and Ph.D’s from all over the world are implementing AI projects and putting them into papers which are then available within a few hours, ready and usable for everyone. 

The idea of the world’s research lab is to be able to stand on the shoulders of giants before even starting your work. This leads to time spent on new innovations or improving from the past, instead of going through all the frustrations required to get to the same place other people have already figured out. 

A major goal of the project is to solve for issues in efficiency and communication through interoperability. For those who are not familiar with this term, it means that the combination and sequence originally used with the data, model and hardware is abstracted up to retain the seed or source in a single callable location. This location is then called upon by an individual researcher to iterate from the exact parameters of their original idea, or is shared and iterated upon by collaborators within their team or other teams. This gives you and the teams you work with the ability to iterate faster and more accurately, with state of the art version control. 

“PyTorch Lighting is a research project – how do you factor out deep learning code and make it interoperable… The outcome of doing anything with AI is a function of how fast you iterate through ideas.”

William Falcon

P3 Grid AI: Power through iterations faster to get applied outcomes


“How fast you can power through ideas is probably the single biggest predictor if that thing is going to work or not.”

William Falcon

Grid tooling is similar to the offerings of database tooling. You can try to in-house the system design for a database, but inevitably the in-house hardware limitation on bursting compute power makes outside solutions a more reliable option. As history has shown with database services overwhelmingly proving out the advantages, one example being Amazon Web Services (AWS) furthering its sector growth in 2021 to make up 52% of Amazon’s Operating Revenue1, the same concept holds for AI and ML service solutions.

“Running on Grid means that we install your dependencies, everything you need to link up your data, in a matter of minutes, if not seconds. It’s just there, and it’s repeatable, and things start immediately, so it’s a lot cheaper. With Grid you can go spin up 200 GPUs, run for five minutes and shut them down, and you just got a lot done. Whereas on your own machines, even if you were to do it yourself on the cloud, you would probably not even get the models running for 20 minutes while you spin up the machines and set up all that stuff.”

William Falcon

Using Grid means that all the hard work has been done in advance through accumulated knowledge. To decide on the capacity to in-house this, access for yourself. Am I using my own hardware networking or AWS? In the former, go for it. If the latter, you will likely save a great deal of headache by leveraging the benefits of specialization.

1 https://www.techradar.com/news/aws-is-now-a-bigger-part-of-amazons-overall-success-than-ever

Categories
1729 Blog Review

Founding vs Inheriting

The truth about mythological founders, or the reality of the Needed Network (NN)

This post is about how one degree of freedom networks overwhelmingly determine outcomes for founders. If you haven’t already, read this essay about the influence of cities from Paul Graham. It is required reading on the subject.


Parallel to the subject post by Balaji, the “East Coast” ideology has permeated the five large technology companies on the West Coast that have entered their institutional era. These technology companies have the closest proximity network to the world of start-ups and ways for an unknown founder to gain start-up capital. As Marc Andreessen said from a non-Ivy League perspective, even those innovators who have built something that people want need to punch into the warm introduction network somehow. Tactically, if starting from scratch, Marc advised getting into one of the big technology companies and working your way up. Let’s call this the Needed Network. (NN)

Below is a grounding connection between the power of institutional ideology0 and the NN of technology founders.

With the exception of Vitalik Buterin, the 11 founders mentioned in the subject post had the NN of an Ivy League1 institution on their Wiki. 

The truth is this:

Social proof from acceptance and association with these institutions has been the dominant way to enter the NN of a technology founder. 

The principal components of this reality are being reconstructed to increase the global NN through technology companies and XDigital Communities. For the purposes of talking about this space, XDigital Communities encompass the identity optional pseudonyms used on Twitter, Reddit, and Discord through the spectrum to decentralized network states.

NNs are expanding from Harvard to Pinterest to XDigital Communities

Smart people make things. And Sahil Lavingia is agreeably so good that he can’t be ignored. However, these two maxims alone are not enough to become a mythological technology founder.

As a quintessential maverick founder, Sahil stands in contrast to the subject post’s selection of people who may have otherwise become professors. Sahil famously built the first iteration of Gumroad in a weekend at the age of 19. He was accepted and dropped out of the University of Southern California, a non-Ivy League institution. What is interesting about Sahil is that he gained his NN through acceptance and admittance into Pinterest, a new West Coast technology company in 2011, before he got Gumroad off the ground.

It was through Pinterest that Sahil gained the NN, including the mentors and credibility, to secure funding for Gumroad’s success in the future.

As Ivy League institutions compete with one another on lowest % acceptance rate, a new pool of people with the aptitude to be successful founders are gaining into NNs through technology companies and XDigital Communities. 

This is not to say that the Ivy League is going away. Harvard is not going away. Ivies will remain stalwart, numerically similar and only relatively smaller to the growing pie of NNs. As the change is net new growth of NNs, or the expanding scope. 

Even as of March 2021, 54% of a16z portfolio companies had made plans to embrace the competitive advantages offered by the remote first model. Along with the rise of asynchronous working, these phenomenon will accelerate into more globally distributed Sahils, more high aptitude founders admitted to the NN slice technology companies offer, in real volume. And this slice was already swelling at a non-linear rate, primarily as a function of software eating the world over the last 20 years. 

Now the question is this:

How and how quickly will XDigital Communities become the majority of the NN pie?


This post covers a sub-section of educational institutions within the larger situation of institutional ideology. Trust in institutions is shrinking quickly. And for good reason. These institutions were simply designed and entrenched at times with less technology and information available while also being varyingly recalcitrant to innovation. Lagging problems have split into more and more avoidable issues such as recent voting right regulations in legal contention, which could be all together bypassed by blockchain based applications usable via a public hub or internet connection.

Ivy League is defined for this post as the Top 20 lowest acceptance rate U.S. schools.
Larry Page, Peter Thiel, Daphne Koller (Stanford)
Andrew Ng, Diane Greene (MIT)
Michael Moritz (Penn)
Paul Graham, Bill Gates, Mark Zuckerberg (Harvard)
Jeff Bezos (Princeton)