Review - buttonmaker.xyz

In this series, I edit podcasts into 3 points.

This first installment is based on Practical AI podcast #138

Episode hosts Dan and Chris have a conversation with William Falcon, creator of PyTorch Lightning & CEO of Grid AI

The Three Points Series #1 – Why Using The World’s Top Research Lab Will Make Your Life Better

P1 Situation: Current private and academic research systems hinder AI and ML progress

a) Dissemination of research: It usually costs thousands of dollars for researchers to publish on Elsevier-managed journals… To read the entire paper, their colleagues have to access these journals such as one large scientific publisher, Springer Nature, which has online-only prices that average about $2,020 per year in 2021.
b) Registration of precedence: It make(s) it a lot more expensive and painful to look for prior art so that you don’t end up repeating someone else’s work by accident.
c) Providing a fixed archival version for future reference: When privately-owned journals go under, their archives can simply vanish from the internet.
d) Certification of quality: The replication crisis that first emerged in the early 2010’s has since spread to numerous other fields. Multiple studies from the past decade show that well over half the surveyed papers in various disciplines failed to replicate.
– Balaji Srinivasan’s, https://1729.com/crypto-sci-hub

P2 PyTorch Lighting: Framework advances research and application

As existing private and academic research systems become diffuse, PyTorch Lighting concentrates resources and information in one place.

William Falcon created PyTorch Lighting along with an open source community of people grappling with the systemic and technical challenges already present with new research. Inspired through the frustrations of AI research during his Ph.D. work at NYU and later on at FAIR. (Facebook Artificial Intelligence Research)

“My vision was
Can we build the world’s research lab?
Can we all have access to top researchers and resources?”

PyTorch Lighting is doing just that. The open source project is approaching 500 contributors. Top researchers and Ph.D’s from all over the world are implementing AI projects and putting them into papers which are then available within a few hours, ready and usable for everyone.

The idea of the world’s research lab is to be able to stand on the shoulders of giants before even starting your work. This leads to time spent on new innovations or improving from the past, instead of going through all the frustrations required to get to the same place other people have already figured out.

A major goal of the project is to solve for issues in efficiency and communication through interoperability. For those who are not familiar with this term, it means that the combination and sequence originally used with the data, model and hardware is abstracted up to retain the seed or source in a single callable location. This location is then called upon by an individual researcher to iterate from the exact parameters of their original idea, or is shared and iterated upon by collaborators within their team or other teams. This gives you and the teams you work with the ability to iterate faster and more accurately, with state of the art version control.

“PyTorch Lighting is a research project – how do you factor out deep learning code and make it interoperable… The outcome of doing anything with AI is a function of how fast you iterate through ideas.”
William Falcon

P3 Grid AI: Power through iterations faster to get applied outcomes

“How fast you can power through ideas is probably the single biggest predictor if that thing is going to work or not.”
William Falcon

Grid tooling is similar to the offerings of database tooling. You can try to in-house the system design for a database, but inevitably the in-house hardware limitation on bursting compute power makes outside solutions a more reliable option. As history has shown with database services overwhelmingly proving out the advantages, one example being Amazon Web Services (AWS) furthering its sector growth in 2021 to make up 52% of Amazon’s Operating Revenue¹, the same concept holds for AI and ML service solutions.

“Running on Grid means that we install your dependencies, everything you need to link up your data, in a matter of minutes, if not seconds. It’s just there, and it’s repeatable, and things start immediately, so it’s a lot cheaper. With Grid you can go spin up 200 GPUs, run for five minutes and shut them down, and you just got a lot done. Whereas on your own machines, even if you were to do it yourself on the cloud, you would probably not even get the models running for 20 minutes while you spin up the machines and set up all that stuff.”
William Falcon

Using Grid means that all the hard work has been done in advance through accumulated knowledge. To decide on the capacity to in-house this, access for yourself. Am I using my own hardware networking or AWS? In the former, go for it. If the latter, you will likely save a great deal of headache by leveraging the benefits of specialization.

¹ https://www.techradar.com/news/aws-is-now-a-bigger-part-of-amazons-overall-success-than-ever

Founding vs Inheriting

The truth about mythological founders, or the reality of the Needed Network (NN)

This post is about how one degree of freedom networks overwhelmingly determine outcomes for founders. If you haven’t already, read this essay about the influence of cities from Paul Graham. It is required reading on the subject.

Parallel to the subject post by Balaji, the “East Coast” ideology has permeated the five large technology companies on the West Coast that have entered their institutional era. These technology companies have the closest proximity network to the world of start-ups and ways for an unknown founder to gain start-up capital. As Marc Andreessen said from a non-Ivy League perspective, even those innovators who have built something that people want need to punch into the warm introduction network somehow. Tactically, if starting from scratch, Marc advised getting into one of the big technology companies and working your way up. Let’s call this the Needed Network. (NN)

Below is a grounding connection between the power of institutional ideology⁰ and the NN of technology founders.

With the exception of Vitalik Buterin, the 11 founders mentioned in the subject post had the NN of an Ivy League¹ institution on their Wiki.

The truth is this:

Social proof from acceptance and association with these institutions has been the dominant way to enter the NN of a technology founder.

The principal components of this reality are being reconstructed to increase the global NN through technology companies and XDigital Communities. For the purposes of talking about this space, XDigital Communities encompass the identity optional pseudonyms used on Twitter, Reddit, and Discord through the spectrum to decentralized network states.

NNs are expanding from Harvard to Pinterest to XDigital Communities

Smart people make things. And Sahil Lavingia is agreeably so good that he can’t be ignored. However, these two maxims alone are not enough to become a mythological technology founder.

As a quintessential maverick founder, Sahil stands in contrast to the subject post’s selection of people who may have otherwise become professors. Sahil famously built the first iteration of Gumroad in a weekend at the age of 19. He was accepted and dropped out of the University of Southern California, a non-Ivy League institution. What is interesting about Sahil is that he gained his NN through acceptance and admittance into Pinterest, a new West Coast technology company in 2011, before he got Gumroad off the ground.

It was through Pinterest that Sahil gained the NN, including the mentors and credibility, to secure funding for Gumroad’s success in the future.

As Ivy League institutions compete with one another on lowest % acceptance rate, a new pool of people with the aptitude to be successful founders are gaining into NNs through technology companies and XDigital Communities.

This is not to say that the Ivy League is going away. Harvard is not going away. Ivies will remain stalwart, numerically similar and only relatively smaller to the growing pie of NNs. As the change is net new growth of NNs, or the expanding scope.

Even as of March 2021, 54% of a16z portfolio companies had made plans to embrace the competitive advantages offered by the remote first model. Along with the rise of asynchronous working, these phenomenon will accelerate into more globally distributed Sahils, more high aptitude founders admitted to the NN slice technology companies offer, in real volume. And this slice was already swelling at a non-linear rate, primarily as a function of software eating the world over the last 20 years.

Now the question is this:

How and how quickly will XDigital Communities become the majority of the NN pie?

⁰This post covers a sub-section of educational institutions within the larger situation of institutional ideology. Trust in institutions is shrinking quickly. And for good reason. These institutions were simply designed and entrenched at times with less technology and information available while also being varyingly recalcitrant to innovation. Lagging problems have split into more and more avoidable issues such as recent voting right regulations in legal contention, which could be all together bypassed by blockchain based applications usable via a public hub or internet connection.

¹Ivy League is defined for this post as the Top 20 lowest acceptance rate U.S. schools.
Larry Page, Peter Thiel, Daphne Koller (Stanford)
Andrew Ng, Diane Greene (MIT)
Michael Moritz (Penn)
Paul Graham, Bill Gates, Mark Zuckerberg (Harvard)
Jeff Bezos (Princeton)