ericjmorey

Video description:

We reproduce the GPT-2 (124M) from scratch.

This video covers the whole process:

First we build the GPT-2 network, then we optimize its training to be really fast, then we set up the training run following the GPT-2 and GPT-3 paper and their hyperparameters, then we hit run, and come back the next morning to see our results, and enjoy some amusing model generations.

Keep in mind that in some places this video builds on the knowledge from earlier videos in the Zero to Hero Playlist (see my channel). You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

The presenter compares some of the functionality to Leap.nvim

Trey Hunner writes:

This article is primarily meant to act as a Python time complexity cheat sheet for those who already understand what time complexity is and how the time complexity of an operation might affect your code. For a more thorough explanation of time complexity see Ned Batchelder's article/talk on this subject.

Read Python Big O: the time complexities of different data structures in Python

Isaac Harris-Holt writes:

Modern software engineering can be a bit of a nightmare. You've got all these different third-party services to keep track of, and often many first-party microservices too! As it turns out, things can be made more simple using Gleam. Let me show you how.

In the command-line window the command line can be edited just like editing text in any window. It is a special kind of window, because you cannot leave it in a normal way.

There are two ways to open the command-line window:

1. From Command-line mode, use the key specified with the 'cedit' option (default CTRL-F).

2. From Normal mode, use the "q:", "q/" or "q?" command.

• This starts editing an Ex command-line ("q:") or search string ("q/" or "q?"). Note that this is not possible while recording is in progress (the "q" stops recording then).

When the window opens it is filled with the command-line history. The last line contains the command as typed so far. The left column will show a character that indicates the type of command-line being edited

Pedro Domingos summarizes 12 key lessons that machine learning researchers and practitioners have learned. These include pitfalls to avoid, important issues to focus on, and answers to common questions.

• 1. Learning = Representation + Evaluation + Optimization
• 2. It's Generalization that Counts
• 3. Data Alone Is Not Enough
• 4. Overfitting Has Many Faces
• 5. Intuition Fails in High Dimensions
• 6. Theoretical Guarantees Are Not What They Seem
• 7. Feature Engineering Is The Key
• 8. More Data Beats a Cleverer Algorithm
• 9. Learn Many Models, Not Just One
• 10. Simplicity Does Not Imply Accuracy
• 11. Representable Does Not Imply Learnable
• 12. Correlation Does Not Imply Causation