Machine Learning

When training a transformer on positionally encoded embeddings, should the tgt output embeddings also be positionally encoded? If so, wouldn't the predicted/decoded embeddings also be positionally encoded?

Someone (Dreamertist on reddit) got tired of depending on Huggingface for downloading models and proposes a torrent tracker to share more efficiently these huge blobs.

It just started, only a few models uploaded yet, but I think it is worth that we all put our local stash online there. Making a new torrent is super easy (one missing step though: when "re-downloading" the model you need to save it in the directory where it already exists. This way it will "resume" at 100% completion and switch to seeding mode)

Imagine AI giving offsprings...

Hey guys,

I have been experimenting with self-supervised visual learning a bit. Until now I have only ever used U-Nets and related architectures.

No matter what specific task, images or other parameters I changed I always encountered these stains on my output-images (here marked with green), although sometimes more, sometimes less.

Now I wondered if anybody could tell me where they came from and how I could prevent them?

In the attached picture the input (left) and target (right) are the same, so that I can be sure these stains do not come from a badly designed learning task, yet they still appear (output is the middle image).

Thanks in advance and all the best :D

Edit: added line breaks

Copilot sounds amazing on paper. The free (to 365 subs) version on the web is just Chat GPT4, so that's familiar enough. The integration with 365 applications is really what grabs me. Stuff like tossing it 10 spreadsheets and asking it to analyze and compare the data, having a virtual assistant to remind me of upcoming actionables, and summarizing a meeting when I zone out - it all sounds really handy.

I met with Microsoft last week and they're down for giving me a 90 day trial if I want to take it for a spin. Any thoughts or suggestions? I ideally want to determine if this will improve productivity for my end users enough to be worth the insane cost of $30/user/mo.

Hi all,

I think around 1 or 2 years ago, I stumbled upon a personal blog of an asian woman (I think) working at OpenAI. She had numerous extensive fascinating blog posts on a black themed blog, going into the technical details of embeddings of language models and such.

I can no longer find that blog and have no other information to go by. Would anyone possibly know which blog I'm referring to? It would be very much appreciated.

2024-02-29 | Christopher Gadzinski writes:

Physics likes optimization! Subject to its boundary conditions, the time evolution of a physical system is a critical point for a quantity called an action. This point of view sets the stage for Noether's principle, a remarkable correspondence between continuous invariances of the action and conservation laws of the system.

In machine learning, we often deal with discrete "processes" whose control parameters are chosen to minimize some quantity. For example, we can see a deep residual network as a process where the role of "time" is played by depth. We may ask:

1. Does Noether's theorem apply to these processes?
2. Can we find meaningful conserved quantities?

Our answers: "yes," and "not sure!"

Anybody got to try it?

Itamar Turner-Trauring writes:

These sort of problems are one of the many reasons you want to “pin” your application’s dependencies: make sure you only install a specific, fixed set of dependencies. Without reproducible dependencies, as soon as NumPy 2 comes out your application might break when it gets installed with new dependencies.

The really short version is that you have two sets of dependency configurations:

• A direct dependency list: A list of libraries you directly import in your code, loosely restricted. This is the list of dependencies you put in pyproject.toml or setup.py.
• A lock file: A list of all dependencies you rely on, direct or indirect (dependencies of dependencies), pinned to specific versions. This might be a requirements.txt, or some other file dependencies on which tool you’re using.

At appropriate intervals you update the lock file based on the direct dependency list.

I’ve written multiple articles on the topic, in case you’re not familiar with the relevant tools:

• “Faster Docker builds with pipenv, poetry, or pip-tools” covers using those three tools to maintain lockfiles.
• For Conda, see “Reproducible and upgradable Conda environments with conda-lock”.

Read NumPy 2 is coming: preventing breakage, updating your code

cross-posted from: https://slrpnk.net/post/3892266

Institution: Cambridge
Lecturer: Petar Velickovic
University Course Code: seminar
Subject: #math #machinelearning #neuralnetworks
Description: Deriving graph neural networks (GNNs) from first principles, motivating their use, and explaining how they have emerged along several related research lines.

cross-posted from: https://slrpnk.net/post/3863486

Institution: MIT
Lecturer: Prof. Manolis Kellis
University Course Code: MIT 6.047
Subject: #biology #computationalbiology #machinelearning

More at [email protected]

Hi! Hopefully this is a good place to ask. I've been googling around a fair bit, but haven't had much luck- I'm either finding ELI5 type articles, or in depth tutorials on setting up a model to tell the difference between a frog and a dog. I'm not sure if those are relevant to my concept.

I would like to implement a ML algorithm to detect a particular type of defect on a production line. Our current camera system isn't quite up to the task, but gives good, consistent imagery, and I have a good historical dataset. The product moves past the camera, it snaps a single black and white image, then the product moves on. This means that most of my images are more or less the same. These defects are obvious to the human eye.

Could someone please give me, a noob, a bird's eye view of how I would go about using ML to create a model for this? There's so many choices of tools and tutorials that I don't know which would be best suited to this use case.

I've had my eyes on optoelectronics as the future hardware foundation for ML compute (add not just interconnect) for a few years now, and it's exciting to watch the leaps and bounds occurring at such a rapid pace.

Hello Machine Learning Community,

The intention of this post is to replicate a similar tradition from R/machinelearning and to trigger engagement. This post will be created weekly.

What are you reading this week and any thoughts to share?

When I train my PyTorch Lightning model on two GPUs on jupyter lab with strategy="ddp_notebook", only two CPUs are used and their usages are 100%. How can I overcome this CPU bottleneck?

Edit: I tested with PyTorchProfiler and it was because of old ssds used on the server

Hello Machine Learning Community,

The intention of this post is to replicate a similar tradition from R/machinelearning and to trigger engagement. This post will be created weekly.

What are you reading this week and any thoughts to share?

Hello Machine Learning Community,

The intention of this post is to replicate a similar tradition from R/machinelearning and to trigger engagement. This post will be created weekly.

What are you reading this week and any thoughts to share?

Hello Machine Learning Community,

The intention of this post is to replicate a similar tradition from R/machinelearning and to trigger engagement. This post will be created weekly.

What are you reading this week and any thought to share on it ?

I'd love to know what others are reading, why they think it's awesome (or not). In general, get an exposure to other sub genres of ML. Most of the papers I read are in the computer vision domain cause of work so I'd appreciate reading more about others.

So...

1. Are you all interested in such a post ?
2. If yes, which day of the week ?