Sal

Cross-posting to the OpenSource community as I think this topic will also be of interest here.

This is an analysis of how "open" different open source AI systems are. I am also posting the two figures from the paper that summarize this information below.

ABSTRACT

The past year has seen a steep rise in generative AI systems that claim to be open. But how open are they really? The question of what counts as open source in generative AI is poised to take on particular importance in light of the upcoming EU AI Act that regulates open source systems differently, creating an urgent need for practical openness assessment. Here we use an evidence-based framework that distinguishes 14 dimensions of openness, from training datasets to scientific and technical documentation and from licensing to access methods. Surveying over 45 generative AI systems (both text and text-to-image), we find that while the term open source is widely used, many models are ‘open weight’ at best and many providers seek to evade scientific, legal and regulatory scrutiny by withholding information on training and fine-tuning data. We argue that openness in generative AI is necessarily composite (consisting of multiple elements) and gradient (coming in degrees), and point out the risk of relying on single features like access or licensing to declare models open or not. Evidence-based openness assessment can help foster a generative AI landscape in which models can be effectively regulated, model providers can be held accountable, scientists can scrutinise generative AI, and end users can make informed decisions.

Figure 2 (click to enlarge): Openness of 40 text generators described as open, with OpenAI’s ChatGPT (bottom) as closed reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points. RL may refer to RLHF or other forms of fine-tuning aimed at fostering instruction-following behaviour. For the latest updates see: https://opening-up-chatgpt.github.io

Figure 3 (click to enlarge): Overview of 6 text-to-image systems described as open, with OpenAI's DALL-E as a reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points.

There is also a related Nature news article: Not all ‘open source’ AI models are actually open: here’s a ranking

PDF Link: https://dl.acm.org/doi/pdf/10.1145/3630106.3659005

ABSTRACT

The past year has seen a steep rise in generative AI systems that claim to be open. But how open are they really? The question of what counts as open source in generative AI is poised to take on particular importance in light of the upcoming EU AI Act that regulates open source systems differently, creating an urgent need for practical openness assessment. Here we use an evidence-based framework that distinguishes 14 dimensions of openness, from training datasets to scientific and technical documentation and from licensing to access methods. Surveying over 45 generative AI systems (both text and text-to-image), we find that while the term open source is widely used, many models are ‘open weight’ at best and many providers seek to evade scientific, legal and regulatory scrutiny by withholding information on training and fine-tuning data. We argue that openness in generative AI is necessarily composite (consisting of multiple elements) and gradient (coming in degrees), and point out the risk of relying on single features like access or licensing to declare models open or not. Evidence-based openness assessment can help foster a generative AI landscape in which models can be effectively regulated, model providers can be held accountable, scientists can scrutinise generative AI, and end users can make informed decisions.

Figure 2 (click to enlarge): Openness of 40 text generators described as open, with OpenAI’s ChatGPT (bottom) as closed reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points. RL may refer to RLHF or other forms of fine-tuning aimed at fostering instruction-following behaviour. For the latest updates see: https://opening-up-chatgpt.github.io

Figure 3 (click to enlarge): Overview of 6 text-to-image systems described as open, with OpenAI's DALL-E as a reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points.

There is also a related Nature news article: Not all ‘open source’ AI models are actually open: here’s a ranking

PDF Link: https://dl.acm.org/doi/pdf/10.1145/3630106.3659005

We are having a pumpking growing competition at work and I live in an apartment, so I'm working with what I have 😆

The plant already produced many male flowers. From what I have read, the male flowers usually come out 10 - 14 days before the female flowers. They open up for a single day and then they close and fall off.

I found out that tey are edible, so I stuffed a few of them with some left overs as a culinary experiment.

And the first female flower has arrived!

TL;DR: This paper describes the finding that there is a specific type of bacterium (Symbiodolus clandestinus) that lives inside of the tissues of several different insects. This bacterium appears to cause no disease, and it is hypothesized that it provides some useful metabolites that the insects are unable to produce themselves. The bacteria can be passed from the mother directly to her offspring. So, this appears to be a widespread symbiotic relation between a bacterium and insects.

The article goes into a lot more depth and describes some other examples of bacteria <-> insect interactions.

Here are some images of the sargassum-looking nudibranch in case you don't want to load the PDF file!

The instance has been updated to version 0.19.4!

You can see the official release post and notes here: https://lemmy.ml/post/16585359?scrollToComments=true

Since this update required some manual intervention anyway to update the database version, I took the opportunity to perform some overdue server maintenance and upgrades.

I have migrated the server to a more powerful configuration, and changed the way image files are being stored and served (object storage as opposed to filesystem storage).

I am also aware of some federation issues that have affected this and other instances. I am not sure whether this upgrade will deal with these problems automatically. I will be looking into it and hopefully will implement some solution soon.

Sorry for the downtime experienced in the past few days, but hopefully with the more powerful server the experience will be smoother and will remain so for a while.

For other admins and users interested, this is a bit of information about the server

The previous configuration:

Hosting provider: serverspace.io

8 GB RAM

4 Core CPU

50 Mbps network

400 GB SSD

Cost: $55 / month

The 400 GB SSD was being used to store both the server's database and the image files, and this component in isolation had a monthly cost of $36. I have migrated the images to an object storage provider ($6/month for 500 GB), and used the resources previously used for disk space to increase the other parameters. The new configuration is a lot better and less expensive.

New configuration:

Hosting provider: serverspace.io

Object storage provider: contabo.com

16 GB RAM

6 core CPU

50 Mbps network

100 GB SSD

500 GB Object storage

Cost: $47/month

Thanks a lot to the lemmy and pict-rs devs for the hard work they put into developing the infrastructure!

I am performing a migration that moves the images files from disk storage to object storage. During this migration the image back-end needs to be turned off, and so images will not be accessible.

It should still be possible to comment.

I am not sure how long this will take.

This is part of a few optimizations that I plan to do before I upgrade the instance to version 0.19.4.

EDIT: Update, June 17 06:08 UTC

The transfer is working, but, after 12 hours, only ~50 GB have been transferred to Contabo's object storage (rate of about 1.16 MB/s). This is a lot slower than I imagined, and, if this rate continues, it will take about 3 more days to transfer the remaining 300 GB.

I am not sure if this slow rate is to be expected for moving files to object storage, or if something is wrong. If any of you has experience with this I would appreciate some tips! I have also asked in the Matrix chat in case someone there can provide me some feedback as well.

EDIT: Update, June 18, 07:44 UTC

Migration is done!

272.16 GB of image data has been transferred to object storage. This took about 37 hours. This means that transfer occurred at a rate of 2 MB/s. From other admin's experience, this transfer was quite slow. I am making use of Contabo as my object storage provider, and I suspect that the speed of the transfer was limited on their end. Another admin reported that they used backblaze b2 as the object storage provider and they experienced much faster transfers. All of this should have no influence on how the site performs from the end-user's point of view, I am documenting this for other admins who learn from other's experiences by searching through lemmy.

Object storage is a much cheaper way to save files in the cloud. Before this, I have stored the image files into the same SSD disk where the instance lives (rented from serverspace.io). The cost of this disk space goes up as the disk space needs increase. At the current disk size of 400 GB renting that space was costing $36/month and increased $0.9/mo for every 10 GB.

Object storage is a specialized way of saving and serving files from the cloud. Object storage is much cheaper, at €6 for 500 GB (from Contabo), and so it is a much more scalable way of storing the instance's files. The only reason I did not do this earlier was because I was not sure about how to do it and kept pushing it back. But now, it's done! I would recommend other admins to do this as early as possible, as migrating many files is more of a hassle than migrating a few.

Istill need to do a few more tasks (moving the instance to a new server with a smaller disk, upgrading the postgres database to be compatible with 0.19.4, changing the pict-rs database format from sled to postgres, and then upgrading to 0.19.4). But first I will let the instance run for a few days with object storage to make sure nothing is breaking.

Thank you for your patience :D