(CivitFowl) (2024)

Image description: A dense, foggy forest bathed in an ethereal light emanating from the mysterious foliage on the ground and the spaces between the trees. The light diffuses through the fog and branches, casting an otherworldly atmosphere over the scene.

(ChameleonAI) (2024)

Image description: A young woman with silver hair and turquoise eyes standing on a paved path in a park setting. She is dressed in a blue cropped hoodie and black pants.

(Raiern) (2024)

Image description: Two children posing for a photo in front of a building. The boy in the foreground is dressed in a cream-colored cape with fluffy white trim and a brown hat. The girl behind him, with bright red hair, is wearing a blue jacket with the red straps of a bag over her shoulder, is resting her hand on the boy's shoulder.

(aramintastudio) (2024)

Image description: A figure stands before a sunset framed by a massive ancient archway situated on rocky, mountainous terrain. A path through the rusty earth leading up to the portal winds between the rugged rocks in the foreground. Above the archway, a faint circle of light intersects dramatically with a vertical beam that pierces the sky.

(jorkington) (2024)

Image description: A girl with short, light-colored hair and green eyes, casually dressed in a yellow t-shirt and white pants. She is seated on a red couch with blue cushions, leaning back slightly in a relaxed posture, and offering a friendly smile.

(ruanyi) (2024)

Image description: An overgrown, flooded city street, flanked by towering buildings that disappear into the misty sky. The foreground is a striking contrast of urban decay, with verdant plants thriving on the balconies of weathered structures. A car navigates the wet road, its glossy surface mirroring the city.

(h_madoka) (2024)

Image description: A young woman with light brown hair is seated on a grassy field. She is dressed in a green and white school uniform with a black bow and leggings. The backdrop of trees under a clear blue sky adds to the idyllic and peaceful atmosphere of the scene.

(AIDigitalMediaAgency) (2024)

Image description: A towering mountain with sharp, snow-covered peaks piercing through a tumultuous sky filled with dark, swirling clouds. Powerful lightning bolts cuts through the scene, adding a sense of raw energy and natural power to the already intense atmosphere of the setting.

Quoted from Reddit:

Hello r/StableDiffusion --

A sincere thanks to the overwhelming engagement and insightful discussions following our announcement yesterday of the Open Model Initiative. If you missed it, check it out here.

We know there are a lot of questions, and some healthy skepticism about the task ahead. We'll share more details as plans are formalized -- We're taking things step by step, seeing who's committed to participating over the long haul, and charting the course forwards. 

That all said - With as much community and financial/compute support as is being offered, I have no hesitation that we have the fuel needed to get where we all aim for this to take us. We just need to align and coordinate the work to execute on that vision.

We also wanted to officially announce and welcome some folks to the initiative, who will support with their expertise on model finetuning, datasets, and model training:

• AstraliteHeart, founder of PurpleSmartAI and creator of the very popular PonyXL models
• Some of the best model finetuners including Robbert "Zavy" van Keppel and Zovya
• Simo Ryu, u/cloneofsimo, a well-known contributor to Open Source AI 
• Austin, u/AutoMeta, Founder of Alignment Lab AI
• Vladmandic & SD.Next
• And over 100 other community volunteers, ML researchers, and creators who have submitted their request to support the project

Due to voiced community concern, we’ve discussed with LAION and agreed to remove them from formal participation with the initiative at their request. Based on conversations occurring within the community we’re confident that we’ll be able to effectively curate the datasets needed to support our work. 

Frequently Asked Questions (FAQs) for the Open Model Initiative

We’ve compiled a FAQ to address some of the questions that were coming up over the past 24 hours.

How will the initiative ensure the models are competitive with proprietary ones?

We are committed to developing models that are not only open but also competitive in terms of capability and performance. This includes leveraging cutting-edge technology, pooling resources and expertise from leading organizations, and continuous community feedback to improve the models. 

The community is passionate. We have many AI researchers who have reached out in the last 24 hours who believe in the mission, and who are willing and eager to make this a reality. In the past year, open-source innovation has driven the majority of interesting capabilities in this space.

We’ve got this.

What does ethical really mean? 

We recognize that there’s a healthy sense of skepticism any time words like “Safety” “Ethics” or “Responsibility” are used in relation to AI. 

With respect to the model that the OMI will aim to train, the intent is to provide a capable base model that is not pre-trained with the following capabilities:

• Recognition of unconsented artist names, in such a way that their body of work is singularly referenceable in prompts
• Generating the likeness of unconsented individuals
• The production of AI Generated Child Sexual Abuse Material (CSAM).

There may be those in the community who chafe at the above restrictions being imposed on the model. It is our stance that these are capabilities that don’t belong in a base foundation model designed to serve everyone.

The model will be designed and optimized for fine-tuning, and individuals can make personal values decisions (as well as take the responsibility) for any training built into that foundation. We will also explore tooling that helps creators reference styles without the use of artist names.

Okay, but what exactly do the next 3 months look like? What are the steps to get from today to a usable/testable model?

We have 100+ volunteers we need to coordinate and organize into productive participants of the effort. While this will be a community effort, it will need some organizational hierarchy in order to operate effectively - With our core group growing, we will decide on a governance structure, as well as engage the various partners who have offered support for access to compute and infrastructure. 

We’ll make some decisions on architecture (Comfy is inclined to leverage a better designed SD3), and then begin curating datasets with community assistance.

What is the anticipated cost of developing these models, and how will the initiative manage funding? 

The cost of model development can vary, but it mostly boils down to the time of participants and compute/infrastructure. Each of the initial initiative members have business models that support actively pursuing open research, and in addition the OMI has already received verbal support from multiple compute providers for the initiative. We will formalize those into agreements once we better define the compute needs of the project.

This gives us confidence we can achieve what is needed with the supplemental support of the community volunteers who have offered to support data preparation, research, and development. 

Will the initiative create limitations on the models' abilities, especially concerning NSFW content? 

It is not our intent to make the model incapable of NSFW material. “Safety” as we’ve defined it above, is not restricting NSFW outputs. Our approach is to provide a model that is capable of understanding and generating a broad range of content. 

We plan to curate datasets that avoid any depictions/representations of children, as a general rule, in order to avoid the potential for AIG CSAM/CSEM.

What license will the model and model weights have?

TBD, but we’ve mostly settled between an MIT or Apache 2 license.

What measures are in place to ensure transparency in the initiative’s operations?

We plan to regularly update the community on our progress, challenges, and changes through the official Discord channel. As we evolve, we’ll evaluate other communication channels.

Looking Forward

We don’t want to inundate this subreddit so we’ll make sure to only update here when there are milestone updates. In the meantime, you can join our Discord for more regular updates.

If you're interested in being a part of a working group or advisory circle, or a corporate partner looking to support open model development, please complete this form and include a bit about your experience with open-source and AI. 

Thank you for your support and enthusiasm!

Sincerely, 

The Open Model Initiative Team

(Raiern) (2024)

Image description: Two young men posing for a picture in a cityscape. The character in the foreground has bright red hair and is smiling, the other character who has green hair and glasses, and is also smiling has his arm wrapped around the character, leaning on his shoulder. They are both dressed casually; the red-haired character wears a white hoodie, while the green-haired character wears a white shirt over a teal t-shirt.

(Harbringer) (2024)

Image description: A cloaked figure standing with their back to the viewer, gazing at a solar eclipse. The sky is a dramatic blend of dark clouds and crimson hues, and the landscape features mountain silhouettes and scattered fires.