2024/08/02 13:40
The original team of Stable Diffusion is officially announced! The new model refreshed the AI painting pattern overnight and has received 230 million yuan in financing
2024/08/02 13:40
Just now, the original team of Stable Diffusion officially announced the start of a new company!
Robin Rombach, who announced his departure from Stability AI in March, is one of the two main authors of Stable Diffusion, and more than a dozen former company partners officially announced the news of forming a group to start a business.
The new company is called Black Forest Lab. As soon as it was unveiled, Kuku released a series of 3 image generation models, 2 of which were open source.
And it supports Chinese input.
The result? Netizens who have seen it called it very wild (wild)!
Enter the prompt word, and test the security measures such as picture effect and face data at one time:
A teenage girl wearing a ski mask does origami crafts in a barn. There is designated yellow text at the bottom of the image. There is a picture frame in the background with a picture of Obama inside.
Just by looking at this set of pictures and texts, some netizens sighed that this is the best image generation effect he has ever seen.
If you want to say that the characteristic of this company is that it focuses on an unabashed.
Today's official announcement of its establishment, the release of a series of models today, and the announcement of the financing progress -
It has closed a $32 million funding round led by a16z, Oculus VR co-creator Brendan Iribe, former YC partner Garry Tan, NVIDIA Research led the computer graphics research group Timo Aila, and Apple Distinguished Scientist (former chief scientist of Intel Intelligent Systems) Vladlen Koltun et al. invested.
It can be said that the Black Forest has been bet on by the capital market and favored by the industry bigwigs.
AI bull Kapathi also sent a congratulatory message online, and also praised the new model of the Black Forest by the way:
Crooked Richgood! The open-source FLUX.1 image gen model looks very powerful.
And note that the open source protocol is a loose Apache 2.0.
Kapathi felt excited, let us visually feel the effect of the model of the Black Forest.
Here, five types of generating effects are selected for qubits, and the pictures are all provided by the official, without indicating which model is used.
The first level is text generation.
Prompt word: A photo of the old classroom chalkboard. The chalkboard reads "let's make some really pretty stuff together" in chalk, with a red chalk heart after the words. Sunlight came in through the windows.
The second level, non-real scene + text generation.
Hint: In the underwater scene, two owls sit at a beautiful dining table with a candle lit in the center of the table, and the two owls are enjoying a delicious dinner together. The owl on the left is wearing a tuxedo, and the owl on the right is wearing a beautiful dress. There is a submarine passing by in the background, the words "What a Hoot" painted on its side. There are small jellyfish swimming at the bottom of the image below the table, a cinematic beautiful digital artwork.
The third level is a real-world scene.
Prompt: Photograph of a beautiful street in Freiburg, a tram passing by, people walking and cycling.
In the fourth level, real characters and anime characters are generated.
Cue: Photograph of three ladies taken on a downtown street, with their hands reaching for the camera.
Cuewords: Beautiful anime artwork, a cute cat lady who looks depressed, holds a piece of paper with a smile drawn on it, and she is about to cry.
In the fifth level, animal image generation.
Cue: A bobcat in the forest, photographed by a professional photographer in bright light.
Cue: Render a mythical creature up close, consisting of detailed spiral fractals and tendrils, detailed recursive skin textures
This time, Black Forest released a total of 3 models of the FLUX.1 series: pro, dev, and schnell.
FLUX.1 [pro]: The strongest note in the series.
The essence of the FLUX.1 series, which provides optimal performance image generation with best-in-class instruction adherence, visual quality, image detail, and output versatility.
The Black Forest team is slowly improving the inference computing capabilities of FLUX.1 [pro] in the API.
This version can be accessed via Replicate and fal.ai; Dedicated and customized enterprise solutions are available.
FLUX.1 [dev]: A cup in the series.
A model that allows for non-commercial use, open-weighted, distilled.
[dev] is distilled directly from [pro] and has similar quality and swift adherence while being more efficient than standard models of the same size.
Try it on Hug Face or directly on Replicate or fal.ai.
FLUX.1 [schnell]: Small whirlwind.
The fastest model in the series, tailored for local developers and individual developers.
FLUX.1 [schnell] is publicly available under the Apache 2.0 license, the model weights can be queried by hugging faces, and the inference code can be found on GitHub.
It has been supported by ComfyUI and can be used directly; It can also be used via Replicate or fal.ai.
Let's get a feel for it!
Here are three photos, which are the generated effects of the above large cups, medium cups, and small cups around the theme of "cake" under different prompts.
△From left to right, the models used are large, medium, and small
After several tests of qubits, it was found that the time to generate an image with the Pro version was between 15s and 25s (the generation time will be displayed below the result graph) if you enter a simple prompt word.
According to Black Forest, all FLUX.1 models are based on a hybrid architecture of multimodal and parallel diffusion Transformer blocks and are extended to 12B parameters.
Among the three models, FLUX.1 [pro] and [dev] surpassed Midjourney v6.0 and DALL · in terms of visual quality, prompt responsiveness, size/aspect ratio flexibility, typography and output diversity. E 3 (HD) and Stable Diffusion 3-Ultra.
FLUX.1 [schnell] has been described by the team as "the most advanced few-step model to date".
Not only does it stand out from its competitors in its class, but it also surpasses more powerful uncompressed models such as Midjourney v6.0 and DALL · E 3 (HD)。
The entire FLUX.1 family has been fine-tuned to preserve the full output diversity of the pre-training phase.
Compared with existing technologies, FLUX.1 also has the following advantages:
Some people inevitably want to ask, you are the OG veterans of Stability AI, and you are the core members.
So, what is the difference between your new model and the Stable Diffusion?
The founding team members responded on Reddit:
Even our weakest model, Schnell, has better build quality and faster builds.
The main focus is that I set up a new company to surpass myself
Now that you've introduced the model, it's time to get to know the new company.
The establishment of the Black Forest Laboratory was just announced today.
On the company's official website, there is a slogan: A new era of creation.
The company's mission is to advance the generation of state-of-the-art, high-quality deep learning models for images and videos and make them available to the widest possible audience.
Huadian has appeared! Their next ambition is obvious, and that is to move into the field of video generation.
Also put it down, it has to be "SOTA".
Core member Robin Rombach is a former research scientist at Stability AI.
During his time at Stability AI, he was one of the main developers of the Stable Diffusion model, and also participated in the research of SDXL, SVD and other projects.
In March of this year, Robin ran away from Stability AI.
The outside world commented that his departure was a mess of the unicorn, which was already a mess - after all, he was one of the two main forces of SD.
Looking back, Robin received his bachelor's and master's degrees in physics from the University of Heidelberg.
In 2020, he started his PhD in computer science in the computer vision group in Heidelberg under the supervision of Björn Ommer, and moved to the University of Munich in 2021 with the research group.
His research focuses on generating deep learning models, specifically text-to-image systems.
Google Scholar has nearly 15,000 citations.
In addition, Andreas Blattmann, Axel Sauer, Dominik Lorenz, Dustin Podel, Frederic Boesel, Patrick Esser, Sumith Kulal, Tim Dockhorn, Yam Levi, and Zion English are all publicly available Stability members Former member of AI.
(Andi Holmes and Jonas Müller have not yet found accurate information)
It can be said that the Black Forest is the core members of the original SD team who left and set sail again.
No wonder Axel Sauer retweeted the official tweet and shouted:
We're alive!
Coincidentally, on the same day, Stability AI also had a new action:
Launched a new AI model, Stable Fast 3D, which officially says it can generate 3D images in half a second.
The previous model took minutes to produce a 3D image of a similar effect, but the new model can do the same thing 1,200 times faster than the existing one.
What is Stability AI's CEO of Stability AI running away in March, Emad Mostaque, doing?
In June, he announced his future to his new company, Schelling AI, which "will build and support open-source code, models, and datasets backed by AI funding."
The focus is on innovative research and the careful construction of culturally aware, scientific, educational, and creative AI.
Three days ago, Schelling AI released the first article in a series, "How To Think About AI."
The article is a bit long, interested friends can search and view it for themselves, here is the core idea -
AI is developing rapidly, advocating open source and openness to accelerate innovation and collaboration.
And, if not everyone is a decent person!
The tweet announcing the establishment of the Black Forest Lab was retweeted by the former CEO of the family (put a dog head here).
Reference links:
[1]https://blackforestlabs.ai
[2]https://news.ycombinator.com/item?id=41130620
[3]. ]https://x.com/EMostaque
[4]https://www.reddit.com/r/StableDiffusion/comments/1eds0l9/does_anyone_have_an_update_on_when_stable/
[5]https://x.com/SchellingAI/status/1818600200232927721
This article is from Xinzhi self-media and does not represent the views and positions of Business Xinzhi.If there is any suspicion of infringement, please contact the administrator of the Business News Platform.Contact: system@shangyexinzhi.com