SeeDance 2.0: потужна AI-генерація відео з використанням Runway

Heat. Hello my friends and how are you doing? Today I want to show you Se dance 2 and I'm going to show it to you on a runway. Now, the reason why I'm going to show it to you on runway is not that they paid me, because they didn't pay me, but because this is one of the rare pages where you don't need a business account to access the model and also you can use it unlimited times with their unlimited plan, which is a little bit pricey, but when you use it a lot, you actually get the value back. So you can see right now I'm in an unlimited mode, but I can generate videos. Now you can generate two videos at a time and the batch is sometimes pretty slow to pick up because you get the idle time. So it can take quite some minutes until it's getting to start. But still you can generate it without using any credits which is pretty amazing. Now let's talk about the models and also about the limitations that we have for the model. As you have seen in the intro video, the model is absolutely powerful and one thing you do need is a very detailed prompt for that. So here on the left side you can see I have a prompt and actually I'm using jet gpt to write the prompts for you. It's important to have a time code in the prompt so that the model knows what you want to do in the individual scenes and then a good description of what should happen inside of the scenes. This is pretty important. Also, this is a multi-reference model. So you can upload not only pictures but also videos. And in the future they're also going to add audio that can be used as a source which is super powerful for example for lip syncing or creating music videos. And another thing that is interesting here is with the ability to use videos here as a source. You can for example cut out the last couple of seconds from your video and then say in the prompt to extend that video from there and render another 15 seconds so that you can actually create an endless video and by that actually create a full movie. Now, what I did here is to use the last frame. And actually, this is made pretty easy by runway because you can go to anywhere in your video and you can say down here, use the frame as a input for the video or as a reference. That is pretty good. Now there's a caveat to that actually because as you have seen here with my starting image there's a person in the image that is realistic and because of copyright reasons and legal reasons seance is actually limiting that in a strong way. Now I found that for example for this shot here where the face of the person is rather small it sometimes actually works and also you don't have any problem with any image as long as it is not realistic. So if you have any cartoon character or the face is covered by a mask or it has an the person is wearing an arm or things like that it should actually work. It's just the likeness of a person that doesn't work. And also, by the way, you can generate people with the prompt. It doesn't have any problem with realistic people created through prompts. Now, that of course is a little bit limiting if you want to extend the video with real people in there. Here, I got lucky with this shot. And actually, one thing you can do is to use that shot as a character reference. And then when you explain in the scene what is going on, you see that this is actually my first scene here. So even from the prompt that didn't realize that I didn't want to have that scene, but it only took like two seconds to realize, oh, this is the next scene. And then you just cut that out and match it together. The only problem you actually have from this method of using a frame instead of a cut video is that when you do that, the sound doesn't really match up afterwards. For example, if there is music, it suddenly just starts with other music or with another sound effect if there is a storm, things like that. Right now, here are some physics simulations. I'm really surprised at how good Seance 2 is at these physics simulations. So, I'm tested out multiple things for you. Now, most of them are first render. I didn't have to rerender them 10 times to get the result. It actually worked right away. The first one is this ball inside of a transparent plastic ball. As you can see, I would say the physics are pretty good, but let me know in the comments what you think about that. Now, here the next one is a towel that is dry and then dipped into water and pulled up. As you can see, it gets wet, which is good. It gets wet a little bit on the wrong parts of it, but overall I would say also pretty good result. What do you think about that? Now, here we have mixing colors. This actually was pretty surprising because you get the right color back. The mixing of the colors looks good. It's not really mixing it as strong as I would love to be. Here's another one that was interesting and that is using a steel ball and then a table tennis ball of the same size but different colors and actually seed dense understood how to do that. But there is also limitations with seed dense that we have to talk about. One of the limitations is actually volume of liquid. So here you can see an example where I have red liquid from a flask poured into a glass and then from the glass into a second glass and that doesn't really work. And also adding ice cubes there. First of all, the number of ice cubes is often wrong if you go above the number of three. Uh, but one thing that I actually like here is that when the ice cubes hit the water, the water level actually rises. So, there is some good physics in there. I also want to point out at this point, I'm completely aware that this is only a visual representation of the physics. It's not an actual physics understanding. So, there is a huge difference, but it still is important that the model can do it. Now, here's also an experiment that was pretty interesting, but I had to have multiple renders to actually get this result. We see the inside of a spoon with the face reflection, which is correctly upside down. And then turning the spoon around, we get a wider fisheye view and the correct upside perspective of the room. So, that is actually pretty cool, even though the reflections are not the same reflections in both cases. Let's go back to some artistic stuff I tried out because there's also some things that are interesting to tell you. Now, one thing I found is that as long as you have scenes that often are in video games or in movies like this, for example, with the dragon that is breathing fire and a woman in a dress, this is working actually very well because the model knows what's going on. Also here we have a werewolf hunting a man and that is also something that you know from movies. So here the scene works well. But one thing I find interesting is that the slow motion actually looks fantastically good. While the normal video is okay, but smashing into these little stands in the background is not exactly super good. Uh there's just too much happening. Now, here we have a red hunted by a cat. And that also works well discounting the look of how these flasks are breaking because there's just too much going on. I just wanted the flasks to fall down and smash on the ground, but that didn't really happen. But again, the slow motion is absolutely beautiful. Now, last but not least, we have here a prison escape scene with a lot of different characters. And there it can get a little bit jumbled when too much is going on in a scene. When there is too many characters you try to describe through the prompt, the model kind of loses track. So a good description of the scene that is focusing on an action and focusing on one or two characters is pretty good. there can be more characters in the background and objects and things like that, but if the scene is getting too crowded, it is kind of getting problematic. And then also we have here this scene with the uh lady running towards her battle and climbing into it. And I found that often seedance can have a little bit of a problem with understanding directions. in what direction is the woman running or the mech pointing. So sometimes you have to adjust the description of that and also sometimes a character that should be in front of the battlemech is behind the battle mech. So there is a little bit of a problem with understanding there. And of course I also had to try the famous dating fruits on Love Island. That turned out I would say pretty well. pretty cool video. And also let me know if you have any other comments that you want to have answered about Cance 2. Plus, don't forget to join my Discord where we try out different models, we talk about our results, share our knowledge, great community. See me there and let's have a nice chat. Bye. Heat. Heat. Heat. Um, Good. Akum suit.