Video Transform Endpoints

Also incorporated into the ComfyUI Agent is FFmpeg, which can be used to process videos. AI Server wraps some common operations into easier-to-use endpoints, such as:

  • Crop Video - Crop a video to a specific size
  • Convert Video - Convert a video to a different format
  • Scale Video - Scale a video to a different resolution
  • Watermark Video - Add a watermark to a video
  • Trim Video - Trim a video to a specific length

Using Video Endpoints​

These endpoints are used in a similar way to the other AI Server endpoints, e.g., you can provide a RefId and Tag to help categorize the request, and for Queue requests, you can provide a ReplyTo URL to send a POST request to when the request is complete.

Crop Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new CropVideo {
        X = 100,
        Y = 100,
        Width = 500,
        Height = 300
    },
    new UploadFile("test_video.mp4", fsVideo, "video"));

File.WriteAllBytes(saveToPath, response.Results[0].Url.GetBytesFromUrl());

Queue Crop Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new QueueCropVideo {
        X = 100,
        Y = 100,
        Width = 500,
        Height = 300
    },
    new UploadFile("test_video.mp4", fsVideo, "video")
);

GetArtifactGenerationStatusResponse status = new();
while (status.JobState is BackgroundJobState.Started or BackgroundJobState.Queued)
{
    status = await client.GetAsync(new GetArtifactGenerationStatus { RefId = response.RefId });
    await Task.Delay(1000);
}

// Download the cropped video
File.WriteAllBytes(saveToPath, status.Results[0].Url.GetBytesFromUrl());

Convert Video​

using var fsVideo = File.OpenRead("files/test_video.webm");
var response = client.PostFileWithRequest(new ConvertVideo {
        OutputFormat = ConvertVideoOutputFormat.MOV
    },
    new UploadFile("test_video.webm", fsVideo, "video"));

File.WriteAllBytes(saveToPath, response.Results[0].Url.GetBytesFromUrl());

Queue Convert Video​

var response = client.PostFilesWithRequest(new QueueConvertVideo {
        Format = "mp4",
        ReplyTo = "https://example.com/my/reply/endpoint"
    },
    [new UploadFile("video", File.OpenRead("video.avi"), "video.avi")]
);

var status = await client.GetAsync(new GetJobStatus { RefId = response.RefId });
var timeout = DateTime.UtcNow.AddMinutes(5);
while (status.JobState is not BackgroundJobState.Completed &&
       DateTime.UtcNow < timeout)
{
    await Task.Delay(1000);
    status = await client.GetAsync(new GetJobStatus { RefId = response.RefId });
}

// Download the converted video
var videoUrl = status.Outputs[0].Url;
videoUrl.DownloadFileTo("converted-video.mp4");

Scale Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new ScaleVideo {
        Width = 1280,
        Height = 720,
    },
    new UploadFile("test_video.mp4", fsVideo, "video"));

File.WriteAllBytes(saveToPath, response.Results[0].Url.GetBytesFromUrl());

Queue Scale Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new QueueScaleVideo {
        Width = 1280,
        Height = 720,
        ReplyTo = "https://example.com/my/reply/endpoint" // optional
    },
    new UploadFile("test_video.mp4", fsVideo, "video"));

GetArtifactGenerationStatusResponse status = new();
while (status.JobState is BackgroundJobState.Started or BackgroundJobState.Queued)
{
    status = await client.GetAsync(new GetArtifactGenerationStatus { RefId = response.RefId });
    await Task.Delay(1000);
}

// Download the scaled video
File.WriteAllBytes(saveToPath, status.Results[0].Url.GetBytesFromUrl());

Watermark Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
using var fsWatermark = File.OpenRead("files/watermark_image.png");
var response = client.PostFilesWithRequest(new WatermarkVideo {
        Position = WatermarkPosition.BottomRight
    },
    [
        new UploadFile("test_video.mp4", fsVideo, "video"),
        new UploadFile("watermark_image.png", fsWatermark, "watermark")
    ]);

File.WriteAllBytes(saveToPath, response.Results[0].Url.GetBytesFromUrl());

Queue Watermark Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
using var fsWatermark = File.OpenRead("files/watermark_image.png");
var response = client.PostFilesWithRequest(new QueueWatermarkVideo {
        Position = WatermarkPosition.BottomRight
    }, [
        new UploadFile("test_video.mp4", fsVideo, "video"),
        new UploadFile("watermark_image.png", fsWatermark, "watermark")
    ]);

GetArtifactGenerationStatusResponse status = new();
while (status.JobState is BackgroundJobState.Started or BackgroundJobState.Queued)
{
    status = await client.GetAsync(new GetArtifactGenerationStatus { RefId = response.RefId });
    await Task.Delay(1000);
}

// Download the watermarked video
File.WriteAllBytes(saveToPath, status.Results[0].Url.GetBytesFromUrl());

Trim Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new TrimVideo {
        StartTime = "00:05",
        EndTime = "00:10"
    },
    new UploadFile("test_video.mp4", fsVideo, "video"));

File.WriteAllBytes(saveToPath, response.Results[0].Url.GetBytesFromUrl());

Queue Trim Video​

using var fsVideo = File.OpenRead("files/test_video.mp4");
var response = client.PostFileWithRequest(new QueueTrimVideo {
        StartTime = "00:05",
        EndTime = "00:10"
    },
    new UploadFile("test_video.mp4", fsVideo, "video"));

GetArtifactGenerationStatusResponse status = new();
while (status.JobState is BackgroundJobState.Started or BackgroundJobState.Queued)
{
    status = await client.GetAsync(new GetArtifactGenerationStatus { RefId = response.RefId });
    await Task.Delay(1000);
}

// Download the trimmed video
File.WriteAllBytes(saveToPath, status.Results[0].Url.GetBytesFromUrl());

INFO

These operations depend on support from FFmpeg, which comes installed with the ComfyUI Agent. Limitations on encoders, decoders, and filters may apply.

Edit this page on GitHub