Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

FFmpeg for Unity

Discussion in 'Assets and Asset Store' started by Max-Bot, Jul 13, 2017.

  1. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi guys!

    For everybody struggling for video editor functionality on mobile.
    Here you go: FFmpeg is integrated into Unity Editor and is reachable from C# code.
    Platforms: Windows, Mac OS, IOS, Android.

    Available at GitHub:
    https://github.com/Spirit30/FFmpegUnityBind2

    What you can do:
    - Convert formats
    - Video to Audio
    - Create video from images and sounds
    - Extract images and sounds from video
    - Record video with audio

    Also there is available powerful console for direct control of any ffmpeg features:
    https://ffmpeg.org

    There is a detailed documentation included into the package, check it out.
    Cheers!
     

    Attached Files:

    Last edited: Feb 24, 2023
    deus0 and Archviz3d like this.
  2. HelloMeow

    HelloMeow

    Joined:
    May 11, 2014
    Posts:
    280
    Isn't FFMPEG licensed under the GPL/LGPL? I'm not sure that 3rd party libraries that are licensed under those licenses are allowed on the asset store.

    See https://unity3d.com/legal/as_provider section 5.10.4
     
    deus0 and CalvinJMKim like this.
  3. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    The product is not a library itself. The product is an integration it to Unity3d.
    You can make it by yourself or buy with a price of one working hour.
    Cheers
     
  4. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    28
    hey,

    can i use it to merge a mp4 file with wav/mp3 audio?
    Also, how long would it takes you to add iOS support? I need it for this week.. :)
     
    Max-Bot likes this.
  5. Archviz3d

    Archviz3d

    Joined:
    Apr 4, 2016
    Posts:
    92
    Hi!! Great that someone finally implemented it! Although for my needs right now i just needed to know if its possible to merge a mp4 video file (example videoplayback.mp4) with a m4a audio file (exampple videoplayback.m4a) and giving a new file called for example output.mp4.

    I think its possible using ffmpeg using this command "ffmpeg -i videoplayback.mp4 -i videoplayback.m4a -c:v copy -c:a copy output.mp4", would it be possible in unity with your asset (for android)?

    If so, id be really interested to have it for android but also for ios!!!

    Thanks
     
    Last edited: Sep 20, 2017
    Max-Bot likes this.
  6. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi guys.

    Thank you for join our video processing community.

    Try to achieve what you are trying to do with this native Android solution.
    APK with direct command line:
    https://github.com/writingminds/ffmpeg-android-java/releases/download/v0.3.2/app-debug.apk
    If it works - It will work in our FFmpeg Unity Bind from C# code as well.

    I believe IOS version will be developed some day.
    Keep in touch for updates.

    Cheers.

    P.S. Don't hesitate to contact our team for development services when you need to extend this product functionality.
    http://giganeo.com/contact-us
     
    Archviz3d likes this.
  7. Archviz3d

    Archviz3d

    Joined:
    Apr 4, 2016
    Posts:
    92
    Yeahhhh!!!! It worked!!! So i can call the command by code in C# with your plugin? Do i have any instructions on how to do it after i buy it?

    Basically i called this command:

    -i videoFileAbsolutePath -i audioFileAbsolutePath -c:v copy -c:a aac -map 0:v:0 -map 1:a:0-shortest destinationAbsolutePath

    Thanks :)
     
    Last edited: Sep 21, 2017
    Max-Bot likes this.
  8. Archviz3d

    Archviz3d

    Joined:
    Apr 4, 2016
    Posts:
    92
    Just bought it!!! And it works!! Awesome!!! Just what i needed!! I was even able to merge watermark in the video with it!!! So cool!!!

    So now, we are just needing it for ios ;-) !!!! Please please please :)
     
    Max-Bot likes this.
  9. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi All

    I'm glad to anounce you a global upgrade of FFmpeg Unity Bind:

    1. Full IOS Support
    2. New Presets:
    - Compress
    - Append / Concat Fast (For same codec)
    - Append / Concat Full (Re-encoding)
    3. Progress Bar Feature
    4. Processing callbacks on correct thread
    5. Debug, Improvements, Optimization

    FFmpeg is a 17 years of Video Processing experience. You'll find everything what you need with this package.
    Have a fun in development.
    Good luck!

    P.S. Don't hesitate to contact our team for development services when you need to extend this product functionality.
    http://giganeo.com/contact-us
     
    HaimBendanan likes this.
  10. HaimBendanan

    HaimBendanan

    Joined:
    May 10, 2016
    Posts:
    28


    So good to suddenly find out that you added iOS support :)
    I will definitely buy your asset now. Btw is it possible to merge a video file and a audio file from a specific point? Like taking all the video and only a portion of the audio?
    Thanks!!!
     
  11. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    I'm glad that you enjoy the product.

    Here is how you can append sound to video in usual cases:
    -y -i video.mp4 -i sound.wav -map 0:v:0 -map 1:a:0 result.mp4

    Here is how you can cut the audio file:
    -ss 0 -t 30 -i file.mp3 output.mp3

    It is a little bit more challenging to make a start offset when merging it with video. Try like this:
    -y -i a.mp4 -itsoffset 00:00:30 sng.m4a -map 0:0 -map 1:0 -c:v copy -preset ultrafast -async 1 out.mp4

    Cheers!
     
    HaimBendanan likes this.
  12. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Dear Developers

    I'm glad to announce our F.A.Q. section where you can learn cases which other teams met.

    It will be updated and boosted, by all our video programmers community. I hope you'll find it useful. To become a part of video development community you just have to post reply to F.A.Q comments or this thread when people ask.

    Good luck and let The Force will be with you.
     
  13. backwheelbates

    backwheelbates

    Joined:
    Jan 14, 2014
    Posts:
    231
    This looks great. For converting image sequences to video, does anyone know if unity's Texture format can be used? Or will everything first need to be converted to a file first?
     
  14. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hey Eric_Bates
    Usual workflow is when you first save to the file and then work with an absolute path.
    If we find opportunity to pass files into FFmpeg lib in another way, we publish update or information how to do it within Unity.
     
  15. backwheelbates

    backwheelbates

    Joined:
    Jan 14, 2014
    Posts:
    231
    Hi @Max-Bot, thanks for letting me know!
     
  16. kushG

    kushG

    Joined:
    May 25, 2017
    Posts:
    22
    I was wondering if there's a way where I can append images at the end of video. I'm capturing an image every 10th frame and approximately after 10 seconds I want all the images captured encoded into video. I tried the traditional way of saving all the captured images and then running the ` //-y -i .../image%1d.jpg -r 30 video.mp4` ffmpeg command but this takes like 15-20 seconds on Android, not tested iOS yet. Wondering if there's a way where I can encode these images on the fly to be part of one single video output ? I've been googling for the last couple of days but none of the solution I've seen so far has worked for me. Also, in all of this, there is no audio involved at all.

    Any help/pointers would be much appreciated.

    @Max-Bot
     
    Last edited: Dec 14, 2017
  17. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
  18. peterahou

    peterahou

    Joined:
    May 19, 2015
    Posts:
    53
    Hi

    Does your implementation of FFmpeg include any screen recording functionality? I found this page in the wiki describing how to do it for desktop, but is it possible on iOS and Android?
    https://trac.ffmpeg.org/wiki/Capture/Desktop
     
  19. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Dear Video Developers

    I'm glad to present to you an upgrade of FFmpeg Unity Bind - version 3.1
    Now it supports Mac OS and Windows.
    That means you can test video operations right in the Editor before deploy to mobile!

    NOTE: Some things may differs on standalone vs mobile due to very different hardware. Always do final testing on end destination device.

    Also we've included SCREEN CAPTURING DEMO.
    It is absolutely cross-platform and easy to use.

    We've added extra UI presets / Examples:
    - Adding sound to existing video
    - Adding your own watermark with scale and position parameterization.

    Good luck with your projects.
    Cheers!
     
  20. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
  21. ikarus1102

    ikarus1102

    Joined:
    Jul 22, 2015
    Posts:
    2
    Hi,

    Does this plugin has streaming features? Can I stream video from device's camera to server?
     
  22. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi

    We didn't prepare a demo for that like for other functions, but FFmpeg lib which is the core of the package have relevant streaming page and you can test does it works as should with preview apk.

    Another option can be stream to the server frame by frame and when it's done - encode video with FFmpeg.

    And a main advantage of FFmpeg Unity Bind is that your mobile device can do locally same video editing operations as server (saving traffic and time).

    Cheers,
    Good luck with a project
     
  23. ikarus1102

    ikarus1102

    Joined:
    Jul 22, 2015
    Posts:
    2
    Thank you for your response,

    I'll give a try
     
  24. Pauleta

    Pauleta

    Joined:
    Aug 8, 2013
    Posts:
    20
    Hello, i just bought your package ( pretty cool ) but i am having a problem running on Windows 8.1: Permission denied when trying to encode a mp4 video from image files. Am i doing something wrong ?
     
  25. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,935
    Hi
    Can we record Windows/Mac Desktop (not the gameview) with it?
     
  26. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi @Pauleta

    I'm glad to hear from you.
    Make sure you have all rights to use ffmpeg binary file in \Assets\FFmpeg\Standalone\Win directory.

    You can even replace it with new one from HERE (choose Release Windows-64bit Static) and rename as original without extension.

    Cheers!
     
  27. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi @JohnGate

    I'm glad to read from you.
    Yes. You can check out Windows and Mac demo versions.
    Enjoy.

    Cheers!
     
  28. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,935
    Sorry for misunerstanding, but this demo records game view (that ball jumping) i want to record my desktop like if im talking on skype it should record whole desktop including skype. is this possible?
     
  29. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi @JohnGate
    Currently FFmpeg Unity Bind can record from Camera view.
    If you can make it see desktop and other applications, it'll record it.

    It's an interesting task. Good luck!
     
  30. LostPanda

    LostPanda

    Joined:
    Apr 5, 2013
    Posts:
    173
    @Max-Bot FFmpeg Unity Bind Can play an image sequence directly as moving images, without having to create a video file first?thanks.
     
  31. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    @LostPanda No, it should encode images to video format first. You can iterate image files with C# code.
     
  32. fssdev

    fssdev

    Joined:
    Jan 16, 2017
    Posts:
    50
    We've been trying to build to iOS, but with the ffmpeg plugin integrated we get build errors. Please see the attached image.
    If we remove ffmpeg from the project, everything builds just fine.

    getting the exact same error on a mac.
    Line 187 in UnityInterface.h
    Unknown type name 'bool'
     

    Attached Files:

    Last edited: Mar 29, 2018
  33. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
  34. fssdev

    fssdev

    Joined:
    Jan 16, 2017
    Posts:
    50
    Has anyone been able to successfully use a Trimmed video? The trim process completes, but the file is corrupted and unusable.
     
  35. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi fssdev

    It should be fine in general cases. With re-encoding video (Decode / Encode Example) you can have perfect accuracy selecting only frames which should be included into resulting video clip.

    Good luck with a project!
     
  36. ahmetardal

    ahmetardal

    Joined:
    Mar 15, 2018
    Posts:
    3
    Hi.
    Watermarking on Android is extremely slow. Is there any solution to make it faster?
    I'm testing on a Galaxy S7 device.
    It takes ~3 minutes to watermark a 5 seconds long video.

    fps=0.4 drop=6 speed=0.0336x

    Thanks!
     
  37. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi Ahmetardal

    FFmpeg library inserts watermark into each key frame of video container. The time mostly depends on resolution of video, it's duration and processor of the device.

    Usually we have much better benchmarks even on slower devices.

    Make sure you video is not too heavy. Otherwise try to play with parameters like described here:
    https://gist.github.com/bennylope/d5d6029fb63648582fed2367ae23cfd6

    And here:
    http://ksloan.net/watermarking-videos-from-the-command-line-using-ffmpeg-filters/

    Also everybody from this thread are welcomed to share own best practices regarding to watermarks.

    Good luck with a project and have a happy coding!
     
    ahmetardal likes this.
  38. fssdev

    fssdev

    Joined:
    Jan 16, 2017
    Posts:
    50
    Has anyone had issues with Trim in iOS? Works find on android, but iOS shuts down the app wheneven I run Trim twice. The first video Trim will run successfully, then on the start of the second, the app crashes.

    Error from console:
    Code (CSharp):
    1. Started
    2. ffmpeg version 3.3.git Copyright (c) 2000-2015 the FFmpeg developers
    3.   built with Apple LLVM version 7.0.0 (clang-700.1.76)
    4.   configuration:
    5.   avutil      configuration: --target-os=darwin --arch=arm64 --cc='xcrun -sdk iphoneos clang' --as='gas-preprocessor.pl -arch aarch64 -- xcrun -sdk iphoneos clang' --enable-cross-compile --disable-debug --disable-programs --disable-doc --enable-pic --enable-gpl --enable-libx264 --extra-cflags='-arch arm64 -mios-version-min=6.0 -fembed-bitcode -I/Users/maximbotvinev/x264-iOS/include' --extra-ldflags='-arch arm64 -mios-version-min=6.0 -fembed-bitcode -L/Users/maximbotvinev/x264-iOS/lib' --prefix=/Users/maximbotvinev/thin/arm64
    6.  
    7. System.Collections.Generic.IEqualityComparer`1:Equals(T, T)
    8. System.Collections.Generic.IEqualityComparer`1:Equals(T, T)
    9. System.Collections.Generic.IEqualityComparer`1:Equals(T, T)
    10. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Export/Debug.bindings.h Line: 43)
    11.  
    12. GameMobile(1764,0x16f493000) malloc: *** error for object 0x14e925a00: pointer being freed was not allocated
    13. *** set a breakpoint in malloc_error_break to debug
    14. SIGABRT received (abort() call) -- WHOA! I did something nasty. Dumping the stack trace:
    15.     0   GameMobile                           0x000000010513cab0 __sig_handler + 180
    16.     1   libsystem_platform.dylib            0x0000000184028b48 _sigtramp + 36
    17.     2   libsystem_pthread.dylib             0x000000018402e288 <redacted> + 376
    18.     3   libsystem_c.dylib                   0x0000000183dfbd0c abort + 140
    19.     4   libsystem_malloc.dylib              0x0000000183ec5838 <redacted> + 0
    20.     5   GameMobile                           0x0000000103c35dd0 main_callback + 48
    21.     6   GameMobile                           0x0000000103c38914 log_callback_help + 112
    22.     7   GameMobile                           0x0000000105121f9c av_log + 104
    23.     8   libsystem_pthread.dylib             0x000000018402d110 <redacted> + 0
    24.     9   libsystem_pthread.dylib             0x000000018402bb10 thread_start + 4
     
  39. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Looks like community is silent to your problem, Van Ferguson.

    Perhaps you have to start new process in this case when previous process is finished.
    Also you can debug and check in one of C files what's actually going on in your usage case.
    Take a look to call of free function which may rise this exception.

    Good luck with a project!
     
  40. fssdev

    fssdev

    Joined:
    Jan 16, 2017
    Posts:
    50
    Your thread handling seems to be in your libraries. Can I get the source code, so I can shut down the processes and start them up again?
     
  41. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi Van Ferguson

    We don't have any personal thread in your application.
    And you are wrong regarding to this.
    All IOS source code is included into the plugin.

    You are welcome (for all previous help from our side).

    Which exactly library are you interested in?
     
  42. fssdev

    fssdev

    Joined:
    Jan 16, 2017
    Posts:
    50
    I see what you are saying. I fired up xcode to dig deeper into the debugging.
    The problem is Line 38 of ffmpeg_wrapper.c:
    Code (CSharp):
    1. free((char *)log_buffer);
    Everything works fine if I add this line below it.
    Code (CSharp):
    1. log_buffer = NULL;
    I'm not sure if this is correct way to go about it, but works. What do you think?

    Thanks
     
    Last edited: May 30, 2018
    NAKAS likes this.
  43. chanyut

    chanyut

    Joined:
    Jul 10, 2013
    Posts:
    3
    Hi,

    My project requirement is to record video from UnityWebcamTexture, save and then upload the video file.

    our approach is
    1. capture each frame from WebcamTexture with method .GetPixels32(...) and Encode it to JPG and save it file by file.
    2. assume that user record for 60 seconds. with framerate = 30 fps, so we get total 1800 image files.
    3. execute ffmpeg with command below

    Code (CSharp):
    1. -f image2 -framerate 30 -i .../frames/frame-%06d.jpg -r 30 -vcodec libx264 -crf 44 -b:v 1k -preset ultrafast -pix_fmt yuv420p .../video.mp4
    4. it takes around 30-40 seconds to complete encoding process.

    Could anyone please recommend the better way for doing this? it would be better if it can finish encoding just 5-10 seconds after user stop recording.
     
  44. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    For me it looks like a proper way of doing that in general.

    You can improve that with small steps like multithreaded encoding of frame groups into video elements which can be appended fast (without re-encoding) in the end.

    Good luck with a project!
     
  45. davidfio

    davidfio

    Joined:
    Oct 16, 2015
    Posts:
    6
    Hi, i'm using FFmpeg for Unity for an Android app. I pick a video from gallery and i need to trim every X seconds. What is the command for that?
    Thanks
     
  46. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi davidfio

    You can use FFmpegCommand.Trim method in the loop starting it every time when OnFinish method of previous operation is called. Also you can do the same with FFmpegCommand.DirectInput method for more flexibility.

    For custom command implementation take a look here:
    https://unix.stackexchange.com/ques...peg-to-split-mpeg-video-into-10-minute-chunks

    To organize commands loop for every sliced part of video you have to know video file duration.
    You can do it parsing first occurrence of Duration info, here is how it looks in the received log:

    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ...

    Metadata:

    ...

    Duration: 00:00:19.26, start: 0.000000, bitrate: 2158 kb/s

    Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 720x480, 1989 kb/s, SAR 1:1 DAR 3:2, 28.66 fps, 29.50 tbr, 22500 tbn, 45k tbc

    As you can see many more info about video file is available for parsing from the log in OnProgress method.
    Take a look how it is used in Demo scene for implementation of Progress Bar.

    Good luck with a project!
     
  47. davidfio

    davidfio

    Joined:
    Oct 16, 2015
    Posts:
    6
    I've resolved with this command:

    Code (CSharp):
    1. "-i input.mp4 -threads 3 -vcodec copy -f segment -segment_time 15 output%04d.mp4"
    It does a cut loop, but isn't accurate because cuts only on the keyframe.

    Last question for you guys: How can i refresh the entire Camera folder of my android from Unity? I've found some code for refresh but only if i pass the exact name of my new file. I'd like to refresh the entire folder instead.

    Can you help me?
     
  48. Max-Bot

    Max-Bot

    Joined:
    Sep 25, 2013
    Posts:
    83
    Hi davidfio

    The recording cache if you mean this is going to:
    string cashDir = Path.Combine(Application.temporaryCachePath, "RecordingCash");
    (line 55 in FFmpegREC.cs)

    So you can refresh it by complete deleting / re-creation of RecordingCash folder at every moment.

    Cheers!
     
  49. jahanzaibf8

    jahanzaibf8

    Joined:
    Oct 17, 2017
    Posts:
    2
    Any one know either FFmpeg is capable of concatenation two videos of same file type i.e .mp4
     
  50. kateryna_sloboda

    kateryna_sloboda

    Joined:
    Jul 23, 2018
    Posts:
    1
    Hi jahanzaibf8
    The answer is Yes.

    Your case is described in the 2nd point here:
    https://stackoverflow.com/a/11175851/3575956
    In FFmpeg Unity Bind plugin it is implemented in AppendFast Example.

    You can also test it with the preview app from the point #2 from F.A.Q.


    Good luck with a project!