Recording High Quality Flash Video Over Slow Internet Connections Part 3 (Behaviour of client side buffer)
This post is part 3 of our 3 part series on recording high quality Flash video over slow connections.
In theory whenever someone is trying to use a flash video recorder to:
- record audio/video at a data rate higher than the upload connection to the media server (Red5, FMS, Wowza, etc..) allows,
- record audio/video at a data rate higher than about 1Mbit/s (which seems to be the upload/streaming limit in Flash Player)
Flash Player should buffer both the audio and video data until it can be properly sent to the video server, but what really happens is it starts buffering just the video data and keeps sending the audio data to the media server.
It does that because Flash Player thinks all streams are live (even those specifically destined for being recorded) so it tries to at least keep the audio “live” if its not possible to keep both audio+video “live”.
What happens now is that the video frames will arrive at the media server:
- later than the corresponding audio frame
- arrive after the media server has already written the corresponding audio frames to the .flv file
- never arrive, because Flash Player has flushed the client buffer that was holding the video data
This can cause several issues in the final .flv file:
- missing or little video data across some sections
- sudden increases in video frame rates (the video plays in a “fast forward” way because several video frames finally reach the server as a bundle)
- flv files with static video, normal audio playback and the rest of the video data grouped at the end
- flv files with large areas where you can not seek (because there is only audio data in those areas)
- out of sync audio and video
To solve this issue, some media servers are now waiting longer for the video data to come from the client (flash video recording software) before writing the existing audio data to the .flv file:
- Wowza Media Server and Flash Media Interactive Server 3.5 both solve this issue out of the box.
- Flash Media Interactive Server 3.0 can also solve this issue by adding some custom/hidden XML tags to Server.xml.
- Red5 up to 0.9.1 has this issue and a bug has been submitted which you can follow here here.
- Red5 1.0 attempts to fix this issue to by exposing a save data to flv treshold variable, more info on that here.
This concludes our 3 parts (part 1, part 2) series on recording high quality audio and video files over slow connections.
Next I think we will talk about increasing the audio and video quality for the average user!
28 April update: updated with latest available information on the issue and link to Red5 1.0 RC1 fix.
April 2nd, 2009 at 15:48
[...] In part 3 we will talk about getting the media server to properly save the video and audio data to the …. [...]
June 19th, 2009 at 0:24
Do you know if FMIS 3.0 has this functionality or does it only exist in 3.5?
“To solve this issue, media servers are now waiting longer for the video data to come from the client (flash video recording software) before writing the existing audio data to the .flv file.”
Thanks
June 19th, 2009 at 1:03
FMIS3 does not have this functionality out of the box.
FMIS 3.5 has a new configuration tag in Server.xml named AllowedVideoLag which controls “the number of seconds to hold audio messages in the rec buffer while waiting for a video message.”
Adobe engineers tough claim that you can add this tag to the Server.xml file from FMIS3 and it will have the same effect!
June 22nd, 2009 at 18:30
Thanks for the articles. They have been very helpful.
July 10th, 2009 at 0:48
Are there particular FMIS config settings you suggest to use when recording video? I have implemented the methods you suggested in your 3 part series, which were very helpful, but i’m still noticing that about 20% of videos recorded appear to fast forward at some point in the video.
July 10th, 2009 at 0:53
@Bob Jones
What are you using as a media server?!
FMIS 3 or 3.5 ?
July 10th, 2009 at 19:22
I’m using FMIS 3.5?
July 10th, 2009 at 19:22
I’m using FMIS 3.5
July 12th, 2009 at 12:08
@Bob Jones
We have not tested yet with FMIS 3.5, but we will and we will post our findings.
You need to change the value of AllowedVideoLag and MaxSize in Server.xml to control how much the media server will wait for the video data before writing the already received audio data to disk!
July 14th, 2009 at 21:22
yes, i have tried setting these parameters to values that should have been sufficient to offset any lag but alas there are still videos that appear to fast forward.
July 16th, 2009 at 17:31
@Bob Jones
We will make some tests and post our findings!
October 15th, 2009 at 23:53
Naicu!
We´re using Red5, and we´re having so many issues!! You describe them here perfectly.
Do you know if there´s a way to make this work in red5 or the bug (your link is broken) is something that cannot be solved?
Should we implement FMS 3.5 instead? Any hosting you`d recommend?
Thank you, Naicu!
October 18th, 2009 at 17:52
@MaX
I do not know of any work around for Red5. Red5 also changed bug databases (actually what they did is they deleted all previously submitted bugs and started from 0 the process of bugs gathering) recently that why the link is broken.
Your best choice right now is Wowza as it is the only one 100% fixing the issue. FMIS 3.5 claims to have solved it but we got reports that it still does not work 100%. For FMIS 3.5 hosting check out http://www.influxis.com .
January 12th, 2010 at 15:08
Thanks Naicu for this information. We were having similar problem in FMS recording will try with 3.5.
Thanks,
Vivek
March 2nd, 2010 at 7:42
Thought I’d note that we too are noticing the odd video appears fast forwarded. Also working with FMIS 3.5 and have tried tweaking the rec settings on server.xml.
August 14th, 2010 at 0:05
Thank you for this interesting article. I haven’t even thought about this problem and I will think how to implement a better way in http://erlyvideo.org/
Perhaps, better way is to do so:
1) send audio frames directly to client
2) buffers some audio frames in memory of writing process.
August 17th, 2010 at 9:47
It seems, that I’ve added to erlyvideo proper reordering of frames. It is worth to test with someone.