DirectShow experience

[Posted 18/7/2007 in http://allencch.multiply.com]

After a long time doing about DirectShow, finally I can grasp what is going on in DirectShow.

For me, the purpose of DirectShow is to get the multimedia input then display the output, eg: *.avi file and video capture device such as webcam. And everything is dealt with “filter” (Please read the documentation for detailed information). And the filters are written in COM (Component Object Module).

To deal with DirectShow, you must download DirectX 9 SDK from MSDN, or download Windows Platform SDK from MSDN, because the DirectShow SDK has removed from DirectX newer SDK. I suggest DirectX 9 SDK, but I don’t know whether one can download it or not.

Because I started the DirectShow with zero knowledge about the COM, it is very very hard to understand the example of the coding. Therefore, one should read the documentation of DirectShow and try some simple examples in the SDK. For learning, trial and error is a very effective method.

GraphEdit.exe is the program you must try first, because this will determine how your algorithm will go.

First of all, remember to use CoInitialize() or CoInitializeEx() (for _WINNT=0400) for COM, and remember to include “atlbase.h” header file.

There are 2 ways for writing the coding about COM. One is using “template”, however I am not familiar with it. But actually they are same, only in different form of expression.

The basic thing we need is IGraphBuilder. All the filters will be added in this IGraphBuilder. Remember to use CoCreateInstance() to create IGraphBuilder. Mostly every filter will be created using CoCreateInstance().

After the IGraphBuilder is created, the filters need to be added in. Which filters you want to add depend on the purpose of your project. Please use GraphEdit.exe to simulate your algorithm first. Then declare your filter in IBaseFilter. This is important, because IGraphBuilder will only deal with IBaseFilter. Besides that, IBaseFilter allows you to enumerate the pin. Please declare/add a filter as a “source filter”, because without “source”, I think nothing can work.

The function of pins is to connect the filters. One filter can have several pins. There are two types of pins, input pin and output pin. Therefore, when two filters are connected, that means the output pin of A filter is connected with the input pin of the B filter. So that, the data will flow from A filter to B filter.

Because all filter are declared using IBaseFilter, the difference is the CLSID (I also don¡¯t know what is this, it is something about the COM). Please refer the documentation about CLSID for each filter. For example, the File Source (Async) Filter uses CLSID_AsyncReader.

After the declaration of IBaseFilter, then add it to the IGraphBuilder using AddFilter() function. However, if it deals with a specific file, then need to use AddSourceFilter(). Like what GraphEdit.exe does, the source file is important. Therefore, using AddSourceFilter() with the specific filename, then it is enough. Why is the file important? I think because DirectShow need to determine what type of file beforehand, then the connection of the filter can be determined.

After the source, you need to add in the other filters. For me, I will find the output pin of the filter first. Pin is very important thing. This is because just using the pin you can render the file already. To deal with the pin, one needs to use IPin interface. Then, use the functions from the documentation and DirectShow example from SDK, “GetUnconnectedPin()”, “GetPin()”, “GetInPin()”, “GetOutPin()” and else, modify it if you like. By using these functions (which I like them very much), you can find the pin of the filter.

After finding the output pin of the source filter, you can render it using render() of IGraphBuilder. What happened if render()? You can try to see it using GraphEdit.exe. But if you want to specify what you want to get from the source, then you will need to add in the filter you like. Therefore, add the filter that you want. Then find the input pin. Then, connect the output pin from the source filter and the input pin from your second filter. Remember to make sure both filters can be connected using GraphEdit.exe. Otherwise, there will be no output.

For me, I would always like to find the output pin only after the filter is connected. This is because, when using the filter like AVI Splitter Filter, if you use GraphEdit.exe, you will find that when you add in the filter, you cannot see any output pin. You can see the output pin only if when you connect the AVI Splitter Filter with the source filter. That is why I will find the output pin only after the filter is connected, or after the source file is determined.

After the second filter is added and connected, find the output pin. Then render() it. Because of DirectShow has “Intelligent Connect”, when you render() the pin, the filters will automatically added without your acknowledgement. Besides that, you can also just add in the other filters without any connection, then you render the output pin, the Intelligent Connection will help you to connect them. (Please read the documentation for detailed information). You can try Intelligent Connection using GraphEdit.exe.

Using GraphEdit.exe, you have filters, source, and connected. Then you would like to run the “run”. Then you click the “play” button at toolbar, then the video works. However, IGraphBuilder does not have any “run: function. Therefore, you need to query interface from other interface (this is a COM method). First, you need to declare IMediaControl. This is to control the media, provides run() function. Then, using QueryInterface() from IGraphBuilder, to query IMediaControl’s interface. QueryInterface() involves IID_IInterfaceName.

For me, query interface is to get the interface of another COM object. So that, IGraphBuilder can run() through IMediaControl. This is similar to the Sample Grabber Filter which query ISampleGrabber interface.

Finally, you can run() the stream using IMediaControl. But, please release() the filter before that. And remember to CoUninitialize() for the COM.

You might debug your program during the development. However, I found an “Error Protection” message that prevents me to debug especially after “render()”. This is because there are 3rd party DirectShow filter. For me, I got installed Nero 6. So, removing the NeVideo.ax filter by using DirectShow Filter Manager from www.softella.com, the debugging problem was fixed.

/* If you don’t like what I write, you can bypass the following part */

What you would like to do after the file is split? Why you want to split the file? Because I need the audio stream while manipulating the video stream. To manipulate the video, that is to get the video buffer (data) and access the buffer, either modify it or copy it. For me, I use the buffer for OpenGL.

Because the stream (I mean the file data) is split into video and audio using AVI Splitter Filter, there are 2 output pins. You should use GraphEdit.exe to check which pin is video and which pin is audio. Using GetPin() function from the GrabBitmaps example of DirectShow SDK, you can get both pins. Both pins are needed because you would like the sound.

Next, you need to add in Sample Grabber Filter to grab the buffer from the output video pin. Thus, you add in the Sample Grabber Filter and then find the input pin as usual. However, you cannot connect the filter first. This is because you need to determine the Media Type of the input stream beforehand. To determine the media type, IBaseFilter cannot do it. Therefore, the filter needs to query interface from ISampleGrabber. So you need to query interface as usual. Then you can SetMediaType() using ISampleGrabber. You can use AM_MEDIA_TYPE variable for SetMediaType(). After SetMediaType(), you can connect the filter.

After the filter is connected, find the output pin. Then, again use AM_MEDIA_TYPE for another variable (remember, it is another variable, should not use the variable before, it might work with same variable, but the program might crash). This variable is used for GetConnectedMediaType(), (if both functions can use the same variable, why does it need 2 functions?).

You need a class that inherits ISampleGrabberCB. ISampleGrabberCB has BufferCB() method which is needed. Then, you need to modify the BufferCB() that the buffer can be copied to the public property (or variable), so that the buffer can be accessed publicly. (Please see the example of GrabBitmaps from SDK).

After GetConnectedMediaType(), you might need to SetSampleBuffer(), SetOneShot() and else. Then, SetCallback() is the important one. This function is used to call ISampleGrabber BufferCB() function. Then, from the ISampleGrabberCB object, you can access the buffer.

By doing these, you can access the buffer of the video file. If you copy the buffer for the OpenGL, you might no need the DirectShow video output. Therefore, you can add in a Null Renderer Filter. Then you render the output pin of the Sample Grabber Filter, it will connect to Null Renderer Filter so that you will not see any video. However, you can access the buffer and produce it in OpenGL.

Besides the Source File (Async) Filter, you can use the webcam as source filter. Therefore, use the FindCaptureDevice() function from PlayCap example from SDK, you can get the video capturing source.