Imaginghub

Documentation

**Using Pylon with GStreamer: The InstantCameraAppSrc class**

The InstantCameraAppSrc class presents the Basler camera as a source element for GStreamer pipelines.
- It combines Pylon's InstantCamera class with GStreamer's AppSrc plugin.
- The InstantCamera class offers access to physical camera functions and Pylon driver functions.
- The AppSrc plugin offers an API to bring user-defined images, data, etc. into GStreamer pipelines.
- InstantCameraAppSrc can be extended via the GenApi to access any camera and driver feature (eg: GetFrameRate()).
- InstantCameraAppSrc cab be extended via GStreamer "bins" to include any other plugins within the source element (eg: Rescale, Rotate, etc.)

Architecture

Sample Programs

Sample programs based on the InstantCameraAppSrc class are found in the Samples folder.
- "DemoPylonGStreamer" is a rich demonstration of possibilities, including a "PipelineHelper" class to assist in making pipelines.
- "SimpleGrab" is an example of the bare minimum code needed to create a GStreamer application.
- Linux makefiles are included for each sample application.
- Windows Visual Studio project files are included for each sample application in the respective "vs" folder.

Requirements

Linux x86/x64/ARM or Windows 7/10. (OSX has not been tested.)
- Pylon 5.0.9 or higher on Linux. Pylon 5.0.10 or higher on Windows. (Older versions down to Pylon 3.0 may work, but are untested.)
- GStreamer 1.0. (GStreamer 0.1 may work, but is untested.)
- Note for Linux users: You may need to install the following libraries:

gstreamer1.0

gstreamer1.0-dev

gstreamer1.0-libav

gstreamer1.0-plugins-bad

libgstreamer-plugins-base1.0-dev

(on Ubuntu systems, all can be isntalled using apt-get install)

Note about missing GStreamer Plugins (elements)

All systems are different. Some GStreamer pipeline plugins used in the Sample Programs may not exist on your system!
- As a result, you may see errors like "Not Negotiated", etc. when running the samples.
- Consult GStreamer directly for more information: https://gstreamer.freedesktop.org/

Code

GitHub Repository

https://github.com/MattsProjects/pylon_gstreamer

Commits

Matthew B	pushed d611a6f20b45df8d2d3b7c10072b69d4f1a1da41	CPipelineHelper.cpp: build_pipeline_display(): Changed from autovideosink to nv3dsink for display.	2021-04-05 20:02:55 UTC
Matthew B	pushed 9d9605b6a8d7153587086ec8e702a6155d66c70c	Added local copies of CInstantCameraAppSrc.cpp and .h to the new jetson-tailored sample for use there (and only there).	2021-03-09 02:55:29 UTC
Matthew B	pushed 02133378f9670c1cfa4ebbad69586e0edb4b7b8b	New Sample for a performance-on-jetson focused demopylongstreamer.	2021-03-09 02:52:31 UTC
Breit	pushed 3434c115900659f1b410ba87162e7d5b996cc625	Made a better check of the pylon image pixel format when determining what the videoconvert's caps format should be.	2020-01-27 15:49:58 UTC
Matthew B	pushed 88506cb423622c4115e27a02a74d5e388cae9273	Merge pull request #16 from TobiasSchaffner/readme_fix	2019-12-02 21:31:55 UTC
Tobias Schaffner	pushed 7a61194fde1e3a7e3d373f6f0541d36ec3fb06aa	Use a code block for overview ascii	2019-11-29 10:30:22 UTC
Matthew B	pushed e8d5fb9adfa8d2e90edeb24c68a5d225013a9cc9	Issue #14: Added check for too many pipelines specified.	2019-11-19 18:36:22 UTC
Breit	pushed 1df8f16942952ed5b12d859cd49279bd66749fd2	Issue #13: Due to else-if, framerate wasn't actually being set. oops! :)	2019-11-18 16:47:07 UTC
Breit	pushed cc1bbb3636f0cb945dc7396f8e090a3d4f2ae104	Added basic draft of multicast pipeline to demopylongstreamer sample (Issue #12).	2019-10-11 15:09:52 UTC
Breit	pushed 8ddb0fb31e93428ccfdccfd110afa3320c7c6480	Added a videoconverter and filter caps to the AppSrc bin so that if the image is color, it's converted to I420 (if it's mono it stays GRAY8).	2019-10-10 21:27:38 UTC
Breit	pushed d3e762ac2412731c160cb9e31744d77317a66678	Changed order of encoders back to omxh264enc first, because v4l2h264enc had some problems on raspberry pi, and we only really use that encoder for the Snapdragon 820 devices which are less common.	2019-10-10 21:26:08 UTC
Breit	pushed f978c65e0bca5323542925bb79f0a92418edd7e3	Removed videoscaler things from CPipelineHelper for simplicity (Issue #8).	2019-10-10 16:51:55 UTC
Breit	pushed 9c50b7cccb9c747c20e9b55251e5d5ac3cf699d5	improvements to h264stream helper for nvidia nano board and changed default port from 5000 to 554.	2019-10-10 16:20:45 UTC
Breit	pushed dd7c852d1c61927101c2b571e6bfdc00224e3325	Adjusted the method of applying rescaler caps, so that always we put caps on it, even if we are not rescaling. This is so we can have a clear caps for the final bin, which includes format, width, height, and framerate.	2019-10-10 16:17:30 UTC
Breit	pushed 82c0d89209deba6ff1898ed58323439789c3a091	Added two new helper functions:	2019-09-25 15:13:44 UTC
Breit	pushed 043d048a9302e0be0554d8c32ec6b04bf6f380dd	Added support for v4l2h264enc H.264 encoder on BCON-for-MIPI Snapdragon 820 devkit.	2019-08-28 16:39:35 UTC
Matthew B	pushed c129d304262926dfa31520cc519b8b2823046ea7	Added additional safety checks around features in case the camera used doesn't support them.	2019-08-27 15:26:35 UTC
Breit	pushed 6f9cd0acb14834321b080a75f86d982feb3b9bcb	Removed unused function definition Increment()	2018-03-30 14:10:25 UTC
Matthew B	pushed 49cfba949a26866173caeee9e456f113f00b5e23	Merge pull request #6 from cirquit/master	2018-03-30 14:08:15 UTC
cirquit	pushed 137c0bd2ca11914cec0c7d2133c0dafe73b4d8af	removed CInstantCameraAppSrc prefix to compile demopylonstreamer	2018-03-30 11:27:41 UTC
Breit	pushed 5246bb52d835e88e2595e80510c045a5292e6d0f	determined grab timeout problem in two camera sample when in OnDemand mode was probably due to a single bad test camera (21949158) not handling software trigger properly. Other combinations of cameras seem to work fine so far...	2018-03-17 02:04:58 UTC
Breit	pushed c1c9f74a04c37e712f3443312cbe517b2cf0d7f6	instantcameraappsrc: fixed potential bug with setting width. added SetFrameRate() api. New Sample: twocameras_compositor.	2018-03-16 19:24:50 UTC
Breit	pushed ceba6307dd84eca6c25bab5d37bc52094f870d8d	improvement: now using unique names based on serial number when creating appsrc and sourcebin. Required when using multiple cameras in the same pipeline.	2018-03-12 21:00:45 UTC
Breit	pushed 2483d4805d31efecceb65d80bc1ba67ec0016ef1	MAJOR CHANGE: New folder structure, renaming of pylongstreamer sample to demopylongstreamer, addition of new sample programs, and cleanup of readme and documentation. The reason is to clarify the relationship of the sample programs to the core of the project, the InstantCameraAppSrc class. Also, this structure offers flexibility for adding more sample programs to help with the diversity of systems which may use this project.	2018-03-12 14:32:05 UTC
Breit	pushed 171cba14245ce58479b01def728884d68403df08	fixed merge conflicts.	2018-03-09 18:15:38 UTC
Breit	pushed 68c9a7b5ebc7610aa938d72aacf083a63f9cbc0d	new feature: use your own gst-launch-1.0 pipeline. Use -parse and paste the command, and the PipelineHelper will parse it and replace your original source with the InstantCameraAppSrc. Also added some more notes to the user about using pylongstreamer with Nvidia TX1/TX2.	2018-03-09 18:10:50 UTC
mattbreit	pushed 6898823b90b0451beb75dd6fea694e4a1c042efa	fixed issue #3 and issue #4. Issue #3 fixed by adding an additional caps filter to the CPipelineHelper::build_pipeline_display(). On Nvidia TX1 and TX2, the videosink found by autovideosink seems to not advertise the formats it supports, so we must tell the converter element manually to convert the RGB image to I420 (or something else). Issue #4 fixed by moving gst buffer wrapping from CInstantCameraAppSrc::InitCamera() to CInstantCameraAppSrc::retrieve_image()	2018-03-06 20:36:25 UTC
Matthew B	pushed 963e80c2c3aae67930639e0aaa842d6f5cbfad55	Delete pylon_gstreamer_orig.zip	2017-11-16 16:12:23 UTC
matt breit	pushed 916944e295704690440b1aea710f3ddbf50a7dda	Bugfix (API Break): moved control of h264file file size from pipelinehelper to instantcameraappsrc. Pipeline helper was setting num-buffers on source, but source is a gstbin made by instantcamerappsrc, so it is not possible to access the child appsrc element like this. pylongstreamer:numFramesToRecord is now applied to appsrc element in instantcameraappsrc:GetSource() before element is binned. additionally, the binned source now forwards the appsrc's EOS message to the pipeline when num-buffers is hit. Finally, removed muxer from pipelinehelper:h264file pipeline. It caused negotiation error with omxh264 (odd, but at least this was seen on rpi3 today...). API Break: pipelinehelper:h264file no longer accepts number of frames to record as argument.	2017-11-10 22:00:00 UTC
MattB	pushed bb542d1c6c3ab9785d12c3f934a3eeea5260ee3a	changed from using std::exception(string) to std::runtime_error(string)	2017-09-26 18:34:47 UTC

Tasks

	Task	Owner	Creation Date
	Add code to automatically check for missing plugins on system	mattb	2017-12-05

Info

Project State

Public Project

Participation wanted

Licences

Software Licence: Apache 2.0

Hardware Licence: Project has no hardware

Does this project pique your interest?

Comments

Streaming video to browser?

Would anyone like an additional pipeline built which would allow browser-based access to a camera's video stream?
If so, do you have a preference on which webserver should be used as part of this reference sample?
(Browser access requires sinking the pipeline to a webserver on the device with the camera. This could be Apache, Icecast, etc.)

Let me know your thoughts!

- Matt

New Feature, and better support for Nvidia Tx2/Tx1! You can now use your own command line pipeline from gst-launch-1.0 with the pylongstreamer -parse argument, and the PipelineHelper will use it with the InstantCameraAppSrc source camera instead! Also, added a little extra support to the PipelineHelper to help with using autovideosink for display on Nvidia tx1/tx2 platforms. Feedback is welcome!

Pylon and GStreamer