Trend Topics
More Topics

Pylon and GStreamer

Introducing the InstantCameraAppSrc class: An integration of Pylon's InstantCamera with GStreamer's AppSrc, so you can use your Basler camera with GStreamer pipelines! Applications include streaming video across network, displaying to framebuffers, etc.

Join Project Follow Project
Documentation
Cancel

Using Pylon with GStreamer: The InstantCameraAppSrc class

  • The InstantCameraAppSrc class presents the Basler camera as a source element for GStreamer pipelines.

    • It combines Pylon's InstantCamera class with GStreamer's AppSrc plugin.
    • The InstantCamera class offers access to physical camera functions and Pylon driver functions.
    • The AppSrc plugin offers an API to bring user-defined images, data, etc. into GStreamer pipelines.
    • InstantCameraAppSrc can be extended via the GenApi to access any camera and driver feature (eg: GetFrameRate()).
    • InstantCameraAppSrc cab be extended via GStreamer "bins" to include any other plugins within the source element (eg: Rescale, Rotate, etc.)

Architecture

Sample Programs

  • Sample programs based on the InstantCameraAppSrc class are found in the Samples folder.

    • "DemoPylonGStreamer" is a rich demonstration of possibilities, including a "PipelineHelper" class to assist in making pipelines.
    • "SimpleGrab" is an example of the bare minimum code needed to create a GStreamer application.
    • Linux makefiles are included for each sample application.
    • Windows Visual Studio project files are included for each sample application in the respective "vs" folder.

Requirements

  • Linux x86/x64/ARM or Windows 7/10. (OSX has not been tested.)

    • Pylon 5.0.9 or higher on Linux. Pylon 5.0.10 or higher on Windows. (Older versions down to Pylon 3.0 may work, but are untested.)
    • GStreamer 1.0. (GStreamer 0.1 may work, but is untested.)
    • Note for Linux users: You may need to install the following libraries:

  gstreamer1.0

  gstreamer1.0-dev

  gstreamer1.0-libav

  gstreamer1.0-plugins-bad

  libgstreamer-plugins-base1.0-dev

  (on Ubuntu systems, all can be isntalled using apt-get install)

Note about missing GStreamer Plugins (elements)

  • All systems are different. Some GStreamer pipeline plugins used in the Sample Programs may not exist on your system!

    • As a result, you may see errors like "Not Negotiated", etc. when running the samples.
    • Consult GStreamer directly for more information: https://gstreamer.freedesktop.org/
Code

GitHub Repository

https://github.com/MattsProjects/pylon_gstreamer

Commits

 Breit 		pushed cc1bbb3636f0cb945dc7396f8e090a3d4f2ae104
Added basic draft of multicast pipeline to demopylongstreamer sample (Issue #12).
2019-10-11 15:09:52 UTC
Breit 		pushed 8ddb0fb31e93428ccfdccfd110afa3320c7c6480
Added a videoconverter and filter caps to the AppSrc bin so that if the image is color, it's converted to I420 (if it's mono it stays GRAY8).
2019-10-10 21:27:38 UTC
Breit 		pushed d3e762ac2412731c160cb9e31744d77317a66678
Changed order of encoders back to omxh264enc first, because v4l2h264enc had some problems on raspberry pi, and we only really use that encoder for the Snapdragon 820 devices which are less common.
2019-10-10 21:26:08 UTC
Breit 		pushed f978c65e0bca5323542925bb79f0a92418edd7e3
Removed videoscaler things from CPipelineHelper for simplicity (Issue #8).
2019-10-10 16:51:55 UTC
Breit 		pushed 9c50b7cccb9c747c20e9b55251e5d5ac3cf699d5
improvements to h264stream helper for nvidia nano board and changed default port from 5000 to 554.
2019-10-10 16:20:45 UTC
Breit 		pushed dd7c852d1c61927101c2b571e6bfdc00224e3325
Adjusted the method of applying rescaler caps, so that always we put caps on it, even if we are not rescaling. This is so we can have a clear caps for the final bin, which includes format, width, height, and framerate.
2019-10-10 16:17:30 UTC
Breit 		pushed 82c0d89209deba6ff1898ed58323439789c3a091
Added two new helper functions:
2019-09-25 15:13:44 UTC
Breit 		pushed 043d048a9302e0be0554d8c32ec6b04bf6f380dd
Added support for v4l2h264enc H.264 encoder on BCON-for-MIPI Snapdragon 820 devkit.
2019-08-28 16:39:35 UTC
Matthew B 		pushed c129d304262926dfa31520cc519b8b2823046ea7
Added additional safety checks around features in case the camera used doesn't support them.
2019-08-27 15:26:35 UTC
Breit 		pushed 6f9cd0acb14834321b080a75f86d982feb3b9bcb
Removed unused function definition Increment()
2018-03-30 14:10:25 UTC
Matthew B 		pushed 49cfba949a26866173caeee9e456f113f00b5e23
Merge pull request #6 from cirquit/master
2018-03-30 14:08:15 UTC
cirquit 		pushed 137c0bd2ca11914cec0c7d2133c0dafe73b4d8af
removed CInstantCameraAppSrc prefix to compile demopylonstreamer
2018-03-30 11:27:41 UTC
Breit 		pushed 5246bb52d835e88e2595e80510c045a5292e6d0f
determined grab timeout problem in two camera sample when in OnDemand mode was probably due to a single bad test camera (21949158) not handling software trigger properly. Other combinations of cameras seem to work fine so far...
2018-03-17 02:04:58 UTC
Breit 		pushed c1c9f74a04c37e712f3443312cbe517b2cf0d7f6
instantcameraappsrc: fixed potential bug with setting width. added SetFrameRate() api. New Sample: twocameras_compositor.
2018-03-16 19:24:50 UTC
Breit 		pushed ceba6307dd84eca6c25bab5d37bc52094f870d8d
improvement: now using unique names based on serial number when creating appsrc and sourcebin. Required when using multiple cameras in the same pipeline.
2018-03-12 21:00:45 UTC
Breit 		pushed 2483d4805d31efecceb65d80bc1ba67ec0016ef1
MAJOR CHANGE: New folder structure, renaming of pylongstreamer sample to demopylongstreamer, addition of new sample programs, and cleanup of readme and documentation. The reason is to clarify the relationship of the sample programs to the core of the project, the InstantCameraAppSrc class. Also, this structure offers flexibility for adding more sample programs to help with the diversity of systems which may use this project.
2018-03-12 14:32:05 UTC
Breit 		pushed 171cba14245ce58479b01def728884d68403df08
fixed merge conflicts.
2018-03-09 18:15:38 UTC
Breit 		pushed 68c9a7b5ebc7610aa938d72aacf083a63f9cbc0d
new feature: use your own gst-launch-1.0 pipeline. Use -parse and paste the command, and the PipelineHelper will parse it and replace your original source with the InstantCameraAppSrc. Also added some more notes to the user about using pylongstreamer with Nvidia TX1/TX2.
2018-03-09 18:10:50 UTC
mattbreit 		pushed 6898823b90b0451beb75dd6fea694e4a1c042efa
fixed issue #3 and issue #4. Issue #3 fixed by adding an additional caps filter to the CPipelineHelper::build_pipeline_display(). On Nvidia TX1 and TX2, the videosink found by autovideosink seems to not advertise the formats it supports, so we must tell the converter element manually to convert the RGB image to I420 (or something else). Issue #4 fixed by moving gst buffer wrapping from CInstantCameraAppSrc::InitCamera() to CInstantCameraAppSrc::retrieve_image()
2018-03-06 20:36:25 UTC
Matthew B 		pushed 963e80c2c3aae67930639e0aaa842d6f5cbfad55
Delete pylon_gstreamer_orig.zip
2017-11-16 16:12:23 UTC
matt breit 		pushed 916944e295704690440b1aea710f3ddbf50a7dda
Bugfix (API Break): moved control of h264file file size from pipelinehelper to instantcameraappsrc. Pipeline helper was setting num-buffers on source, but source is a gstbin made by instantcamerappsrc, so it is not possible to access the child appsrc element like this. pylongstreamer:numFramesToRecord is now applied to appsrc element in instantcameraappsrc:GetSource() before element is binned. additionally, the binned source now forwards the appsrc's EOS message to the pipeline when num-buffers is hit. Finally, removed muxer from pipelinehelper:h264file pipeline. It caused negotiation error with omxh264 (odd, but at least this was seen on rpi3 today...). API Break: pipelinehelper:h264file no longer accepts number of frames to record as argument.
2017-11-10 22:00:00 UTC
MattB 		pushed bb542d1c6c3ab9785d12c3f934a3eeea5260ee3a
changed from using std::exception(string) to std::runtime_error(string)
2017-09-26 18:34:47 UTC
MattB 		pushed 5e65197c18649d3226b071b528f8ebd0462473e1
pylongstreamer: updated the sample program to support new InstantCameraAppSrc feature (it now uses the rescaler and rotator from the instantcameraappsrc instead of the pipelinehelper). Also started to clean up exception handling.
2017-09-22 14:59:16 UTC
MattB 		pushed 743882f926d0b4ec91f329e6d34beb427b728af9
InstantCameraAppSrc: new features (api break): now provides a gst bin element as a source, which includes appsrc+rescaler+rotator (previously just appsrc was provided). GetGstAppSrc() now changed to GetSource() and m_source is now m_appsrc. Fix: moved device enumeration and attachment of camera to device from InitCamera() to constructor. This makes it possible to change settings of camera before calling InitCamera(), for example if you want to reset the camera to default settins first.
2017-09-22 14:57:21 UTC
Matthew B 		pushed 5fb1ea6d061c98bf8528169cbb395dc9d5cc9ab7
CPipelineHelper: cleaned up rescaler code.
2017-09-21 01:54:51 UTC
Matthew B 		pushed 543afeb6c9fcb4a4e364dc0e433e084ef9c44333
StopCamera(): Added safety measure to send EOS event before StopGrabbing(), which prevents need-data signals from appsrc during StopGrabbing(). StopGrabbing clears grab engine buffers, which renders the ptrGrabResult pointer invalid (aka no grab data referenced error in retrieve_image().
2017-09-10 05:32:47 UTC
Matthew B 		pushed fe1677c73999c441920e9b80b2f45647b16346c9
pylongstreamer.cpp: corrected recommended order of operations - shutdown pipeline before stopping camera to avoid requests for image during a StopGrabbing().
2017-09-10 05:17:32 UTC
Matthew B 		pushed aaddb9b0799fd4a9a00678f18457e7108b875ec2
pylongstreamer demo app: fixed stupid mistake, forgot to use proper name for rescaler option (-rescale)
2017-09-09 03:54:54 UTC
Matthew B 		pushed 826977c6993b3262ae02ddb9ef4068dc410296a8
pylongstreamer demo application: added support for new features (defaulting and rescaling). Added -aoi and -window console option (-width/-height/-display still available but deprecated) Cleaned up code and comments.
2017-09-09 03:40:14 UTC
Matthew B 		pushed 26711a456734026d2ebbc8cf05b028f26b13ca96
CPipelineHelper: Added option to rescale the image when creating the pipelinehelper
2017-09-09 03:37:12 UTC
Tasks
Task Owner Creation Date
Add code to automatically check for missing plugins on system
mattb 		2017-12-05
Info

Project State

Public Project Participation wanted

Licences

Software Licence: Apache 2.0
Hardware Licence: Project has no hardware

Project Tags

Tutorial Windows Linux Image Pipeline ARM x86 Raspberry Pi C / C++

Admins

 mattb

Members

 heshanfer
Andy504
dcfortin
kpelou-manzamasso
lee-kenny

Does this project pique your interest?

Login or register to join or follow this project.

Comments
mattb report abuse

Streaming video to browser?

Would anyone like an additional pipeline built which would allow browser-based access to a camera's video stream?
If so, do you have a preference on which webserver should be used as part of this reference sample?
(Browser access requires sinking the pipeline to a webserver on the device with the camera. This could be Apache, Icecast, etc.)

Let me know your thoughts!

- Matt
mattb report abuse
New Feature, and better support for Nvidia Tx2/Tx1! You can now use your own command line pipeline from gst-launch-1.0 with the pylongstreamer -parse argument, and the PipelineHelper will use it with the InstantCameraAppSrc source camera instead! Also, added a little extra support to the PipelineHelper to help with using autovideosink for display on Nvidia tx1/tx2 platforms. Feedback is welcome!
Back to top

Your comments, please!

Want to comment this ... Show more