1. Home
  2. Projects
  3. Pylon and GStreamer

Pylon and GStreamer

Introducing the InstantCameraAppSrc class: An integration of Pylon's InstantCamera with GStreamer's AppSrc, so you can use your Basler camera with GStreamer pipelines! Applications include streaming video across network, displaying to framebuffers, etc.

Join Project Follow Project

Project State

Public Project Participation wanted

Licences

Software Licence: Apache 2.0
Hardware Licence: Project has no hardware

Project Tags

Tutorial Windows Linux Image Pipeline ARM x86 Raspberry Pi C / C++

Admins

 mattb

Members

 heshanfer
Andy504
dcfortin
kpelou-manzamasso
lee-kenny

Does this project pique your interest?

Login or register to join or follow this project.

Cancel

Using Pylon with GStreamer: The InstantCameraAppSrc class

  • The InstantCameraAppSrc class presents the Basler camera as a source element for GStreamer pipelines.

    • It combines Pylon's InstantCamera class with GStreamer's AppSrc plugin.
    • The InstantCamera class offers access to physical camera functions and Pylon driver functions.
    • The AppSrc plugin offers an API to bring user-defined images, data, etc. into GStreamer pipelines.
    • InstantCameraAppSrc can be extended via the GenApi to access any camera and driver feature (eg: GetFrameRate()).
    • InstantCameraAppSrc cab be extended via GStreamer "bins" to include any other plugins within the source element (eg: Rescale, Rotate, etc.)

Architecture

Sample Programs

  • Sample programs based on the InstantCameraAppSrc class are found in the Samples folder.

    • "DemoPylonGStreamer" is a rich demonstration of possibilities, including a "PipelineHelper" class to assist in making pipelines.
    • "SimpleGrab" is an example of the bare minimum code needed to create a GStreamer application.
    • Linux makefiles are included for each sample application.
    • Windows Visual Studio project files are included for each sample application in the respective "vs" folder.

Requirements

  • Linux x86/x64/ARM or Windows 7/10. (OSX has not been tested.)

    • Pylon 5.0.9 or higher on Linux. Pylon 5.0.10 or higher on Windows. (Older versions down to Pylon 3.0 may work, but are untested.)
    • GStreamer 1.0. (GStreamer 0.1 may work, but is untested.)
    • Note for Linux users: You may need to install the following libraries:

  gstreamer1.0

  gstreamer1.0-dev

  gstreamer1.0-libav

  gstreamer1.0-plugins-bad

  libgstreamer-plugins-base1.0-dev

  (on Ubuntu systems, all can be isntalled using apt-get install)

Note about missing GStreamer Plugins (elements)

  • All systems are different. Some GStreamer pipeline plugins used in the Sample Programs may not exist on your system!

    • As a result, you may see errors like "Not Negotiated", etc. when running the samples.
    • Consult GStreamer directly for more information: https://gstreamer.freedesktop.org/

GitHub Repository

https://github.com/MattsProjects/pylon_gstreamer

Commits

 Breit 		pushed 6f9cd0acb14834321b080a75f86d982feb3b9bcb
Removed unused function definition Increment()
2018-03-30 14:10:25 UTC
Matthew B 		pushed 49cfba949a26866173caeee9e456f113f00b5e23
Merge pull request #6 from cirquit/master
2018-03-30 14:08:15 UTC
cirquit 		pushed 137c0bd2ca11914cec0c7d2133c0dafe73b4d8af
removed CInstantCameraAppSrc prefix to compile demopylonstreamer
2018-03-30 11:27:41 UTC
Breit 		pushed 5246bb52d835e88e2595e80510c045a5292e6d0f
determined grab timeout problem in two camera sample when in OnDemand mode was probably due to a single bad test camera (21949158) not handling software trigger properly. Other combinations of cameras seem to work fine so far...
2018-03-17 02:04:58 UTC
Breit 		pushed c1c9f74a04c37e712f3443312cbe517b2cf0d7f6
instantcameraappsrc: fixed potential bug with setting width. added SetFrameRate() api. New Sample: twocameras_compositor.
2018-03-16 19:24:50 UTC
Breit 		pushed ceba6307dd84eca6c25bab5d37bc52094f870d8d
improvement: now using unique names based on serial number when creating appsrc and sourcebin. Required when using multiple cameras in the same pipeline.
2018-03-12 21:00:45 UTC
Breit 		pushed 2483d4805d31efecceb65d80bc1ba67ec0016ef1
MAJOR CHANGE: New folder structure, renaming of pylongstreamer sample to demopylongstreamer, addition of new sample programs, and cleanup of readme and documentation. The reason is to clarify the relationship of the sample programs to the core of the project, the InstantCameraAppSrc class. Also, this structure offers flexibility for adding more sample programs to help with the diversity of systems which may use this project.
2018-03-12 14:32:05 UTC
Breit 		pushed 171cba14245ce58479b01def728884d68403df08
fixed merge conflicts.
2018-03-09 18:15:38 UTC
Breit 		pushed 68c9a7b5ebc7610aa938d72aacf083a63f9cbc0d
new feature: use your own gst-launch-1.0 pipeline. Use -parse and paste the command, and the PipelineHelper will parse it and replace your original source with the InstantCameraAppSrc. Also added some more notes to the user about using pylongstreamer with Nvidia TX1/TX2.
2018-03-09 18:10:50 UTC
mattbreit 		pushed 6898823b90b0451beb75dd6fea694e4a1c042efa
fixed issue #3 and issue #4. Issue #3 fixed by adding an additional caps filter to the CPipelineHelper::build_pipeline_display(). On Nvidia TX1 and TX2, the videosink found by autovideosink seems to not advertise the formats it supports, so we must tell the converter element manually to convert the RGB image to I420 (or something else). Issue #4 fixed by moving gst buffer wrapping from CInstantCameraAppSrc::InitCamera() to CInstantCameraAppSrc::retrieve_image()
2018-03-06 20:36:25 UTC
Matthew B 		pushed 963e80c2c3aae67930639e0aaa842d6f5cbfad55
Delete pylon_gstreamer_orig.zip
2017-11-16 16:12:23 UTC
matt breit 		pushed 916944e295704690440b1aea710f3ddbf50a7dda
Bugfix (API Break): moved control of h264file file size from pipelinehelper to instantcameraappsrc. Pipeline helper was setting num-buffers on source, but source is a gstbin made by instantcamerappsrc, so it is not possible to access the child appsrc element like this. pylongstreamer:numFramesToRecord is now applied to appsrc element in instantcameraappsrc:GetSource() before element is binned. additionally, the binned source now forwards the appsrc's EOS message to the pipeline when num-buffers is hit. Finally, removed muxer from pipelinehelper:h264file pipeline. It caused negotiation error with omxh264 (odd, but at least this was seen on rpi3 today...). API Break: pipelinehelper:h264file no longer accepts number of frames to record as argument.
2017-11-10 22:00:00 UTC
MattB 		pushed bb542d1c6c3ab9785d12c3f934a3eeea5260ee3a
changed from using std::exception(string) to std::runtime_error(string)
2017-09-26 18:34:47 UTC
MattB 		pushed 5e65197c18649d3226b071b528f8ebd0462473e1
pylongstreamer: updated the sample program to support new InstantCameraAppSrc feature (it now uses the rescaler and rotator from the instantcameraappsrc instead of the pipelinehelper). Also started to clean up exception handling.
2017-09-22 14:59:16 UTC
MattB 		pushed 743882f926d0b4ec91f329e6d34beb427b728af9
InstantCameraAppSrc: new features (api break): now provides a gst bin element as a source, which includes appsrc+rescaler+rotator (previously just appsrc was provided). GetGstAppSrc() now changed to GetSource() and m_source is now m_appsrc. Fix: moved device enumeration and attachment of camera to device from InitCamera() to constructor. This makes it possible to change settings of camera before calling InitCamera(), for example if you want to reset the camera to default settins first.
2017-09-22 14:57:21 UTC
Matthew B 		pushed 5fb1ea6d061c98bf8528169cbb395dc9d5cc9ab7
CPipelineHelper: cleaned up rescaler code.
2017-09-21 01:54:51 UTC
Matthew B 		pushed 543afeb6c9fcb4a4e364dc0e433e084ef9c44333
StopCamera(): Added safety measure to send EOS event before StopGrabbing(), which prevents need-data signals from appsrc during StopGrabbing(). StopGrabbing clears grab engine buffers, which renders the ptrGrabResult pointer invalid (aka no grab data referenced error in retrieve_image().
2017-09-10 05:32:47 UTC
Matthew B 		pushed fe1677c73999c441920e9b80b2f45647b16346c9
pylongstreamer.cpp: corrected recommended order of operations - shutdown pipeline before stopping camera to avoid requests for image during a StopGrabbing().
2017-09-10 05:17:32 UTC
Matthew B 		pushed aaddb9b0799fd4a9a00678f18457e7108b875ec2
pylongstreamer demo app: fixed stupid mistake, forgot to use proper name for rescaler option (-rescale)
2017-09-09 03:54:54 UTC
Matthew B 		pushed 826977c6993b3262ae02ddb9ef4068dc410296a8
pylongstreamer demo application: added support for new features (defaulting and rescaling). Added -aoi and -window console option (-width/-height/-display still available but deprecated) Cleaned up code and comments.
2017-09-09 03:40:14 UTC
Matthew B 		pushed 26711a456734026d2ebbc8cf05b028f26b13ca96
CPipelineHelper: Added option to rescale the image when creating the pipelinehelper
2017-09-09 03:37:12 UTC
Matthew B 		pushed b3e59168ffac6ba7dda3f3491901db76f3db84d4
CInstantCameraAppSrc: added ability to default camera to max aoi and framerate by passing arguments of -1. Fixed order-of-operations by setting framerate after aoi.
2017-09-09 03:34:17 UTC
Matthew B 		pushed e8e5669e9d8d699e1986eb1291cd34aa5a20306a
Added original code in zip file
2017-09-05 20:15:59 UTC
Matthew B 		pushed a7ac50c9a4fd5f6570fa7c043e04d0023ead2da4
Merge pull request #1 from MattsProjects/wraps_gst
2017-09-05 19:40:33 UTC
Matt B 		pushed 0ce25d85e4eed07db975558b378caf573bb8854b
Corrected makefile string - CInstantCameraForAppSrc --> CInstantCameraAppSrc
2017-09-04 21:45:07 UTC
MattB 		pushed 1bc3ba83b5d95edf8210730b6490625800620ae3
(api break from master branch). This new version wraps a lot of the code found in the implementation into the InstantCameraAppSrc class.
2017-09-01 19:47:35 UTC
Matt 		pushed d8dfc92d8e2a6b18fab7f0f8055b96b230a7eff7
Bringing everything about the source element (setup, callback handler, pushing buffers, etc.) into CInstantCameraApp.
2017-09-01 05:55:43 UTC
MattB 		pushed 1ff125a14fc2e72a618f4d7cf54fb1eedc637975
License change to Apache 2.0.
2017-08-30 20:31:33 UTC
MattB 		pushed a83304ca3bf2be99bb84b5a709a6834d8a40459b
Now using int instead of int64_t for GetWidth/Height (int64_t had problems on one 32bit arm platform).
2017-08-30 20:14:19 UTC
mattbreit 		pushed 401b29add43b276b88a2b6d996b5ac18e6cf3ff8
for bcon support: removed migration mode and will manage SFNC3 names manually.
2017-08-29 00:01:43 UTC
Task Owner Creation Date
Add code to automatically check for missing plugins on system
mattb 		2017-12-05

Comments
mattb report abuse

Streaming video to browser?

Would anyone like an additional pipeline built which would allow browser-based access to a camera's video stream?
If so, do you have a preference on which webserver should be used as part of this reference sample?
(Browser access requires sinking the pipeline to a webserver on the device with the camera. This could be Apache, Icecast, etc.)

Let me know your thoughts!

- Matt
mattb report abuse
New Feature, and better support for Nvidia Tx2/Tx1! You can now use your own command line pipeline from gst-launch-1.0 with the pylongstreamer -parse argument, and the PipelineHelper will use it with the InstantCameraAppSrc source camera instead! Also, added a little extra support to the PipelineHelper to help with using autovideosink for display on Nvidia tx1/tx2 platforms. Feedback is welcome!

Your comments, please!

Want to comment this ... Show more