[App] libSambilight E/F/H (MST-only)

Here are software that related with Samsung F series TVs.
Please don't create any new topic here unless you have software to post/release.
Post Reply

tasshack
SamyGO Project Donor
Posts: 65
Joined: Thu Aug 23, 2018 5:54 am
Location: Istanbul, TURKEY

[App] libSambilight E/F/H (MST-only)

Post by tasshack »

High FPS Ambient Lighting Solution for Samsung TVs

Image

Description
This library utilizes USB serial connection for controlling externally connected leds to achieve high FPS and low latency ambient lighting experience. It has multiple profile support different led area configurations with color correction parameters.
For high frame rate capturing without performance issues, library uses native gfx library (libUTOPIA) to capture screen.

I highly recommend watching the demo

Features
- Easy to configure with testing features
- Multiple profile support
- TV Remote for controlling state, profiles and brightness (Only E/F for now)
- Automatic profile switching with black border detection
- HSV correction
- Brightness correction
- Uses native graphics library for high FPS and almost zero performance loss (not needed on H)
- Captures with or without OSD (Only E/F)

Hardware
TV has only FDTI and ACM kernel modules therefore you need to use an Arduino Pro Micro (Only on F/H) or FTDI USB serial converter (Fake ones will work too).

Only on F/H
You can use an Arduino Pro Micro/Leonardo or another board with USB CDC capability as is. This setup is recommended since it does not requires to connect an external hardware with onboard USB socket is used for connecting to TV. (You can use the attached ino with this configuration)
Image

If you need WiFi, you can connect an ESP8266 NodeMCU to FTDI USB serial converter as shown below. Board can be powered via it's USB port rather than VIN pin if you want to. (You need to use the attached ino with this configuration!)
Image

You can connect a Arduino Pro Mini to FTDI USB serial converter as shown below. (Not recommended for high number of leds!)
Image

Firmware
Library uses Adalight protocol therefore you can use any Adalight compatible sketch as firmware. I if you are using an ESP8266 board with onboard USB Serial adapter, you need to use the attached sambilight-arduino.ino sketch.
Sketch: https://github.com/dmadison/Adalight-FastLED
More information

Configuration
Configuration parameters are similar to hyperion so you can use HyperCon config utility to easily determine values below.

Image

Library can support multiple led area configurations. Profiles can be edited from libSambilight.config file and is switchable via remote control or automatically by black border detection.

01_99_Sambilight.init
- H_LEDS: Horizontal leds count (Top Side)
- V_LEDS: Vertical leds count (Left/Right Side)
- BOTTOM_GAP: Bottom side gap (Top Size - Bottom Size)
- START_OFFSET: Index of first led (Top Left is zero) (H_LEDS + V_LEDS + (H_LEDS - BOTTOM_GAP / 2))
- CLOCKWISE: Led order (Set to 0 for anti-clockwise)
Optional
- COLOR_ORDER: Capture color order
- CAPTURE_POS: Capture starts from bottom (Set to 1 for upside down and dont forget to change color order too)
- BLACK_BORDER: Black border detection for automatic profile switching (16:9, 21:9, 4:3, 3:2)
- TV_REMOTE: Enable control with TV remote buttons (Red, Green, Yellow, Blue) (SET TO 0 ON H)
- DEVICE: USB Serial device path (/dev/ttyUSB0)
- BAUDRATE: USB Serial baudrate. (921600 is highly recommended)
- GFX_LIB: Capture directly from graphics driver resulting +50 FPS and prevents UI from lagging during capture. (Not needed on H)
- CAPTURE_FREQ: Capture frequency in ms, increase this value if your TV having performance issues (Higher the value, lower the refresh rate.)
- CAPTURE_SIZE: Higher capture size resulting less flickering and more accurate blacks but it affects overall FPS (1=96x54 2=160x90 3=240x135 4=320x180 5=480x270 6=960x540)
- OSD_CAPTURE: Capture with OSD (Only works with GFX_LIB=1)

libSambilight.config (For profiles)
- name: Name of profile
- saturation_gain_percent: Saturation gain in percent
- value_gain_percent: Value gain in percent
- brightness_correction: Brightness correction value (To compensate TVs brightness setting)
- horizontal_depth_percent: Horizontal depth in percent
- vertical_depth_percent: Vertical depth in percent
- overlap_percent: Overlap in percent
- h_padding_percent: Horizontal padding in percent
- v_padding_percent: Vertical padding in percent

Installation
- Enable USB Serial support from factory menu. > Control > Sub Option > USB Serial > ON (Skip this step if you are not using FTDI)
- Install latest samyGOso if needed.
- Copy libSambilight.so and libSambilight.config to /mnt/opt/privateer/usr/libso
- Copy 01_99_Sambilight.init to /mnt/etc/init.d/ (Skip this step if you want to test the configuration first)
- Set file permissions with

Code: Select all

chmod +x /mnt/opt/privateer/usr/libso/libSambilight.so

Code: Select all

chmod +x /mnt/etc/init.d/01_99_Sambilight.init
- Plug your USB Serial adapter and restart the TV. Camera port wont work!

Testing
If you want to test your led configuration, you can run library in single shot mode without copying the libSambilight.config file to init.d folder.
When you run library on singleshot mode, TV will render a test pattern for each led on it's capture area. You can verify that the each corresponding area is matching with it's led index and color. TV also saves a capture to /dtv/Sambilight.bmp so you can verify that your capture parameters are correct.
If everything is fine, BMP file, TV screen and led colors should match perfectly. (You can clear the test pattern by pressing any key on remote)

Image

Behaviour of singleshot mode can be controlled with parameters below.
- TEST_PROFILE: Index of profile will be used on singleshot mode
- TEST_PATTERN: Set to 0 if you don't want to see the test pattern
- TEST_SAVE_CAPTURE: Set to 0 if you don't want to capture images as BMP to /dtv/Sambilight.bmp

For E/F Series

Code: Select all

samyGOso -d -A -B -l /mnt/opt/privateer/usr/libso/libSambilight.so H_LEDS:35 V_LEDS:19 BOTTOM_GAP:7 START_OFFSET:68 CLOCKWISE:1 BAUDRATE:921600 TEST_FRAMES:1 TEST_PATTERN:1 TEST_PROFILE:0 TEST_SAVE_CAPTURE:1
For H Series
For rendering test pattern

Code: Select all

samyGOso -d -A -B -l /mnt/opt/privateer/usr/libso/libSambilight.so H_LEDS:35 V_LEDS:19 BOTTOM_GAP:7 START_OFFSET:68 CLOCKWISE:1 BAUDRATE:921600 TEST_FRAMES:1 TEST_PATTERN:1 TEST_PROFILE:0 TEST_SAVE_CAPTURE:0
For capturing bmp and updating the leds

Code: Select all

samyGOso -d -T -B -l /mnt/opt/privateer/usr/libso/libSambilight.so H_LEDS:35 V_LEDS:19 BOTTOM_GAP:7 START_OFFSET:68 CLOCKWISE:1 BAUDRATE:921600 TEST_FRAMES:1 TEST_PATTERN:0 TEST_PROFILE:0 TEST_SAVE_CAPTURE:1

To Do
- WLED Support
- gfx_lib for E Series
- gfx_lib for H Series
- Better configuration testing support
- Support for Non-MST H Series
- Arduino Pro Micro support for E series
- Remote control support for H series
- ECO sensor support for dynamic brightness adjustment
- USB HID or MSC device for not requiring an FTDI device
- Library as native executable app
- Socket support
- Smoothing for low frame rates (May not be needed)
- Custom ESP32 hardware and software (Work in progress)

Technical Details
Capturing
I basically reverse engineered the SdDisplay_CaptureScreen function from libSDAL.so static library and optimized it to take captures continuously without using mutexes. Turns out mali gpu has a feature to capture screen frames to a designated frame buffer and it can be configured to take screen captures continuously but exeTV or exeAPP does not use this feature to record any frames. I still use library on single shot mode because it is easier to handle frame buffer sync issues but i think this feature can also be utilized to record any source from TV without performance issues.
SdDisplay_CaptureScreen function uses native gpu functions for scaling down the image to another designated half size framebuffer but the original capture in 1920x1080 format.
This library uses only the static libraries already in TV for capturing, therefore it does not need to be injected in exeTV or exeAPP to work. It can be easily modified to compile and run as an arm executable.
Processing
I borrowed some features from hyperion and Prismatik projects but i wrote image processor completely from scratch. Library calculates led coordinates continuously according to the current selected profile therefore switching on the fly is possible. Library also handles the smooth transition of any profile setting in configuration. After calculating the average color values of each area, library makes color correction of each led according to the configuration settings.
I did not want to fully dynamic black border detection to prevent false positives on movies so i use black border detection only for profile switching. This way i can use completely different profile settings for 21:9 movies instead of only black border detection.
Currently library has no support for smoothing between frames to save CPU but it can be implemented on USB device.
Transporting
I tried too many things before finding out that when USB Serial support enabled from factory menu, OS loads ftdi_sio.ko and binds connected device to /dev/ttyUSB0.
I will find another way to communicate between TV and the USB device in the future. May be native USB HID driver can be utilized or an libusb virtual mass storage USB device can be configured to receive led data as files.. Later, I found a way to load cdc_acm.ko kernel module and map the usb cdc acm device with mknod. Now TV can attach a USB CDC device to /dtv/ttyACM* via this library without needing an external FTDI USB to serial adapter.

Important Notes
- Only FTDI USB Serial and USB CDC devices (Only on F/H) are supported for device connection. (Boards with cp210x or ch340 will not work!)
- TV_REMOTE must be set to 0 on H series.
- High baudrate is recommended for USB Serial connection.
- USB device must be connected before turning on the TV. There is no USB plug in detection.
- If you enable TV remote feature, you can use Red button for turning leds on/off, Green button for switching between profiles and Yellow button for adjusting brightness level of the leds.
- Black border detection is for only automatic profile switching. When enabled it adds a new profile named "Auto" that switches between other profiles by comparing its padding values with the black borders on the screen. You can still switch another profile from TV remote that and disables black border detection temporarily. You can select Auto profile to enable it again.
- Default profiles are optimized for 16:9, 21:9, 4:3 and 3:2 aspect ratios. I do not recommend to edit it's padding settings unless if you don't want to use black border detection feature. You can edit other parameters of default profiles or you can add another profiles if you want to.
- Library does not work well with libSoftPowerOff. Maybe fixed in the future.
- FPS drops while navigating on the TV menus due to performance limitations of injecting to the exeAPP, but you can fix it by disabling remote control support to inject with -T option.
- Native gfx library capture feature is only tested on F and still experimental (Not required for H) but necessary for smooth operation of the TV. If you are having crashes/reboots on your TV you should disable it. When this feature not enabled you have to increase capture rate and decrease capture size to prevent TV performance issues but smoothing must be handled on the usb device.
- If you have a TV with ECO sensor you may want to disable the eco feature because it interferes with the ambient lighting and panel looks dark all the time.
- Flashing lights may trigger epileptic seizure on certain people and I think that this is the reason of that original ambilight does not uses this kind of high FPS. Instead it uses some PID logic to change the colors by checking the duration on screen and smoothly fades between colors to prevent flashing lights.
- There may be bugs. I did not find time to test all configuration parameters for TV models other than F. If you are having crashes with specific configuration setting, please send me a pm and i will fix it as soon as possible i can.

Changelog
v1.0.4: First sable release
v1.0.5: Capture order feature
v1.0.6: Singleshot mode crash fix
v1.0.7: Baudrate config fix
v1.0.8: Test pattern and test capture feature
v1.1.0: CDC ACM support
v1.1.1: CDC Crash fix
v1.1.2: USB CDC fixes
v1.2.0: Load FTDI kernel modules dynamically
v1.2.1: USB CDC fixes
v1.2.9: Fixes for H Series
v1.2.10: Green flashing and black border detection fixes
v1.2.11: Brightness correction and serial not found fix
v1.3.0: E Series GFX_LIB support and ESP32 support
v1.3.1: Single-shot mode fix

Image
Image
You do not have the required permissions to view the files attached to this post.
Last edited by tasshack on Mon Sep 05, 2022 10:07 pm, edited 59 times in total.
zoelechat
SamyGO Moderator
Posts: 8615
Joined: Fri Apr 12, 2013 7:32 pm
Location: France

Re: [App] libSambilight E/F/H (MST-only)

Post by zoelechat »

Nice work :)
I do NOT receive any PM. Please use forum.
aimaim
SamyGO Project Donor
Posts: 70
Joined: Wed Jan 02, 2013 10:46 pm

Re: [App] libSambilight E/F/H (MST-only)

Post by aimaim »

I've been using adonis' solution for a few years. But this looks pretty cool. I'd like to get rid of the wifi transmission. Also my tv crashes regularly xD.

I just ordered the FTDI USB serial converter and I'm looking forward to trying this out once I have some time off.

Thanks for sharing!
User avatar
notengo13
SamyGO Project Donor
Posts: 674
Joined: Mon Nov 30, 2015 12:31 pm
Location: SK --> P9

Re: [App] libSambilight E/F/H (MST-only)

Post by notengo13 »

Good job. Congrats looks great.
On new Neo qled I have to use govee immersion which has worst performance compared to Adonis libs or this one.

I hope @sectroyer will cook something soon. :mrgreen:
UE48H6640 T-MST14DEUC 2781.0 root latestH --> 2130.0 skypeRoot-->0000-->2115.3 "Skype refueled" Root
QE55QN93A - - >firmware T-NKM2DEUC-1805.4 (backup 1590.0) - - >update CURL60 Error --> CURL28 Error
User avatar
erdem_ua
SamyGO Admin
Posts: 3125
Joined: Thu Oct 01, 2009 6:02 am
Location: Istanbul, Turkey
Contact:

Re: [App] libSambilight E/F/H (MST-only)

Post by erdem_ua »

Wow. Very good work taashack!
Edit: This app gains $500 bounty from project!
aimaim
SamyGO Project Donor
Posts: 70
Joined: Wed Jan 02, 2013 10:46 pm

Re: [App] libSambilight E/F/H (MST-only)

Post by aimaim »

When I inject libSambilight.so via ssh the tv turns off (too fast to grab the log file).
USB Serial is activated and the FTDI USB serial converter is plugged into the tv.

I'm on F-Series (UE55F6770).

Any hints are appreciated, thanks :)
aimaim
SamyGO Project Donor
Posts: 70
Joined: Wed Jan 02, 2013 10:46 pm

Re: [App] libSambilight E/F/H (MST-only)

Post by aimaim »

I managed to get the the log before the tv crashed (see attachment).

It seems the tv doesn't recognize the usb serial adapter / arduino. I flashed it successfully using this tutorial:
https://alselectro.wordpress.com/2017/0 ... load-code/
and these parameters:

Code: Select all

const uint16_t 
  Num_Leds   =  216;         // strip length
const uint8_t
  Brightness =  255;         // maximum brightness

// --- FastLED Setings
#define LED_TYPE     APA102   // led strip type for FastLED
#define COLOR_ORDER  GBR      // color order for bitbang
#define PIN_DATA     2        // led data output pin
#define PIN_CLOCK    0        // led data clock pin (uncomment if you're using a 4-wire LED type)

// --- Serial Settings
const unsigned long
  SerialSpeed    = 115200;  // serial port speed
const uint16_t
  SerialTimeout  = 60;      // time before LEDs are shut off if no data (in seconds), 0 to disable

// --- Optional Settings (uncomment to add)
#define SERIAL_FLUSH          // Serial buffer cleared on LED latch
// #define CLEAR_ON_START     // LEDs are cleared on reset

// --- Debug Settings (uncomment to add)
// #define DEBUG_LED 13       // toggles the Arduino's built-in LED on header match
// #define DEBUG_FPS 8        // enables a pulse on LED latch
Here are the components I used.
https://www.amazon.de/gp/product/B0753G ... UTF8&psc=1
https://www.amazon.de/gp/product/B07VKL ... UTF8&psc=1

After flashing I changed the pinning as described at the top of the page (connecting the cts port aswell).

UPDATE:
tasshack was nice enough to help me via pn. It turns out, the one-shot-mode might have a bug.
When I used the init file instead, I had the problem, that no other samygo components were loaded, so I had no chance to connect via ssh or ftp. tasshack suggested to rename the init file to 99_99_Sambilight.init, so that it is loaded last, resulting in everything else starting before it.
This workaround did the trick for me. (Of course it does not explain why it behaves that way,)

Now I'm stuck at getting the arduino to actually put something out to the led strip. I'll update this in order for people to avoid reproducing my noob mistakes.
Speaking of which: It was not a great idea on my part to solder the arduino and the usb serial adapter directly together, to find out later, that in order to flash the arduino, the pinning needs to be slightly different. :oops: (tutorial under the first link in this post)

UPDATE:
It turns out both pinning variants seem to work.
You do not have the required permissions to view the files attached to this post.
Last edited by aimaim on Thu Nov 11, 2021 4:04 pm, edited 1 time in total.
tasshack
SamyGO Project Donor
Posts: 65
Joined: Thu Aug 23, 2018 5:54 am
Location: Istanbul, TURKEY

Re: [App] libSambilight E/F/H (MST-only)

Post by tasshack »

aimaim wrote: Wed Nov 10, 2021 10:09 pm I managed to get the the log before the tv crashed (see attachment).

It seems the tv doesn't recognize the usb serial adapter / arduino. I flashed it successfully using this tutorial:
https://alselectro.wordpress.com/2017/0 ... load-code/
and these parameters:

Code: Select all

const uint16_t 
  Num_Leds   =  216;         // strip length
const uint8_t
  Brightness =  255;         // maximum brightness

// --- FastLED Setings
#define LED_TYPE     APA102   // led strip type for FastLED
#define COLOR_ORDER  GBR      // color order for bitbang
#define PIN_DATA     2        // led data output pin
#define PIN_CLOCK    0        // led data clock pin (uncomment if you're using a 4-wire LED type)

// --- Serial Settings
const unsigned long
  SerialSpeed    = 115200;  // serial port speed
const uint16_t
  SerialTimeout  = 60;      // time before LEDs are shut off if no data (in seconds), 0 to disable

// --- Optional Settings (uncomment to add)
#define SERIAL_FLUSH          // Serial buffer cleared on LED latch
// #define CLEAR_ON_START     // LEDs are cleared on reset

// --- Debug Settings (uncomment to add)
// #define DEBUG_LED 13       // toggles the Arduino's built-in LED on header match
// #define DEBUG_FPS 8        // enables a pulse on LED latch
Here are the components I used.
https://www.amazon.de/gp/product/B0753G ... UTF8&psc=1
https://www.amazon.de/gp/product/B07VKL ... UTF8&psc=1

After flashing I changed the pinning as described at the top of the page (connecting the cts port aswell).

UPDATE:
tasshack was nice enough to help me via pn. It turns out, the one-shot-mode might have a bug.
When I used the init file instead, I had the problem, that no other samygo components were loaded, so I had no chance to connect via ssh or ftp. tasshack suggested to rename the init file to 99_99_Sambilight.init, so that it is loaded last, resulting in everything else starting before it.
This workaround did the trick for me. (Of course it does not explain why it behaves that way,)

Now I'm stuck at getting the arduino to actually put something out to the led strip. I'll update this in order for people to avoid reproducing my noob mistakes.
Speaking of which: It was not a great idea on my part to solder the arduino and the usb serial adapter directly together, to find out later, that in order to flash the arduino, the pinning needs to be slightly different. :oops: (tutorial under the first link in this post)
I fixed the issue and updated the zip files. Can you try the singleshot mode with new version.
I didn't realized that the pin configuration was wrong on my first post. I will update the image with correct one, sorry about that and thanks for your help.
aimaim
SamyGO Project Donor
Posts: 70
Joined: Wed Jan 02, 2013 10:46 pm

Re: [App] libSambilight E/F/H (MST-only)

Post by aimaim »

Thanks for the fix. Singleshot mode works for me now.

Yet my leds are still all white or all off.

I attached a photo of the boards. My init parameters are as follows:

Code: Select all

# Required Parameters
H_LEDS=70;				# Horizontal leds count (Top Side)
V_LEDS=38;				# Vertical leds count (Left/Right Side)
BOTTOM_GAP=0;			# Bottom side gap (Top Size - Bottom Size)
START_OFFSET=143;		# Index of first led (Top Left is zero) (H_LEDS + V_LEDS + (H_LEDS - BOTTOM_GAP / 2))

# Optional Parameters
COLOR_ORDER=RGB;		# Capture color order
CAPTURE_POS=0;			# Capture starts from bottom (Set to 1 for upside down and dont forget to change color order too)
CLOCKWISE=1;			# Led order (Set to 0 for anti-clockwise)
BLACK_BORDER=1;			# Black border detection for automatic profile switching (16:9, 21:9, 4:3, 3:2)
TV_REMOTE=1;			# Enable control with TV remote buttons (Red, Green, Yellow, Blue)
EXTERNAL=0;				# On/Off support for remotely connected leds (Handled on USB device)
DEVICE=/dev/ttyUSB0;	# USB Serial device path
BAUDRATE=921600;		# USB Serial baudrate (High baudrate is highly recommended)
GFX_LIB=1;				# Capture directly from graphics driver resulting +50 FPS and prevents UI from lagging during capture (Experimental! Tested on F)
CAPTURE_FREQ=20;		# Capture frequency in ms, increase this value if your TV having performance issues (Higher the value, lower the refresh rate. Smooting must be handled on USB device)
CAPTURE_SIZE=5;			# Higher capture size resulting less flickering and more accurate blacks but it affects overall FPS (1=96x54 2=160x90 3=240x135 4=320x180 5=480x270 6=960x540)
OSD_CAPTURE=1;			# Capture with OSD (Only works with GFX_LIB=1)

# For Testing
TEST_FRAMES=0;			# For configuration and performance testing (Set to 1 for single-shot mode)
You do not have the required permissions to view the files attached to this post.
tasshack
SamyGO Project Donor
Posts: 65
Joined: Thu Aug 23, 2018 5:54 am
Location: Istanbul, TURKEY

Re: [App] libSambilight E/F/H (MST-only)

Post by tasshack »

aimaim wrote: Thu Nov 11, 2021 11:06 am Thanks for the fix. Singleshot mode works for me now.

Yet my leds are still all off or all white.
Serial speed must match with the baudrate in config parameters. Can you update the code in the sketch like below.

Code: Select all

const unsigned long
	SerialSpeed    = 921600;  // serial port speed

Post Reply

Return to “[F] Software”