Adding IPTV Channels to Bouquet

[snb41]

I love Xmas holidays, family, food, alcohol and I get to tinker with my beyonwiz

So I have found a way to add Australian free iptv channels to a bouquet on the T, U, V series, my main motivation was to find a way to watch the horse racing channels on my TV instead of my phone, but for others the ability to watch FTA channels from other states might be of interest. I am in Wollongong NSW but the picture below shows that I can Watch CH7 News in Perth, pretty cool.. So basically you can add any other channel from any other state and switch channels to them in your normal channel list, I have them in a separate bouquet called IPTV but you can also just add them to your normal channel list. There will be no EPG for the channels, probably doable but above my pay grade

Anyway, it will take me a while to write up a bit of a guide so just wanted to see if there is any interest out there first of all, if so I will put something together.

[IanSav]

Hi Snb41,

Is there a handy reference site for all the service URLs?

Regards,
Ian.

[snb41]

Hi Snb41,

Is there a handy reference site for all the service URLs?

Regards,
Ian.

I only know of this one which I got from a post on the Whirpool Forums

https://docs.google.com/spreadsheets/d/ ... edit#gid=0

[MrQuade]

That's a fantastic resource, thanks for posting the link!

Do any(all?) of the international links require a VPN service?

[Paul_oz53]

...
Anyway, it will take me a while to write up a bit of a guide so just wanted to see if there is any interest out there first of all, if so I will put something together.

I am intrigued how to add the .m3u8 entry to the bouquet. Don't recall ever seeing a "how to" on that score.

Is it only the bouquets.tv file that must be edited or is it the lamedb and lamedb5 too or something more than that that's required.

Don't want to put you to too much effort because I'm probably more resilient than most but a guide would be great for others.

Cheers,

[MrQuade]

Following these instructions should work:
http://www.vuplus-community.net/board/t ... ams.20673/

[prl]

I had a bit of a play with streaming URLs. I just got a timeout for the one for ABC News (the https://abc-iview-mediapackagestreams-1.akamaized.net/out/v1/50345bf35f664739912f0b255c172ae9/index_1.m3u8 URL in the spreadsheet that Snb41 posted).

The one for SBS One in the same spreadsheet worked.

[prl]

To use streaming servicerefs, you don't need to change anything in lamedb or lamedb5. lamedb5 isn't used (though it is written) by the Beyonwiz firmware, anyway. The lamedb* files are only for broadcast (FTA/cable/satellite) services.

The form of the favourites service entries using .m3u8 URLs for the bouquets files is:
#SERVICE 4097:0:1:0:0:0:0:0:0:3:https%3a//name.of.server/path/to/service.m3u8:Name of Service

The :3: field is for buffering flags. 1 to enable buffering, 2 to enable progressive download. Progressive download needs buffering, so the legal values are 0 (unbuffered), 1 (buffered) and 3 (buffered with progressive download.

For example, here's an entry that works for SBS One:
#SERVICE 4097:0:1:0:0:0:0:0:0:3:https%3a//sbs-live.akamaized.net/hls/live/2002827/channel1/geo/master_2500K.m3u8:SBS One Stream

For streaming URLs that end in .ts, the form of the bouquet file line is:
#SERVICE 1:0:0:0:0:0:0:0:0:0:https%3a//name.of.server/path/to/service.ts:Name of Service

All of the URLs in the file spreadsheet that snb41 linked to are .m3u8-style URLs.

In the URL and NAME fields, ":" is encoded as "%3a", "%" is encoded as "%25", and control characters (0x01-0x1f) are encoded similarly: 0x0a (NEWLINE) as "%0a". You can encode ASCII NUL as %00, but don't. That's string terminator in C (and C++).

The cleanest way to experiment with this is to create a new bouquet/favourites list for the streaming services (use the PVR UI or Open Webif), and then to add the streaming servicerefs to the bouquet file.

Find the bouquet file (it will be in the form /etc/enigma2/userbouquet.*, where the last part of the name will contain the name that you gave to the bouquet). Add the servicerefs you want to have to the bouquet file. Then in the PVR UI, make sure that you are not currently using the modified bouquet, and do FAV, MENU>Reload services.

Bouquet files can have either U*ix line endings (<LF>) or DOS/Windows line endings (<CR><LF>). You can even mix line ending styles in the same file, if you must.

It may be more convenient to copy the bouquet file to your computer, modify it there, and the copy it back. The UI must have been stopped before you copy it back.

Does that give enough information?

[prl]

For heavy-duty users, streaming URLs (the 4097 reftype) are handled in lib/service/servicemp3.{h,cpp}. That's where I got the information about the buffering flags from.

[Grumpy_Geoff]

It may be more convenient to copy the bouquet file to your computer, modify it there, and the copy it back. The UI must have been stopped before you copy it back.

I think you can do a 'Reload' in the OWIF BouquetEditor after copying the new bouquet contents to the Wiz without needing a GUI stop/restart.

[snb41]

That's a fantastic resource, thanks for posting the link!

Do any(all?) of the international links require a VPN service?

I haven't had a chance to try them but I would imagine so, I hd this working a few days ago but could not get the picons working, finally got those working this morning

[snb41]

Here is a tool that will change the links in the Google doc to links that will work in the Beyonwiz bouquet files

http://iptv-dailylinks.blogspot.com/p/c ... igma2.html

[prl]

Here is a tool that will change the links in the Google doc to links that will work in the Beyonwiz bouquet files

http://iptv-dailylinks.blogspot.com/p/c ... igma2.html

It made a right proper mess of the description, though.

[snb41]

It seems like you guys have started playing and I have not had time to write the process, so a little tip on getting the picons to work.
Using Ten Perth as an example this is what the M3U to Enigma2 converter spits out:
#NAME IPTV
#SERVICE 4097:0:1:0:0:0:0:0:0:0:https%3a//network10sec-live-me.akamaized.net/hls/live/2000077/10WA/stream.m3u8: 0=0; 0
#DESCRIPTION 0=0; 0

The Picon png file that you load should be named as so: 4097_0_1_0_0_0_0_0_0_0.png

Also if you don't load them all in the converter together they will have that exact same number, so you need to edit them in the bouquet file to be a different number, I just changed the last number so 4097010000001, 4097010000002 etc..

[snb41]

It made a right proper mess of the description, though.

Yes I had to add them back in manually, just replace the 0=0; 0 with the name, e.g Ten Perth

There are probably way easier ways to do it but at least we know it works, now just need to fine-tune the process

[prl]

It may be more convenient to copy the bouquet file to your computer, modify it there, and the copy it back. The UI must have been stopped before you copy it back.

I think you can do a 'Reload' in the OWIF BouquetEditor after copying the new bouquet contents to the Wiz without needing a GUI stop/restart.

Reload in Open Webif didn't work for me, but FAV, MENU>Reload services did. But in the second case, if the altered bouquet is the current bouquet, I needed to switch to a new bouquet and then back to see the changes.

[prl]

Following these instructions should work:
http://www.vuplus-community.net/board/t ... ams.20673/

Some of them work.

However, the suggestion to add descriptions this way:
#SERVICE 1:X:X:X:X:X:X:X:X:X:link_to_your_stream#DESCRIPTION any_stream_name_you_want
or
#SERVICE 4097:X:X:X:X:X:X:X:X:X:link_to_your_stream#DESCRIPTION any_name_you_want

will not work.

The following patterns work for descriptions:
#SERVICE 1:X:X:X:X:X:X:X:X:X:link_to_your_stream:any_stream_name_you_want
or
#SERVICE 1:X:X:X:X:X:X:X:X:X:link_to_your_stream
#DESCRIPTION any_stream_name_you_want

[IanSav]

Hi Snb41,

Thank you for the information. It looks like others have been interested as well.

Regards,
Ian.

[Paul_oz53]

Hi Snb41,

Thank you for the information. It looks like others have been interested as well.

Regards,
Ian.

+1 - Plenty to play with while on an enforced holiday!

Thanks all.

[prl]

It seems like you guys have started playing and I have not had time to write the process, so a little tip on getting the picons to work.
Using Ten Perth as an example this is what the M3U to Enigma2 converter spits out:
#NAME IPTV
#SERVICE 4097:0:1:0:0:0:0:0:0:0:https%3a//network10sec-live-me.akamaized.net/hls/live/2000077/10WA/stream.m3u8: 0=0; 0
#DESCRIPTION 0=0; 0

The Picon png file that you load should be named as so: 4097_0_1_0_0_0_0_0_0_0.png

Also if you don't load them all in the converter together they will have that exact same number, so you need to edit them in the bouquet file to be a different number, I just changed the last number so 4097010000001, 4097010000002 etc..

The last numerical entry in an MP3 servicetype (4097) serviceref is reserved for the buffering flags, even though some of the code documentation says that it is unused. That should be set to one of 0 (unbuffered), 1, (buffered) or 3 (buffered, progressive load). Other bits are probably ignored, but the the code says that if the 2 bit is set, the 1 bit must also be set, so even numbers other than 0 are not permitted.

Better to use another field. I suggest using the SID field (the 4th numerical field), so use servicerefs like 4097:0:1:0:0:0:0:0:0:0:..., 4097:0:1:1:0:0:0:0:0:0:..., 4097:0:1:2:0:0:0:0:0:0:... and the corresponding .png file names, like 4097_0_1_2_0_0_0_0_0_0.png for the third of those servicerefs. However, I've only done limited testing to see whether that is OK in all cases.

Also note that the first two numerical fields are decimal, and the remaining numerical fields are hexadecimal. Why? I have no idea.

[prl]

Here is a bouquet file for (almost) all the streaming services in the spreadsheet that Snb41 posted.

Missing services: ABC News National (YT Live) and Fox News Talk Radio United States.

Known problems:
None of the ABC services I tried out seemed to work. They all gave an error in the debug log.
The Uni Of California TV United States service hangs for a while (spinner) and then times out.
There are what appear to be audio-only streams. They should probably go in separate radio-style bouquets and be played in radio mode. They may work.

If anyone has correct URLs for the ABC services, I'll make a new bouquet that has them.

Doing this as a single bouquet is clunky. Each of the sections of the bouquet file should probably be its own bouquet. I'm happy to do that, but installing it will be a little more complicated.

I use the SID field of the serviceref to allow the streaming services to link to a unique picon file. E.g:
4097:0:1:1A:0:0:0:0:0:3:...:The Outdoor Cooking Channel United States
and correspondingly:
4097_0_1_1A_0_0_0_0_0_3.png

Install

Do a settings backup.

Create a new bouquet called "Streaming Services" (must have the space, but case is unimportant) in either the PVR GUI or in Open Webif.

Unzip the attached ZIP file. That will create a file called userbouquet.streaming_services__tv_.tv. Have a look at it and edit it if you want to.

Copy that file into /etc/enigma2 on the PVR, overwriting the userbouquet.streaming_services__tv_.tv created when the bouquet was created.

In the PVR GUI, make sure that you are not currently using the modified bouquet, and do FAV, MENU>Reload services.

Doing a Reload in Open Webif Bouquet editor doesn't seem to do what's needed to update the bouquet in the running system.

Uninstall

Switch to a bouquet other than Streaming Services and delete the Streaming Services bouquet, then purge deleted bouquets.

[prl]

Here's the streaming services spreadsheet as a .csv, which is what I've used in my script that creates the bouquet.

[snb41]

Here is a bouquet file for (almost) all the streaming services in the spreadsheet that Snb41 posted.

You are the best! I just spent 40mins writing up a guide on how I did it that is way more difficult to follow and allot more work, I might edit it and put it in the first post adding in the bouquets that you created if that is ok with you?

[prl]

That's fine by me.

Have you tried following my installation instructions?

[snb41]

Have you tried following my installation instructions?

Yes worked a treat, I probably found it easier because I knew where everything was, around 2 weeks ago I wouldn't have had a clue what you were on about I will try and do a step by step appealing to where I was at before I started looking into this, your ready-made bouquet makes it easy, people can just edit out what they don't want.

[Grumpy_Geoff]

In the PVR GUI, make sure that you are not currently using the modified bouquet, and do FAV, MENU>Reload services.

Doing a Reload in Open Webif Bouquet editor doesn't seem to do what's needed to update the bouquet in the running system.

Perhaps the OWIF reload just grabs a new copy in the browser from disk, then one has to make an "edit" to force a disk/memory re-write

[prl]

In the PVR GUI, make sure that you are not currently using the modified bouquet, and do FAV, MENU>Reload services.

Doing a Reload in Open Webif Bouquet editor doesn't seem to do what's needed to update the bouquet in the running system.

Perhaps the OWIF reload just grabs a new copy in the browser from disk, then one has to make an "edit" to force a disk/memory re-write

It doesn't even seem to do that. All it seems to do is update the list of bouquets from the memory on the PVR. If the contents of the bouquets have changed, that's not updated.

[prl]

This set of bouquet files puts the streaming service lists that were sections in a large single bouquet into bouquets of their own, one for each section:
ABC Streams (TV)
Community Television Streams (TV)
Foxtel Streams (TV)
General Entertainment Streams (TV)
Multilingual Streams (TV)
Music Streams (TV)
...

The services themselves have the same issues as the single-bouquet version.

This form of the streaming services allows you to easily delete sections that you're not interested in by simply deleting the corresponding bouquet.

The installation is similar to, but not the same as, the single-bouquet version.

Install

If you have installed the single streams bouquet, you may want to delete it and purge deleted bouquets, but you can keep it if you want to. Both contain the same set of streaming services, just arranged differently

Do a settings backup.

Unzip the attached ZIP file, possibly into a separate directory: it contains 17 files. There is one file called append.bouquets.tv, and the other files have the form userbouquet.*_streams__tv_.tv.

Copy all the userbouquet.*_streams__tv_.tv files into /etc/enigma2 on the PVR.

Append the contents of append.bouquets.tv to /etc/enigma2/bouquets.tv.

In the PVR GUI do FAV, MENU>Reload services.

Doing a Reload in Open Webif Bouquet editor doesn't seem to do what's needed to update the bouquet in the running system.

Uninstall

Switch to a bouquet other than any of the streaming bouquets and delete the streaming bouquets, then purge deleted bouquets.

[Grumpy_Geoff]

The one big "Streaming Services" bouquet isn't such a handful, as one can move to the different sections (bouquet markers) by using PREV/NEXT.

[prl]

The one big "Streaming Services" bouquet isn't such a handful, as one can move to the different sections (bouquet markers) by using PREV/NEXT.

I wasn't aware of that

Anyway, it wasn't much work on the generator script to have separate bouquets.

[adoxa]

If anyone has correct URLs for the ABC services, I'll make a new bouquet that has them.

They need permission, but the method I use for downloading doesn't work. The method used by the Kodi plugin does work, though. If you have any idea how to add the token to the URL (and replace it once it's expired) I can provide what's required.

[IanB]

Hacky idea, add a local url to the bouquet, the local url is a "cgi" that calculates the appropriate true url and issues it in a 302 redirect (temporary move). Adding additional handlers to the inbuilt webserver is trivial.

[adoxa]

That'll work; attached is a program to do it (tested on Windows, but not on the box; I also haven't tested the resulting stream actually plays). It's pretty raw atm, but could probably turn it into a plugin if desired, or maybe just include it as part of pre-start. Not even sure the best of way testing it, probably run it in SSH and ^C/kill to stop. Those without SSH could create a script containing "$(dirname "$0")"/iviewserver.py& and run that, but there's no way to terminate it.

[Grumpy_Geoff]

That'll work; attached is a program to do it (tested on Windows, but not on the box; I also haven't tested the resulting stream actually plays).

That didn't work for me on my U4.

I started up the "local iview redirection web server" -

Code: Select all

root@beyonwizu4:/hdd/Runnables# /usr/bin/python ./iviewserver.py &

I added this "redirection" entry for ABC News 24 to the bouquet -

Code: Select all

4097:0:1:900:0:0:0:0:0:3:http%3a//127.0.0.1%3a4097/api/iview/abc-news-24:ABC-News-24	

But I got nowt -

Code: Select all

{840}<  3153.648> [eServiceMP3] construct!
{840}<  3153.649> [eServiceMP3] playbin uri=http://127.0.0.1:4097/api/iview/abc-news-24
{840}<  3153.649> [eServiceMP3] starting pipeline
{840}<  3153.657> [SCREENNAME]  InfoBarSummary
{840}<  3153.660> KEY: 352 break KEY_OK ('OK',)
{840}<  3153.664> [SCREENNAME]  HideVBILine
{840}<  3153.667> [eServiceMP3] state transition NULL -> READY
{840}<  3153.672> [SCREENNAME]  InfoBar
{840}<  3153.677> [SCREENNAME]  HideVBILine
{3695}<  3153.678> [eBackgroundFileEraser] deleting '/media/hdd/timeshift/pts_livebuffer_1.meta.del'
{3695}<  3153.679> [eBackgroundFileEraser] deleting '/media/hdd/timeshift/pts_livebuffer_1.eit.del'
{840}<  3153.682> [Notifications] RemovePopup, id = ZapError
{840}<  3153.683> [eServiceMP3] GST_STREAM_STATUS_TYPE_CREATE -> setting timeout on souphttpsrc to 30s
{840}<  3154.705> [eConsoleAppContainer] Starting /usr/bin/grab
{840}<  3154.886> [eServiceMP3] Gstreamer error: Not authorized to access resource. (15) from hlsdemux0
{840}<  3154.886> [eServiceMP3] Gstreamer error: Invalid manifest. (7) from hlsdemux0
{840}<  3155.630> [DVBCAHandler] no more services

What did I do wrong?

[prl]

Adoxa, are your iView access tokens still current?

[Paul_oz53]

The one big "Streaming Services" bouquet isn't such a handful, as one can move to the different sections (bouquet markers) by using PREV/NEXT.

It's early days but I am favouring the separate streams approach.

I think the biggest 'problem' is weeding out irrelevant and dead links. Not a criticism - just a practical reality that the source has been faithfully converted (thanks PRL ) but is not necessarily current.

Two observations:

Some of the dead links resulted in a lock up on the U4, necessitating a power switch reboot. Not sure if it was a timeout issue but I wasn't patient waiting to try over 300 streams.

Some streams start with video and sound but sound drops after a few seconds (e.g. 7 main channels). For most streams, cycling to another stream and back restores sound but some streams die and don't recover.

I suspect some of the dead OS links may be geoblocked but not sure I can configure the VPN to redirect the U4. Nord client on the PC works fine but untested on these urls and no client on the U4. Anyway, local USA news is of limited appeal.

Overall, an interesting development with some promise for the future.

Grateful for the efforts being expended all round.

[MrQuade]

Some streams start with video and sound but sound drops after a few seconds (e.g. 7 main channels). For most streams, cycling to another stream and back restores sound but some streams die and don't recover.

I wonder if that means the problematic ones need the buffering flags to be set differently.

I suspect some of the dead OS links may be geoblocked but not sure I can configure the VPN to redirect the U4. Nord client on the PC works fine but untested on these urls and no client on the U4. Anyway, local USA news is of limited appeal.

I was eyeing off the Adult Swim stream as a useful one, but they mostly look pretty unappealing.
There are several VPN plugins available as plugins if I recall correctly. Have you looked into running the VPN client on U4 directly?

[Grumpy_Geoff]

Some streams start with video and sound but sound drops after a few seconds (e.g. 7 main channels). For most streams, cycling to another stream and back restores sound but some streams die and don't recover.

I wonder if that means the problematic ones need the buffering flags to be set differently.

Good thinking "99".
I added a second entry with different buffering for Seven Sydney and it now has audio, whereas the former didn't -

Code: Select all

#SERVICE 4097:0:1:CF:0:0:0:0:0:3:https%3a//npc.cdn.7livecloud.io/hls/live/SYD1/master.m3u8:Seven Sydney
#SERVICE 4097:0:1:CF:0:0:0:0:0:1:https%3a//npc.cdn.7livecloud.io/hls/live/SYD1/master.m3u8:Seven Sydney (1.buffered)

[Paul_oz53]

...

I suspect some of the dead OS links may be geoblocked but not sure I can configure the VPN to redirect the U4. Nord client on the PC works fine but untested on these urls and no client on the U4. Anyway, local USA news is of limited appeal.

I was eyeing off the Adult Swim stream as a useful one, but they mostly look pretty unappealing.
There are several VPN plugins available as plugins if I recall correctly. Have you looked into running the VPN client on U4 directly?

Not yet. Will have to find time to do it but with tennis about to start, access will be limited.

[Paul_oz53]

Some streams start with video and sound but sound drops after a few seconds (e.g. 7 main channels). For most streams, cycling to another stream and back restores sound but some streams die and don't recover.

I wonder if that means the problematic ones need the buffering flags to be set differently.

Good thinking "99".
I added a second entry with different buffering for Seven Sydney and it now has audio, whereas the former didn't -

Code: Select all

#SERVICE 4097:0:1:CF:0:0:0:0:0:3:https%3a//npc.cdn.7livecloud.io/hls/live/SYD1/master.m3u8:Seven Sydney
#SERVICE 4097:0:1:CF:0:0:0:0:0:1:https%3a//npc.cdn.7livecloud.io/hls/live/SYD1/master.m3u8:Seven Sydney (1.buffered)

That's a quick fix. I take it the difference is between the 1 and the 3.

[MrQuade]

That's a quick fix. I take it the difference is between the 1 and the 3.

Yep. See prl's post about it here.

[prl]

I'm happy to prune dead links and make adjustments to the buffering flags. Perhaps it's safer to default buffering to 1 (buffered) instead of 3 (buffered, progressive load).

On the one hang that I had in my limited testing, The Uni California link took some time to time out, but eventually it did. The only on-screen indication I got was that when it timed out the spinner disappeared.

I have a feeling that collections of links like this might need ongoing curation.

[adoxa]

That'll work; attached is a program to do it (tested on Windows, but not on the box; I also haven't tested the resulting stream actually plays).

That didn't work for me on my U4.

Okay, tried on my T2 and got the same thing. Wget does retrieve the manifest, so the redirection works. Adding the tokenised URL directly to the bouquet fails to resolve the server name, although it looks right. Trying gst-launch directly says "Couldn't retrieve key for decryption". Bugger, looks like a GStreamer problem with the stream itself (the Windows version (1.16.2) works, though).

[freihaut]

Did you ever create a guide (for dummies) snb41? I got as far as creating a new bouquet called "Streaming Services" following prl's instructions but I don't know where to put the url's.

[Grumpy_Geoff]

I got as far as creating a new bouquet called "Streaming Services" following prl's instructions but I don't know where to put the url's.

From the "StreamingServices.zip" file (here)?
If so, first take a settings backup.
Then once that's done, put the unzipped bouquet file onto a USB thumb drive. Insert the drive into the V2.
Open File Commander (MENU>>Sources / Files) -

• Navigate the left panel to open "Internal flash", then "etc", then "enigma2"
  This is the "target" panel (directory /etc/enigma2).
• Press NEXT to swap to the right panel
  This will be the "source" panel
• Navigate to the thumb drive and open it
• Select the file "userbouquet.streaming_services__tv_.tv" (have the cursor on it)
• Press YELLOW to copy the selected file to the target panel's location
  Reply yes to the copy prompt
• EXIT back to the main menu.

Go to the "Plugins" column and scroll towards the bottom and select the "SimpleUmount" entry and open it (alternatively, select and open it from the Plugin browser).
Select the thumb drive if it's not already selected, and press OK. Reply yes to the "really umount..." prompt.
Once the unmount has occurred, exit back to live TV and detach the thumb drive.
Press FAV to bring up the channels (services) list, then MENU>>Reload services. The streaming services bouquet should now show as populated.

[freihaut]

Great. Thanks Grumpy, that worked, streaming services bouquet is populated. The entire channel list is there and I can view a few channels (local and overseas) however most don't work. When trying to switch channels, the channel being watched freezes and an Error message comes up: "Timeshift on a stream is not supported!". Is this fixable or are streams just not that reliable?

[MrQuade]

an Error message comes up: "Timeshift on a stream is not supported!". Is this fixable or are streams just not that reliable?

Not so much an error, its just a warning that timeshift doesn't work in streams.

You can turn off that popup warning via an option in the Timeshft Settings screen MENU->Setup->TV->Timeshift settings

That setting won't change your ability to timeshift in streams, but it will stop hassling you about it.

[Henk]

You can however record the stream (press RED button),and accept the defaults for testing purposes.

Then, by hitting the Media button, go to the recording and simulate the "time shifting" by playing/pausing the recording.

[adoxa]

With a change in data plan I'm more free to experiment, so here's working ABC & SBS live streams (ABC would have worked back then, just needed to override the user agent). ATM you still need to run the server manually, I may do it by plugin in future...