Provider visiting - keeping the FTA EPG current

Moderators: Gully, peteru

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun Nov 25, 2018 22:36

Provider visiting: knock knock... who's there? It's me, the EPG :)

The below is of no use to those using IceTV for their EPG as you already get "frequent updates" to the EPG.

Background -
A few weeks back, due to the change in the Malaysian MotoGP race scheduling that was caused by bad weather, early on that Sunday morning Network TEN altered its program lineup for that day to cater for the change - for their motorsport coverage, this brought the MotoGP live telecast forward and put the V8 supercars highlights program back (it also affected other programs as well).
I was lucky that on that morning I just happened to have the Wiz (using FTA EIT and the AutoTimer plugin) zapped to a Network TEN service for long enough to scavenge the new event times, and thus populate the EPG with those new event changes. This then allowed the existing timers to be updated with new start times by the AutoTimer plugin well enough in advance of the MotoGP start. The V8 supercars event start time changed again about an hour and a half later.
I normally rely on EPGRefresh to keep the EPG data (i) populated, and (ii) current/accurate, but as most of you know, the plugin only runs once per day, and in my case I have it set for the late afternoon to try and cater for the night-time program drifts. So, I wouldn't normally have picked up the MotoGP change until it was too late.

This then got me thinking (a shower/dishwashing/garden watering moment) as to how I could get EPGRefresh to run in a far more frequent manner.

Having dabbled a small amount in OpenWebif command-line access to control the Wiz, I thought I had the answer - use a shell script to drive frequent executions of EPGRefresh's 'Refresh now' option (available from its YELLOW tab).
Then all sorts of issues appeared to me -
  • It would disrupt live TV.
  • I couldn't find a way to directly invoke the plugin as its positioning in the main menu was likely not place specific.
  • The alternative access via the BLUE extensions menu was similarly likely not place specific.
  • I wouldn't be able to get it to run in standby; this was the killer for me!
I then thought of bringing back into use my previous daily repeated short recording timers on each provider, but having more regular repeats of them would mean a lot more timers. I didn't like how they "cluttered" up the timer list when they were just once per day per provider, and I thought the evening-time timer list check would look poxy with so many of them required to run multiple times per provider. So I discounted that idea.

This then led me to deciding to roll my own, and which would use either service zapping if the box is in standby, or short timers if the box is in use.

So, on I went and what started out as a proof of concept soon evolved into something a bit more. I now present the results of that here just in case anyone wants to use it.
I no longer use EPGRefresh, as this script I wrote is doing the job.

[edited for version 1.2, see base of post changes]
Some script "features" -
  • * no longer applicable
    If the box is in standby then use zapping, or if the box is not in standby then use short timers and clean up after ourselves (both timers and recordings).
    Zapping when in standby doesn't bring the box out of standby; it also doesn't start timeshift, and hence being my choice of mechanism if the box is in standby.
  • Execution cycle is conditional on having a free tuner, and also conditional on not having timers due to fire in the next {x} mins that would exceed the current number of free tuners (that's ~4 mins based on scavenging from 6 providers x 30 secs per tune, plus the 20-secs of prepare time for the upcoming "real" timer).
  • * no longer applicable
    The current bouquet is used to select one service per provider to scan so as to not cause a jarring (for the user) bouquet switch.
  • * no longer applicable
    If using zapping, grab the current (last used) service so as to zap back to it at the end (again, user expectation on startup service).
  • It doesn't waste time/energy scanning a provider that the box is already recording or showing live TV from
    * no longer applicable (besides, if you happen to randomise to a service that is already currently being recorded, setting a short recording timer that is a subset of the existing timer's run window is ignored by the box and then the subsequent short timer deletion/cleanup fails with an error)
  • * no longer applicable
    Handle the box being brought out of standby during a run cycle (it swaps to using timers)
  • You can have basic control of go/pause/stop execution cycles via "flag" files
  • Of course, most importantly, execute this on a frequent basis.
Flag files:
The script will exit if it doesn't detect the presence of its processing flag file named 'AutoVisit.allow', therefore you can just rename it to stop the script.
If the script detects a pause file named 'AutoVisit.pause' then the next execution cycle will be skipped.


Guinea pigs are welcome - give it a burl if you're so inclined. Likely I haven't thought of everything - suggestions are welcome. Bug reports too of course.

For those without home network access to their Wiz, you can copy the shell script to a USB drive and then sneaker-net it to your Wiz and copy onto it.
I suggest locating it in a "local scripts" or "run stuff" directory on the hard drive (the same place you chuck your ipk files for the plugins such as Series2Folder, or ShootYourScreen).
You can use File Commander (Sources / Files) to perform some basic editing on the Wiz to alter the values of the variables in the top dozen lines of the script (unless you have access to an editor on Windows that keeps UNIX-like line endings).
Create an 'AutoVisit.allow' file on your computer and copy it to the Wiz. If you wish you can create a non-matching pause file (e.g. 'AutoVisit.pauseXYZ') that can be later easily renamed on the Wiz if you need to pause the processing.

I envisaged the script would be invoked from the enigma2 pre-start script, so as to automatically start after a boot or UI-restart.
There's a copy of that invocation code attached. For those with an existing pre-start script, you'll obviously need to make an edit to slot in the invocation code.

For those who like to keep their Wiz in deep standby when not in use, you can still achieve semi-regular EPG updating by utilising Power Timers.
Create a few repeated 'wakeup to standby' timers at your required "EPG updating" times. For a T3/T4/U4 you can have that same Power Timer have an end time set for, say, 20 mins after the start time. Set the after event action to be 'go to deep standby' to put the box back to the shutdown state.
Likely you'll need to push back your initial AutoTimer parsing startup delay setting (Autotimer settings: 'Startup delay (in min)') to ensure the initial AutoTimer parsing is started after the provider visiting has finished. A value of 9 mins should be enough. There is an earlier parsing started by the 'Timezones' instance but we can't control the timing of that one.

For a T2 though, since it doesn't have a smart front panel, the Power Timer bootup 'wakeup' and 'wakeup to standby' types are treated the same (a wakeup to full running state) as the T2 doesn't know the specifics of the boot (I don't think it even knows it was for a Power Timer), so it can't actually do the direct-to-standby part of the action. As a work-around, you can set an end time for the Power Timer for 3 mins after the start time and with an after event action of 'go to standby' (don't make the end time any shorter as the UI startup needs to complete before the UI can get around to detecting a "mid-stream" timer and effect that after event action).
Create another 'go to deep standby' Power Timer for your required shutdown time, at the above 20-mins interval.

Don't forget that provider visiting will occur during recordings (propitious conditions extant) so if you've regular daily recording timers set then the task objective is already being performed at those times.

________________________________________________________________________

Changes:
Version 1.1:
Changes made in an attempt to better handle live radio. Still had same issues with the system changing the TV bouquet to Last Scanned.

Version 1.2:
The AutoVisit script has been re-worked and it now uses streaming for both live and standby modes, instead of the previous methods of zapping (when in standby) or recording (live/running).
Because zapping is no longer used, it's also not restricted to using the current TV bouquet (which may not have been your usual bouquet and thus not have had all of your providers, missing out scanning some of them). All providers are now scanned as they're taken from the Terrestrial TV LCN bouquet.
Streaming is a lot gentler on the recording drive too.
There's no longer a need to adjust the number of providers in your broadcast area, as the script works it out.
So, all in all, the new version is a lot simpler and less intrusive :)

Version 1.3:
AutoVisit now detects any tuner use through current live TV/radio streaming, and takes that into account when deciding if there's enough spare tuner capacity for an AutoVisit run.
________________________________________________________________________

The simplest way to prove AutoVisit has updated the EPG is, prior to your first run of AutoVisit, enable IceTV and then disable it as this will clear the EPG of data. Then bring up the EPG and if you're real quick it'll be empty then after a few seconds you'll see it has populated for the current provider/broadcaster, or if you're a bit slower to open the EPG then you'll see that provider's guide data only. If you've got any recordings running then the guide data will be populated for those providers too.
Then let AutoVisit run and check the EPG and you'll find now its full of data :)


If you want to test it out directly without restarting the UI, you can simply invoke the script from the command line (but create the "allow" file first) -
e.g. /media/hdd/Runnables/AutoVisit.sh & (this will run it in the background).
Then you can watch the log file by tailing it -
ls -altr /media/hdd/logs | tail
tail -f /media/hdd/logs/AutoVisit.{date}_{time}.log
Then once you've seen enough, rename the "allow" file to stop it, or find the process to kill it -
ps -ef | grep [A]utoVisit


Attached:
/usr/bin/enigma2_pre_start.sh
AutoVisit.sh
AutoVisit_v1.3.zip
(5.65 KiB) Downloaded 82 times

Cheers,
Geoff

[edit - for new version 1.3]
Last edited by Grumpy_Geoff on Tue Sep 29, 2020 18:30, edited 3 times in total.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Thu May 21, 2020 19:36

Thanks Geoff, AutoVisit is an awesome script. I can't believe it hasn't been downloaded thousands of times.

My only issue is, that I like to listen to radio channels on the T4. And, AutoVisit doesn't work while I'm listening to the radio, unless I'm already recording something on the TV.

How can I set AutoVisit to always use my TV favourite bouquet providers for EPG updates?

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Thu May 21, 2020 19:51

ETA_2016 wrote:
Thu May 21, 2020 19:36
Thanks Geoff, AutoVisit is an awesome script.

I'm glad you find it useful.
ETA_2016 wrote: My only issue is, that I like to listen to radio channels on the T4. And, AutoVisit doesn't work while I'm listening to the radio, unless I'm already recording something on the TV.

How can I set AutoVisit to always use my TV favourite bouquet providers for EPG updates?

It never occured to me that a radio service could be active. I'll try to see what I can do about that.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Thu May 21, 2020 22:09

I should have also said that when AutoVisit runs while using the radio service (only), the log states that it can't detect AutoVisit.allow, which stops it running (it won't run again without a reboot)! AutoVisit.allow is still present.

And, AutoVisit_errs.log shows that line 212 is returning -1.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Thu May 21, 2020 22:47

ETA_2016 wrote:
Thu May 21, 2020 22:09
I should have also said that when AutoVisit runs while using the radio service (only), the log states that it can't detect AutoVisit.allow, which stops it running (it won't run again without a reboot)! AutoVisit.allow is still present.

That's not caused by having a radio service active though, as I get it on occasion too and I don't play a radio service.
You just need to restart the UI (not reboot) when it fails to detect the AutoVisit.allow file.
I'll try a different test on the allow file to see if this error goes away.
ETA_2016 wrote: And, AutoVisit_errs.log shows that line 212 is returning -1.

I get that on occasions too. I think it's failing to grab the current service. Do you have renamed services in your Favourites?

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 13:49

Grumpy_Geoff wrote:
Thu May 21, 2020 22:47
ETA_2016 wrote:
Thu May 21, 2020 22:09
I should have also said that when AutoVisit runs while using the radio service (only), the log states that it can't detect AutoVisit.allow, which stops it running (it won't run again without a reboot)! AutoVisit.allow is still present.

That's not caused by having a radio service active though, as I get it on occasion too and I don't play a radio service.
100% of my AutoVisit.allow file not detected log entries, are while using the radio service.

Since listening to a radio service kills AutoVisit, I don't know whether the same would happen, when I'm using a TV service. All I can say, is that error has never happened while using a TV service (according to the log files).
Grumpy_Geoff wrote: You just need to restart the UI (not reboot) when it fails to detect the AutoVisit.allow file.
You mean Restart GUI, right? (There isn't a command in the menu to Restart UI.)
Grumpy_Geoff wrote:
ETA_2016 wrote: And, AutoVisit_errs.log shows that line 212 is returning -1.

I get that on occasions too. I think it's failing to grab the current service.
For my problems, I believe that error is linked to the AutoVisit.allow file not detected error (when one happens the other does as well).

As for, failing to grab the current service, does displaying the current service in the log negate that possibility?

-----------------
22/05/20 14:51:23
Free tuners: 3
* timer due to fire within our window: 1590123084<=1590123300<=1590123469 (now<=timer-start<=window-end)
* timer due to fire within our window: 1590123084<=1590123300<=1590123469 (now<=timer-start<=window-end)
The box is not in standby, therefore using recording timers
Current TV service: triple j
Number of active service names: 1
Active services: triple j
AutoVisit.allow file not detected
Exiting at: 22/05/20 14:51:24

AutoVisit_errs.log was updated at the same time (to the second) with:

/media/hdd/plugins+skins/AutoVisit.sh: line 212: -1: substring expression < 0

Unfortunately, as the error log doesn't time stamp the entries, there's no way to work out when the errors are occurring (other than the most recent).
Grumpy_Geoff wrote: Do you have renamed services in your Favourites?
I don't think I've renamed any services since I updated the T4 from factory, for the last update.


Seperately, when AutoVisit does run, it's failing to delete, both the latest EPG#Upd8 recordings, and older EPG#Upd8 recordings. It's identifying these files in the log, but not deleting them!

-----------------
24/05/20 13:16:59
Free tuners: 3
The box is not in standby, therefore using recording timers
Current TV service: ABC NEWS
Number of active service names: 1
Active services: ABC NEWS
Service reference: 1:0:19:235:231:1010:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of 231:1010 with pattern 231:1010
- Skipping this serviceref as the box is already chomping on this provider
Service reference: 1:0:19:315:310:3202:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of 310:3202 with pattern 231:1010
- Setting a short timer
Service reference: 1:0:19:431:430:1012:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of 430:1012 with pattern 231:1010
- Setting a short timer
Service reference: 1:0:19:534:503:1013:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of 503:1013 with pattern 231:1010
- Setting a short timer
Service reference: 1:0:19:634:603:1014:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of 603:1014 with pattern 231:1010
- Setting a short timer
Service reference: 1:0:1:E01:E00:3220:EEEE0000:0:0:0:
- Comparing the provider/broadcaster's TSID:ONID of E00:3220 with pattern 231:1010
- Setting a short timer
Finished looping through services to zap/record
Trash listing to delete: 20200524 1321 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200524 1320 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200524 1319 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200524 1318 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200524 1317 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200524 1200 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200524 1159 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200524 1158 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200524 1157 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200522 1256 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200522 1255 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200522 1254 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200522 1253 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200522 1252 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200522 1057 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200522 1056 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200522 1055 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200522 1054 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200522 1053 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200520 2149 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200520 2148 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 2147 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 2146 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200520 2145 - ABC HD - EPG#Upd8.ts
Trash listing to delete: 20200520 2117 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200520 2116 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200520 2115 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 2114 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 2113 - SBS ONE HD - EPG#Upd8.ts
Trash listing to delete: 20200520 2112 - ABC HD - EPG#Upd8.ts
Trash listing to delete: 20200520 1917 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200520 1916 - 10 HD - EPG#Upd8.ts
Trash listing to delete: 20200520 1915 - 7HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 1914 - 9HD Melbourne - EPG#Upd8.ts
Trash listing to delete: 20200520 1913 - SBS ONE HD - EPG#Upd8.ts
Finished cycle: 24/05/20 13:22:10

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 14:20

ETA_2016 wrote:
Sun May 24, 2020 13:49
100% of my AutoVisit.allow file not detected log entries, are while using the radio service.

Since listening to a radio service kills AutoVisit, I don't know whether the same would happen, when I'm using a TV service. All I can say, is that error has never happened while using a TV service (according to the log files).

Hang fire!

If you play the radio service from the Last Scanned bouquet then I think the script wouldn't crap its pants, as it will be able find the radio service name details from it (assuming it was a full scan).

I've fixed this and a few other bugs too.
I'm just preparing it for copying to the forum. This should be within the hour.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 14:29

ETA_2016 wrote:
Sun May 24, 2020 13:49
Seperately, when AutoVisit does run, it's failing to delete, both the latest EPG#Upd8 recordings, and older EPG#Upd8 recordings. It's identifying these files in the log, but not deleting them!

Hmm.
Is that always the case?

They delete for me, example snippet from the debug log -

Code: Select all

{15179}< 19019.107> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts...
{15179}< 19019.107> [eThread] old thread joined 0
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.meta...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.ap...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.sc...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.cuts...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.eit...
{16618}< 19019.109> [setIoPrio] best-effort level 7 ok
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.del'
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.meta.del'
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.ap.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.sc.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.cuts.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.eit.del'

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 15:09

ETA_2016 wrote:
Sun May 24, 2020 13:49
Seperately, when AutoVisit does run, it's failing to delete, both the latest EPG#Upd8 recordings, and older EPG#Upd8 recordings. It's identifying these files in the log, but not deleting them!

...

Finished looping through services to zap/record
Trash listing to delete: 20200524 1321 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200524 1320 - 10 HD - EPG#Upd8.ts

In addition to the above, can you open this link in a browser - http://beyonwizt4/api/movielist?dirname ... ie/.Trash/ please.
Find one of the EPG#Upd8 recordings and paste the '"serviceref"' line (example below) -

Code: Select all

   "serviceref": "1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1301 - SBS ONE HD - EPG#Upd8.ts", 

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 16:28

Grumpy_Geoff wrote:
Sun May 24, 2020 14:20
ETA_2016 wrote:
Sun May 24, 2020 13:49
100% of my AutoVisit.allow file not detected log entries, are while using the radio service.

Since listening to a radio service kills AutoVisit, I don't know whether the same would happen, when I'm using a TV service. All I can say, is that error has never happened while using a TV service (according to the log files).

Hang fire!

If you play the radio service from the Last Scanned bouquet then I think the script wouldn't crap its pants, as it will be able find the radio service name details from it (assuming it was a full scan).

I've fixed this and a few other bugs too.
I'm just preparing it for copying to the forum. This should be within the hour.
I'll try playing it from the Last Scanned bouquet, reboot the GUI and let you know what happens.
Grumpy_Geoff wrote:
Sun May 24, 2020 14:29
ETA_2016 wrote:
Sun May 24, 2020 13:49
Seperately, when AutoVisit does run, it's failing to delete, both the latest EPG#Upd8 recordings, and older EPG#Upd8 recordings. It's identifying these files in the log, but not deleting them!

Hmm.
Is that always the case?

They delete for me, example snippet from the debug log -

Code: Select all

{15179}< 19019.107> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts...
{15179}< 19019.107> [eThread] old thread joined 0
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.meta...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.ap...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.sc...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.cuts...
{15179}< 19019.108> [eDVBPVRServiceOfflineOperations] deleteFromDisk Removing /media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.eit...
{16618}< 19019.109> [setIoPrio] best-effort level 7 ok
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.del'
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.meta.del'
{16618}< 19019.109> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.ap.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.sc.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.ts.cuts.del'
{16618}< 19019.110> [eBackgroundFileEraser] deleting '/media/hdd/movie/.Trash/20200524 1219 - 10 HD - EPG#Upd8.eit.del'
Since 20/05? Yes. But, I have manually deleted the EPG#Upd8 recordings in the past.

But, that didn't occur when I first installed the script, so I wonder if it's anything to do with the latest OS update.

Does the debug log exist by default? If so, where can I find mine?

Grumpy_Geoff wrote:
Sun May 24, 2020 15:09
ETA_2016 wrote:
Sun May 24, 2020 13:49
Seperately, when AutoVisit does run, it's failing to delete, both the latest EPG#Upd8 recordings, and older EPG#Upd8 recordings. It's identifying these files in the log, but not deleting them!

...

Finished looping through services to zap/record
Trash listing to delete: 20200524 1321 - C31 - EPG#Upd8.ts
Trash listing to delete: 20200524 1320 - 10 HD - EPG#Upd8.ts

In addition to the above, can you open this link in a browser - http://beyonwizt4/api/movielist?dirname ... ie/.Trash/ please.
Find one of the EPG#Upd8 recordings and paste the '"serviceref"' line (example below) -

Code: Select all

   "serviceref": "1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1301 - SBS ONE HD - EPG#Upd8.ts", 
Here you go:

Code: Select all

serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1321 - C31 - EPG#Upd8.ts"
serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1320 - 10 HD - EPG#Upd8.ts"
Last edited by ETA_2016 on Sun May 24, 2020 16:35, edited 2 times in total.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 16:59

ETA_2016 wrote:
Sun May 24, 2020 16:28
Grumpy_Geoff wrote:
Sun May 24, 2020 14:20
ETA_2016 wrote:
Sun May 24, 2020 13:49
100% of my AutoVisit.allow file not detected log entries, are while using the radio service.

Since listening to a radio service kills AutoVisit, I don't know whether the same would happen, when I'm using a TV service. All I can say, is that error has never happened while using a TV service (according to the log files).

Hang fire!

If you play the radio service from the Last Scanned bouquet then I think the script wouldn't crap its pants, as it will be able find the radio service name details from it (assuming it was a full scan).

I've fixed this and a few other bugs too.
I'm just preparing it for copying to the forum. This should be within the hour.
I'll try playing it from the Last Scanned bouquet, reboot the GUI and let you know what happens.

Of course you are correct. It didn't have the AutoVisit.allow file not detected error, but once again, failed to delete the EPG#Upd8 recordings.

I haven't changed the radio service names. I just added them to radio favourites bouquet. My radio favourites bouquet has four services, all from the ABC. Could it be that having only one provider in the current bouquet, is causing the error?

Thank you for all your good work.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 17:09

ETA_2016 wrote:
Sun May 24, 2020 16:28
Since 20/05? Yes. But, I have manually deleted the EPG#Upd8 recordings in the past.

If the script gets interrupted then I'd expect them to hang around until next time.
ETA_2016 wrote: But, that didn't occur when I first installed the script, so I wonder if it's anything to do with the latest OS update.

What firmware version are you on? Also the OpenWebif version? You can get both from OWIF -
Box Info -> 'Firmware version:'
and About link at bottom of page
ETA_2016 wrote: Does the debug log exist by default? If so, where can I find mine?

Not by default
Setup>>System>>Log settings and restart the GUI (or in OWIF settings, Log settings, [but restart the GUI twice] if setting it this way]).
ETA_2016 wrote: Here you go:

Code: Select all

serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1321 - C31 - EPG#Upd8.ts"
serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1320 - 10 HD - EPG#Upd8.ts"

Hmm, did you miss the prefixed-" and trailing-": from 'serviceref' or wasn't it there?

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 17:18

ETA_2016 wrote:
Sun May 24, 2020 16:59
Of course you are correct. It didn't have the AutoVisit.allow file not detected error, but once again, failed to delete the EPG#Upd8 recordings.

I haven't changed the radio service names. I just added them to radio favourites bouquet. My radio favourites bouquet has four services, all from the ABC. Could it be that having only one provider in the current bouquet, is causing the error?

The "allow file" non-detection is the end-result of a bug caused by failing to find a service in the current TV mode (usually Favourites) bouquet. Thus when playing a radio service through Last Scanned it finds it as the bouquet contains all services (assuming a full scan). I've changed the method in the new version.

I think I'll change the trash deletion method too.
I'll code and test that now.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 17:47

Out of interest, when this is entered in a browser does it delete the "20200524 1321 - C31 - EPG#Upd8.ts" recording from Trash?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 18:02

Grumpy_Geoff wrote:
Sun May 24, 2020 17:09
ETA_2016 wrote:
Sun May 24, 2020 16:28
Since 20/05? Yes. But, I have manually deleted the EPG#Upd8 recordings in the past.

If the script gets interrupted then I'd expect them to hang around until next time.
Could the script be interrupted, every time, though.
Grumpy_Geoff wrote:
ETA_2016 wrote: But, that didn't occur when I first installed the script, so I wonder if it's anything to do with the latest OS update.
What firmware version are you on? Also the OpenWebif version? You can get both from OWIF -
Box Info -> 'Firmware version:'
and About link at bottom of page
Firmware version: beyonwiz 19.3.20191106 (19.3-273-g6f501ae)
OWIF Version: 1.3.8
Grumpy_Geoff wrote:
ETA_2016 wrote: Here you go:

Code: Select all

serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1321 - C31 - EPG#Upd8.ts"
serviceref	"1:0:0:0:0:0:0:0:0:0:/media/hdd/movie/.Trash/20200524 1320 - 10 HD - EPG#Upd8.ts"

Hmm, did you miss the prefixed-" and trailing-": from 'serviceref' or wasn't it there?
Using Firefox, and they weren't there. The only other thing, was an arrow immediately preceding serviceref, that shortens the directory and name information visible.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 18:16

Grumpy_Geoff wrote:
Sun May 24, 2020 17:47
Out of interest, when this is entered in a browser does it delete the "20200524 1321 - C31 - EPG#Upd8.ts" recording from Trash?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1
I got this result:
message "The movie '20200524 1321 - C31 - EPG' has been deleted successfully"
result true

however, the recording wasn't deleted!

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 18:26

ETA_2016 wrote:
Sun May 24, 2020 18:02
Could the script be interrupted, every time, though.

You're right, it wouldn't.
ETA_2016 wrote: Firmware version: beyonwiz 19.3.20191106 (19.3-273-g6f501ae)
OWIF Version: 1.3.8

I'm on 20200405 beta and OWIF 1.3.9
In any case the deletions have worked for me since Adam was a boy.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 18:29

ETA_2016 wrote:
Sun May 24, 2020 18:16
Grumpy_Geoff wrote:
Sun May 24, 2020 17:47
Out of interest, when this is entered in a browser does it delete the "20200524 1321 - C31 - EPG#Upd8.ts" recording from Trash?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1
I got this result:
message "The movie '20200524 1321 - C31 - EPG' has been deleted successfully"
result true

however, the recording wasn't deleted!

The # strikes again. Works for me, not you :( Notice the '#Upd8' is missing.
Can you try this please?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 18:38

Grumpy_Geoff wrote:
Sun May 24, 2020 18:26
ETA_2016 wrote:
Sun May 24, 2020 18:02
Could the script be interrupted, every time, though.

You're right, it wouldn't.
Actually, I might be wrong.

As AutoVisit was being killed every time I listened to the radio, it was only running when I restarted the GUI or rebooted. Since it was only running at start-up and while I was using the T4, it's possible that it was getting interrupted every time it ran (meaning it was only running once before being killed).

I've left the T4 in the Last Scanned bouquet (though you cant tell, as info shows Favourites (radio)!) I'll see if leaving it in this bouquet fixes the deletion issue, as it will be able to run more than once.
Last edited by ETA_2016 on Sun May 24, 2020 19:02, edited 1 time in total.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 18:45

Grumpy_Geoff wrote:
Sun May 24, 2020 18:29
ETA_2016 wrote:
Sun May 24, 2020 18:16
Grumpy_Geoff wrote:
Sun May 24, 2020 17:47
Out of interest, when this is entered in a browser does it delete the "20200524 1321 - C31 - EPG#Upd8.ts" recording from Trash?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1
I got this result:
message "The movie '20200524 1321 - C31 - EPG' has been deleted successfully"
result true

however, the recording wasn't deleted!

The # strikes again. Works for me, not you :( Notice the '#Upd8' is missing.
No I didn't. And, more concerning is the fact that it was indicating that the file was deleted, when it wasn't.
Grumpy_Geoff wrote: Can you try this please?
http://beyonwizt4/api/moviedelete?sRef= ... ts&force=1
message "The movie 'EPG#Upd8' has been deleted successfully"
result true

and the recording was deleted, though it's got an odd name.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 19:08

ETA_2016 wrote:
Sun May 24, 2020 18:45
message "The movie 'EPG#Upd8' has been deleted successfully"
result true

and the recording was deleted, though it's got an odd name.

I thought so.

Previously I was using the web interface to do the delete of the .ts file and the rest of the recording set were deleted with it, now I'm gonna take a shotgun to it and use the shell to delete 'em :)

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 19:16

Grumpy_Geoff wrote:
Sun May 24, 2020 19:08
ETA_2016 wrote:
Sun May 24, 2020 18:45
message "The movie 'EPG#Upd8' has been deleted successfully"
result true

and the recording was deleted, though it's got an odd name.

I thought so.

Previously I was using the web interface to do the delete of the .ts file and the rest of the recording set were deleted with it, now I'm gonna take a shotgun to it and use the shell to delete 'em :)
Would changing the hash to an underscore resolve any issues?

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 19:28

ETA_2016 wrote:
Sun May 24, 2020 19:16
Would changing the hash to an underscore resolve any issues?

The hash was a difference between Firefox and Chrome I think.
The script code already used '%23' coding to quote the hash. Somehow your system wasn't liking it, perhaps, possibly, maybe, a difference in your MIPS (T4) and my ARM (U4) architecture.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sun May 24, 2020 19:38

Here's the changes for the new version.
It now handles both live and recording radio services, and also a TV service not within your active TV bouquet (essentially the same bug).
I've changed the trash recording deletion method so you shouldn't see any of the EPG recordings being left behind.
I fixed another bug that'd cause a cycle skip if (i) there was only one free tuner left and (ii) there was a timer due to fire within the run window and (iii) the timer was disabled (as previously it never checked the timer status).
I've also shortened the default scan period to 30 seconds.
I've also fixed the pre-start script snippet to place the killing of any prior script instance closer to the new invocation. Previously you could have two instances running if you restarted the GUI within the 3-minute wait period.

I've placed the updated versions in a ("v1.1") zip file in the first post.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Sun May 24, 2020 20:32

Grumpy_Geoff wrote:
Sun May 24, 2020 19:38
It now handles both live and recording radio services, and also a TV service not within your active TV bouquet (essentially the same bug).
:D :D :D
Grumpy_Geoff wrote: I've changed the trash recording deletion method so you shouldn't see any of the EPG recordings being left behind.
:D
Grumpy_Geoff wrote: I've also fixed the pre-start script snippet to place the killing of any prior script instance closer to the new invocation. Previously you could have two instances running if you restarted the GUI within the 3-minute wait period.
Bizarrely, the first time I encountered this was when I restarted the GUI, after copying the new version to the T4. I managed to kill the old version (with your help).

Thanks for all your work, & time spent, on this update, Geoff.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Mon May 25, 2020 02:55

Grumpy_Geoff wrote:
Sun May 24, 2020 19:38
Here's the changes for the new version...
Sorry Geoff, I've found a mistake in AutoVisit.sh. :wink:

Line 65 (comment) refers to a 60-second visit. That's the version 1.0 AutoVisit.sh visiting time!

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Mon May 25, 2020 09:30

Grumpy_Geoff wrote:
Sun May 24, 2020 19:28
ETA_2016 wrote:
Sun May 24, 2020 19:16
Would changing the hash to an underscore resolve any issues?

The hash was a difference between Firefox and Chrome I think.
The script code already used '%23' coding to quote the hash. Somehow your system wasn't liking it, perhaps, possibly, maybe, a difference in your MIPS (T4) and my ARM (U4) architecture.

Okay, I now realise why your trash deletes weren't working and mine were - it would've stopped deleting once you updated to (19.3) 20191106.
Back when I updated to the first public 19.3-series beta in July last year I found a difference in the way the '#' was handled when performing the deletes, and adoxa pointed out to me it was likely due to a new version of the 'wget' command in the system. I then modified the code to use the percent-encoding for the hash sign and the deletes started working again. I forgot to publish that new code :(

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Mon May 25, 2020 09:42

ETA_2016 wrote:
Mon May 25, 2020 02:55
Sorry Geoff, I've found a mistake in AutoVisit.sh. :wink:

Line 65 (comment) refers to a 60-second visit. That's the version 1.0 AutoVisit.sh visiting time!

That's line 75 :)
Yes true, the comment would probably be better stated as "example, for 6x60 we will need..." and "for 5x30 we will need...".
We now only have 5 providers here in Perth, as the community broadcaster went belly up.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Mon May 25, 2020 17:06

Grumpy_Geoff wrote:
Mon May 25, 2020 09:42
ETA_2016 wrote:
Mon May 25, 2020 02:55
Sorry Geoff, I've found a mistake in AutoVisit.sh. :wink:

Line 65 (comment) refers to a 60-second visit. That's the version 1.0 AutoVisit.sh visiting time!

That's line 75 :)
Yes true, the comment would probably be better stated as "example, for 6x60 we will need..." and "for 5x30 we will need...".
We now only have 5 providers here in Perth, as the community broadcaster went belly up.
Thanks for debugging my post :lol:

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Thu May 28, 2020 04:29

Geoff, for some reason, AutoVisit is using openshop (channel 7) for recordings & zapping, whereas all the other providers are the primary HD channels (except for C31).

Also, when zapping, it changes the current service from triple J to openshop. That's very annoying.

I've now deleted all the shopping channels, & the racing channel, from the Last Scanned bouquet (the only place these channels existed). Hopefully doing that will resolve these issues.

However, it would be good if you could update AutoVisit (when convenient), to prevent these issues.

Is AutoVisit using the Last Scanned bouquet to update the EPG & what would happen if only a single providers channels were in the Last Scanned bouquet?
Last edited by ETA_2016 on Thu May 28, 2020 19:31, edited 1 time in total.

prl
Wizard God
Posts: 32697
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Re: Provider visiting - keeping the FTA EPG current

Post by prl » Thu May 28, 2020 11:03

ETA_2016 wrote:
Thu May 28, 2020 04:29
Geoff, for some reason, AutoVisit is using openshop (channel 7) for recordings & zapping, whereas all the other providers are the primary HD channels (except for C31).

Do you care which service it zaps to?
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Thu May 28, 2020 11:40

ETA_2016 wrote:
Thu May 28, 2020 04:29
Geoff, for some reason, AutoVisit is using openshop (channel 7) for recordings & zapping, whereas all the other providers are the primary HD channels (except for C31).

Also, when zapping, it changes the current service from triple J to openshop. That's very annoying.

Have you been playing triple j through the Last Scanned bouquet or Terrestrial RADIO LCN or your Favourites (Radio)?

When it starts, AutoVisit should grab the last used TV bouquet and last used service (config.tv.lastroot and config.tv.lastservice). It then randomly picks a (non-radio) service from each provider from that bouquet.
If the box is in standby, then AutoVisit zaps through those services. Then it zaps back to the last used service that was remembered when it started.
If the box is in use, then the short timers are set.
Therefore it shouldn't be the case that openshop is chosen if that current bouquet was your Favourites (TV) bouquet.
What do you have for the "last" settings config.tv.lastroot, config.tv.lastservice, config.radio.lastroot and config.radio.lastservice settings from -
http://beyonwizt4/api/settings
ETA_2016 wrote:
Thu May 28, 2020 04:29
Is AutoVisit using the Last Scanned bouquet to update the EPG & what would happen if only a single providers channels were in the Last Scanned bouquet?

It isn't set to use Last Scanned, but if that was your last TV mode bouquet then it will. Same if you had a single-provider bouquet set up and it was used as the last bouquet.

This doesn't mean to say I didn't cock something up though. We just need to find the scenario if I did.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Thu May 28, 2020 18:16

Yes, strange behaviour observed.
I had the TV bouquet as Favourites (TV) and last service played was 7HD. I swapped to radio mode and Favourites (Radio) bouquet and zapped to Double J. Then I went to standby.
I ran AutoVisit, and it zapped through its "random" services. Then AutoVisit zapped back to 7HD, and then to close the tuner used for the zap, it comes out of standby and then goes back into standby.

After this, the settings shows the TV bouquet had auto swapped to the Last Scanned bouquet! :o I bring the box out of standby and the infobar still reports Favourites (Radio), but the settings for config.tv.lastroot shows the value of 'userbouquet.LastScanned.tv'.
It's as if when coming out of standby, to play that last service (which was radio), it does a search through all of the bouquets listed in 'bouquets.tv' looking for the first bouquet that has that last service. It finds Last Scanned qualifies.
It probably should be looking through bouquets.radio. :?:

If I modify AutoVisit to not go out of and back into standby, then the bouquet change doesn't happen between runs.
But what does appear to happen is the value of config.tv.lastservice changes to be that of the first service for that provider from Favourites (TV), not the last zap that was to 7HD.
It changed to ABC HD, when the last service was Double J (even though that was in radio mode).

This all seems to happen if the last service played was a radio service.
It doesn't happen if a TV service was last played.

More investigation needed.
I may well have to dump standby/background zapping. Ideally if I could detect that radio was the last service before standby, then I'd just swap to setting timers in that case, and leave zapping for when TV was the last mode before standby. The problem is I don't know how to detect that in the settings. I don't know if there's an OWIF command for it neither.
The UI keeps a record of similar, because you can use BACK (or 0) to swap between a radio service and a TV service and the correct bouquet is shown for that service.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Thu May 28, 2020 19:48

prl wrote:
Thu May 28, 2020 11:03
ETA_2016 wrote:
Thu May 28, 2020 04:29
Geoff, for some reason, AutoVisit is using openshop (channel 7) for recordings & zapping, whereas all the other providers are the primary HD channels (except for C31).

Do you care which service it zaps to?
No. I just thought it was odd.
Grumpy_Geoff wrote:
Thu May 28, 2020 11:40
ETA_2016 wrote:
Thu May 28, 2020 04:29
Geoff, for some reason, AutoVisit is using openshop (channel 7) for recordings & zapping, whereas all the other providers are the primary HD channels (except for C31).

Also, when zapping, it changes the current service from triple J to openshop. That's very annoying.

Have you been playing triple j through the Last Scanned bouquet or Terrestrial RADIO LCN or your Favourites (Radio)?
Favourites (Radio), since the update.
Grumpy_Geoff wrote: When it starts, AutoVisit should grab the last used TV bouquet and last used service (config.tv.lastroot and config.tv.lastservice). It then randomly picks a (non-radio) service from each provider from that bouquet.
If the box is in standby, then AutoVisit zaps through those services. Then it zaps back to the last used service that was remembered when it started.
If the box is in use, then the short timers are set.
Therefore it shouldn't be the case that openshop is chosen if that current bouquet was your Favourites (TV) bouquet.
What do you have for the "last" settings config.tv.lastroot, config.tv.lastservice, config.radio.lastroot and config.radio.lastservice settings from -
http://beyonwizt4/api/settings
"config.tv.lastroot"
"1:7:1:0:0:0:0:0:0:0:FROM BOUQUET \"bouquets.tv\" ORDER BY bouquet;1:7:1:0:0:0:0:0:0:0:FROM BOUQUET \"userbouquet.LastScanned.tv\" ORDER BY bouquet;"

"config.tv.lastservice"
"1:0:2:239:231:1010:EEEE0000:0:0:0:" (triplej)

"config.radio.lastroot"
"1:7:2:0:0:0:0:0:0:0:FROM BOUQUET \"bouquets.radio\" ORDER BY bouquet;1:7:2:0:0:0:0:0:0:0:FROM BOUQUET \"userbouquet.favourites.radio\" ORDER BY bouquet;"

"config.radio.lastservice"
"1:0:2:239:231:1010:EEEE0000:0:0:0:" (triplej)

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Thu May 28, 2020 20:10

After I deleted all the bloody annoying shopping channels, and the cruel sport of horse racing channel, entirely from my T4, the AutoVisit zapping EPG Update runs returned to the following channels:

06:05 ABC News

08:05 triplej

10:05 triplej

12:05 triplej

At some point between 4 AM and now AutoVisit had changed the TV bouquet from Favourites to Last Scanned too.

Would the debug log show when that happened?

Also, deleting now works, and AutoVisit doesn't kill itself when using a service from my Favourites (Radio) bouquet, so it's working continuously now, which is great. The log file is a whopping 68KB! Thanks again, for all the improvements, Geoff.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Thu May 28, 2020 21:01

ETA_2016 wrote:
Thu May 28, 2020 20:10
At some point between 4 AM and now AutoVisit had changed the TV bouquet from Favourites to Last Scanned too.

Would the debug log show when that happened?

Try this for me -
Before putting the box into standby, if you're currently in a radio service swap to a TV service first, then put the box into standby.
After you know that AutoVisit has run, bring the box out of standby and see if both the bouquet and service are as prior to the standby.

I've seen some strange behaviour at times where toggling (BACK) between services will give rise to the bouquet name disappearing from the infobar, and when bringing up the channel list (FAV) it will then show Last Scanned as the bouquet (as if the system has lost the plot between TV/Radio modes).
This only seems to happen if one of the services is a radio service.

The debug log reveals nowt.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 03:06

ETA_2016 wrote:
Thu May 28, 2020 20:10
After I deleted all the bloody annoying shopping channels, and the cruel sport of horse racing channel, entirely from my T4, the AutoVisit zapping EPG Update runs returned to the following channels:

06:05 ABC News

08:05 triplej

10:05 triplej

12:05 triplej
Whoops. I should have said my AutoVisit only runs once every 2 hours, and I didn't use the T4 during the above times. So, AutoVisit has changed the current service to ABC News, and then changed it back to triple j, where it remained. Curious!
Grumpy_Geoff wrote:
Thu May 28, 2020 21:01
ETA_2016 wrote:
Thu May 28, 2020 20:10
At some point between 4 AM and now AutoVisit had changed the TV bouquet from Favourites to Last Scanned too.

Would the debug log show when that happened?

Try this for me -
Before putting the box into standby, if you're currently in a radio service swap to a TV service first, then put the box into standby.
After you know that AutoVisit has run, bring the box out of standby and see if both the bouquet and service are as prior to the standby.
Done. I'll wait 12 hours or so, to get a good sample size (that's only 7 runs), and report the results.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 16:33

Grumpy_Geoff wrote:
Thu May 28, 2020 21:01
ETA_2016 wrote:
Thu May 28, 2020 20:10
At some point between 4 AM and now AutoVisit had changed the TV bouquet from Favourites to Last Scanned too.

Would the debug log show when that happened?

Try this for me -
Before putting the box into standby, if you're currently in a radio service swap to a TV service first, then put the box into standby.
After you know that AutoVisit has run, bring the box out of standby and see if both the bouquet and service are as prior to the standby.
Yes, the service was not changed when zapping (8 times), nor was the bouquet.

Regarding the bouquet, how do I confirm which one is in use? Do I press TV or FAV, and see which bouquet is highlighted, or opened?

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Fri May 29, 2020 16:51

ETA_2016 wrote:
Fri May 29, 2020 16:33
Yes, the service was not changed when zapping (8 times), nor was the bouquet.

Thought so.
ETA_2016 wrote: Regarding the bouquet, how do I confirm which one is in use? Do I press TV or FAV, and see which bouquet is highlighted, or opened?

FAV to bring up the channel (service) list in the "current" bouquet.

I had an interesting thing happen when my U4 booted this morning, as it started on a radio service. That's lead me to trying some new stuff.
I'll be back with a new version for you to test with radio mode active before standby.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 17:05

Grumpy_Geoff wrote:
Fri May 29, 2020 16:51
ETA_2016 wrote:
Fri May 29, 2020 16:33
Yes, the service was not changed when zapping (8 times), nor was the bouquet.

Thought so.
ETA_2016 wrote: Regarding the bouquet, how do I confirm which one is in use? Do I press TV or FAV, and see which bouquet is highlighted, or opened?

FAV to bring up the channel (service) list in the "current" bouquet.

I had an interesting thing happen when my U4 booted this morning, as it started on a radio service. That's lead me to trying some new stuff.
I'll be back with a new version for you to test with radio mode active before standby.
Thanks.

As, you can see from my post above, it appears that AutoVisit is no longer changing the current service when zapping. I'm going to do some more zapping testing, with a radio service active.
Last edited by ETA_2016 on Fri May 29, 2020 17:08, edited 1 time in total.

User avatar
adoxa
Wizard
Posts: 1490
Joined: Thu Feb 23, 2017 22:58
Location: CQ
Contact:

Re: Provider visiting - keeping the FTA EPG current

Post by adoxa » Fri May 29, 2020 17:05

ETA_2016 wrote:
Fri May 29, 2020 16:33
Regarding the bouquet, how do I confirm which one is in use? Do I press TV or FAV, and see which bouquet is highlighted, or opened?
It's at the bottom-left of the infobar (OverlayHD always had it, I added it for the default skins).

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 17:13

adoxa wrote:
Fri May 29, 2020 17:05
ETA_2016 wrote:
Fri May 29, 2020 16:33
Regarding the bouquet, how do I confirm which one is in use? Do I press TV or FAV, and see which bouquet is highlighted, or opened?
It's at the bottom-left of the infobar (OverlayHD always had it, I added it for the default skins).
Thanks, but had you read the topic, you would have realised that, under certain conditions, the bouquet can change, without the bouquet shown in the infobar changing. The only change visible, would be the channel number, if it was different in the two bouquets. (And, I'm using OverlayHD.)

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Fri May 29, 2020 17:19

ETA_2016 wrote:
Fri May 29, 2020 17:05
I'm going to do some more zapping testing, with a radio service active.

It won't be a happy outcome :P
Unless, possibly, you delete the Last Scanned bouquet.

I've been testing a new AutoVisit version that detects the radio mode prior to standby, thus zapping back to the radio service after it's finished. It seems to work okay. Some more testing to do.

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 17:29

Grumpy_Geoff wrote:
Fri May 29, 2020 17:19
ETA_2016 wrote:
Fri May 29, 2020 17:05
I'm going to do some more zapping testing, with a radio service active.

It won't be a happy outcome :P
Well, I'm going to leave triplej in my Favourites (radio) bouquet active, let AutoVisit log the results, and whatever happens, happens. But, there is no chance whatsoever, that one of those bloody annoying shopping channels, or the cruel sport of horse racing channel, will be on the screen when I take the T4 out of standby. I can live with that for a little while.

And, who knows; the results could be interesting.

User avatar
adoxa
Wizard
Posts: 1490
Joined: Thu Feb 23, 2017 22:58
Location: CQ
Contact:

Re: Provider visiting - keeping the FTA EPG current

Post by adoxa » Fri May 29, 2020 17:30

ETA_2016 wrote:
Fri May 29, 2020 17:13
Thanks, but had you read the topic, you would have realised that, under certain conditions, the bouquet can change, without the bouquet shown in the infobar changing.
I had the read topic, but not close enough - "though you cant tell, as info shows Favourites (radio)". :oops:

ETA_2016
Apprentice
Posts: 64
Joined: Thu Aug 27, 2015 04:42
Location: Melbourne

Re: Provider visiting - keeping the FTA EPG current

Post by ETA_2016 » Fri May 29, 2020 17:41

adoxa wrote:
Fri May 29, 2020 17:30
ETA_2016 wrote:
Fri May 29, 2020 17:13
Thanks, but had you read the topic, you would have realised that, under certain conditions, the bouquet can change, without the bouquet shown in the infobar changing.
I had the read topic, but not close enough - "though you cant tell, as info shows Favourites (radio)". :oops:
Okay. Sorry, I should have said "had you read every single line in the topic". I had forgotten it was only mentioned once, with little detail. Thanks, for trying to help.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Fri May 29, 2020 17:43

adoxa wrote:
Fri May 29, 2020 17:30
ETA_2016 wrote:
Fri May 29, 2020 17:13
Thanks, but had you read the topic, you would have realised that, under certain conditions, the bouquet can change, without the bouquet shown in the infobar changing.
I had the read topic, but not close enough - "though you cant tell, as info shows Favourites (radio)". :oops:

plus these -
Grumpy_Geoff wrote:
Thu May 28, 2020 21:01
I've seen some strange behaviour at times where toggling (BACK) between services will give rise to the bouquet name disappearing from the infobar, and when bringing up the channel list (FAV) it will then show Last Scanned as the bouquet (as if the system has lost the plot between TV/Radio modes).
This only seems to happen if one of the services is a radio service.
Grumpy_Geoff wrote:
Thu May 28, 2020 18:16
After this, the settings shows the TV bouquet had auto swapped to the Last Scanned bouquet! :o I bring the box out of standby and the infobar still reports Favourites (Radio), but the settings for config.tv.lastroot shows the value of 'userbouquet.LastScanned.tv'.
It's as if when coming out of standby, to play that last service (which was radio), it does a search through all of the bouquets listed in 'bouquets.tv' looking for the first bouquet that has that last service. It finds Last Scanned qualifies.
It probably should be looking through bouquets.radio. :?:

The weird stuff seems to happen if the last service played was a radio service.
It doesn't happen if a TV service was last played.

Then again, I think the system gets a bit confused if you use BACK (or 0) to toggle between a radio service and a TV service. I think it loses track of config.servicelist.lastmode=radio.

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Fri May 29, 2020 22:32

ETA_2016 wrote:
Fri May 29, 2020 17:29
Well, I'm going to leave triplej in my Favourites (radio) bouquet active, let AutoVisit log the results, and whatever happens, happens. But, there is no chance whatsoever, that one of those bloody annoying shopping channels, or the cruel sport of horse racing channel, will be on the screen when I take the T4 out of standby. I can live with that for a little while.

And, who knows; the results could be interesting.

Shed loads of testing/debugging has shown -
If in standby from radio mode, after an AutoVisit run:
Swaps TV bouquet to Last Scanned and drops out of radio mode if Last Scanned bouquet exists in the system (or likely more precisely it has the radio service).
Keeps existing TV bouquet and stays in radio mode if Last Scanned bouquet doesn't exist.

I removed all of the radio services from Last Scanned (it's very quick with OWIF, ctrl-click on each to select 'em all, then delete button), then with them removed, after AutoVisit zaps back to the last-used radio service, the radio bouquet is swapped to Terrestrial LCN RADIO (I'm guessing as it's the first in the list with that service).

If I re-order the radio bouquets to be Favourites first then Terrestrial LCN RADIO then --
Bingo :)

Code: Select all

Finished looping through services to process
Muting & coming out of standby
Zapping back to original TV service: 1:0:19:564:506:1013:EEEE0000:0:0:0:
Zapping back to original radio service: 1:0:2:2E6:261:3201:EEEE0000:0:0:0:
Going back into standby & unmuting
* settings now -
"config.radio.lastroot",
"1:7:2:0:0:0:0:0:0:0:FROM BOUQUET "bouquets.radio" ORDER BY bouquet;1:7:2:0:0:0:0:0:0:0:FROM BOUQUET "userbouquet.favourites.radio" ORDER BY bouquet;"
"config.radio.lastservice",
"1:0:2:2E6:261:3201:EEEE0000:0:0:0:"
"config.tv.lastroot",
"1:7:1:0:0:0:0:0:0:0:FROM BOUQUET "bouquets.tv" ORDER BY bouquet;1:7:1:0:0:0:0:0:0:0:FROM BOUQUET "userbouquet.favourites.tv" ORDER BY bouquet;"
"config.tv.lastservice",
"1:0:19:564:506:1013:EEEE0000:0:0:0:"
"config.servicelist.lastmode",
"radio
In other words, it stays in radio mode and doesn't change either of the two radio or TV bouquets.

The other option I can think of is to use timers when the box is in standby from the radio mode (keeping zapping when in TV mode).

So, what do you think?
You edit the radio bouquets and put your Favourites first.
You also edit your Last Scanned and remove the radio services, or delete the bouquet all together. Remember it'll be re-populated or re-created when you scan again.
Or I use timers in standby after being in radio mode?

Grumpy_Geoff
Uber Wizard
Posts: 6490
Joined: Thu Mar 05, 2009 22:54
Location: Perth

Re: Provider visiting - keeping the FTA EPG current

Post by Grumpy_Geoff » Sat May 30, 2020 15:49

Instead of using timers or zapping, I always wanted to stream each service as that is what I think EPGRefresh uses for its fake recording option, but I couldn't figure out a way to do it.
After trying again today with many different ways and failing, something hit me in the face and I had an "oh, of course moment" :idea: :oops:

So, I think I've got a new plan :)
There'll be no need for timers nor zapping, therefore no disturbance of bouquets nor services. No need to worry about radio mode.
Just streaming (to the bitbucket). Rudimentary testing shows the EPG is updated just fine.

Hopefully a version out in a day or two.

Post Reply

Return to “Hacks & Tricks”